Skip to main content
Question

Converting FME transformer functioning into Python code

  • September 28, 2020
  • 8 replies
  • 568 views

Forum|alt.badge.img

Hi,

I am new to FME ..but now at least I know how to use different transformers in FME and to create a workbench and export the related outcome into a CSV or other file formats I want to use for further analysis.

Now, I am keen to learn the automation process...i.e. instead of using the similar transformation for so many times... I am looking for the Python code for the same...which I can convert to make process automated when I have new data set to work with.

Any help on this will be really appreciated.

8 replies

david_r
Evangelist
  • September 28, 2020

Have you looked into creating custom transformers for repeating workflows?

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/custom_transformer_creating.htm

I'm not sure Python is the way to go in this use case, unless I'm missing something.


virtualcitymatt
Celebrity
Forum|alt.badge.img+34

Yeah - it's not like ESRI tools. There is no python equivalent for each tool. You would need to use FME's python API to build your own process http://docs.safe.com/fme/html/fmepython/ and I wouldn't recommend that to anyone. Python in FME should only really be used if FME doesn't have a transformer which does what you want.

 

Usually the best option for automation is FME Server (or FME Cloud). Trigger events can be set up to kick of a fme workspace.

 

FME can also be called from the command line if you want to include a workspace into you own custom pipeline or run a workspace in batch with a number of different input files. https://community.safe.com/s/article/fme-batch-processing-methods-1

 

FME can also be used to handle different incoming schemas or even varying input formats. Understanding the dynamic workflows concept in FME is a real help with automation. https://community.safe.com/s/article/dynamic-workflow-tutorial-introduction

 

 

 


Forum|alt.badge.img
  • Author
  • September 30, 2020
david_r wrote:

Have you looked into creating custom transformers for repeating workflows?

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/custom_transformer_creating.htm

I'm not sure Python is the way to go in this use case, unless I'm missing something.

Hi @david_r​,

This creating custom transformer is working well..however, it doen't serve my objective.

Well, my intension is to get the Python or C++ code for each individual transformer I use in FME...and when I want to modify it for other dataset...I don't have to rely on FME.

In short, for more degree of freedom...it's great if I can use the programming software (instead of just using FME capability) so that we can integrate other programmes into one system if needed.


Forum|alt.badge.img
  • Author
  • September 30, 2020
virtualcitymatt wrote:

Yeah - it's not like ESRI tools. There is no python equivalent for each tool. You would need to use FME's python API to build your own process http://docs.safe.com/fme/html/fmepython/ and I wouldn't recommend that to anyone. Python in FME should only really be used if FME doesn't have a transformer which does what you want.

 

Usually the best option for automation is FME Server (or FME Cloud). Trigger events can be set up to kick of a fme workspace.

 

FME can also be called from the command line if you want to include a workspace into you own custom pipeline or run a workspace in batch with a number of different input files. https://community.safe.com/s/article/fme-batch-processing-methods-1

 

FME can also be used to handle different incoming schemas or even varying input formats. Understanding the dynamic workflows concept in FME is a real help with automation. https://community.safe.com/s/article/dynamic-workflow-tutorial-introduction

 

 

 

Hi @virtualcitymatt​ ,

Can you explain a bit more abou FME server/FME cloud and how it could for automation process in FME or getting the Python/C++ code of each of transformers in use?

 


virtualcitymatt
Celebrity
Forum|alt.badge.img+34
ppp19 wrote:

Hi @david_r​,

This creating custom transformer is working well..however, it doen't serve my objective.

Well, my intension is to get the Python or C++ code for each individual transformer I use in FME...and when I want to modify it for other dataset...I don't have to rely on FME.

In short, for more degree of freedom...it's great if I can use the programming software (instead of just using FME capability) so that we can integrate other programmes into one system if needed.

Wouldn't that be nice. Unfortunately the code under FME and its transformers is not open source so you will be unable to get access to it. All of the cpp files and python scripts used are encrypted.


virtualcitymatt
Celebrity
Forum|alt.badge.img+34
ppp19 wrote:

Hi @virtualcitymatt​ ,

Can you explain a bit more abou FME server/FME cloud and how it could for automation process in FME or getting the Python/C++ code of each of transformers in use?

 

FME Server can be configured to run a job based on several triggers.

 

For example when a file is added to a folder, or from a database trigger. You can read a little more about it here. https://community.safe.com/s/article/getting-started-with-automations

 

What specifically are you wanting to change to suit your process? You will not be able to run FME processes without being dependant on FME and it's license.


david_r
Evangelist
  • October 1, 2020
ppp19 wrote:

Hi @david_r​,

This creating custom transformer is working well..however, it doen't serve my objective.

Well, my intension is to get the Python or C++ code for each individual transformer I use in FME...and when I want to modify it for other dataset...I don't have to rely on FME.

In short, for more degree of freedom...it's great if I can use the programming software (instead of just using FME capability) so that we can integrate other programmes into one system if needed.

The closest you'll get would be to use the programming API, available for several programming languages. See in your installation folder, in the fmeobjects subdirectory for documentation and samples.

Note that not all APIs are quite equal, e.g. the Python API lacks some functionality that is available in the C/Java API.


f.kemminje
Contributor
Forum|alt.badge.img+11
  • Contributor
  • October 2, 2020
  1. You can Use parent script and child script method :

Here fmeserverjobsubmitter / workspace runner transformer will call its child script , if it is a repeated job. You can change the parameters on fly.

2.You can use fme server's scheduler function, to schedule a job with different parameters

3.You can use fme topcs, so that fme server will call another fmw file , when you submit a topic to fme server.

here topics can be a text string and you can parse the texts and use as input parameters for fmw file

 

there are many....

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings