Skip to main content
Solved

How to perform a 'for each' loop in FME?

  • December 24, 2020
  • 4 replies
  • 2210 views
F

Dear all,

 

I use FME to perform a complex comparison analysis between two large datasets. I am looking for a way to analyze various subpopulations in succession, in order to reduce memory usage and increase performance. I have defined the desired subpopulations in a Reader file, so they can feed the SQLExecutor transformers to obtain the subsets. After performing the analysis on the first subset (i.e. 'Apple') and writing the results to a report file, the next subset (i.e. 'Banana') should be analyzed and so on.

 

fme 

How can I implement such a 'for each loop' within FME?

 

Thanks in advance.

 

Remy

Best answer by ebygomm

I'd personally use a parent workspace with a workspace runner to run the workspace for each line in your file, you should be able to use send Source_1 and Source_2 as parameters in the child workspace

View original
Did this help you find an answer to your question?

4 replies

hkingsbury
Celebrity
Forum|alt.badge.img+55
  • hkingsburyCelebrity
  • Celebrity
  • December 24, 2020

You're going to want to look at group based custom transformers

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Group_Based_Transformers.htm

https://community.safe.com/s/question/0D54Q0000849jVa/how-use-group-by-on-custom-transformer

 

And looping in custom transformers

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/transformers_custom_looping.htm

ebygomm
Influencer
Forum|alt.badge.img+39
  • ebygommInfluencer
  • Influencer
  • Best Answer
  • December 29, 2020

I'd personally use a parent workspace with a workspace runner to run the workspace for each line in your file, you should be able to use send Source_1 and Source_2 as parameters in the child workspace

bjudes
1 person upvoted this
  • bjudes
    bjudes
J
Forum|alt.badge.img+2

Workspace runner or a batch file will be your best bet.

 

If you are running something in a specific order and one at a time the runner works well.

 

If you running a comparison between the first two, comparing the result of that run to the next, etc. A batch file with a series of calls/start would let you spawn all the instances at once as independent processes. You can run up to 8 processes at a time.

 

F
  • fentanyl
  • Author
  • December 31, 2020
ebygomm wrote:

I'd personally use a parent workspace with a workspace runner to run the workspace for each line in your file, you should be able to use send Source_1 and Source_2 as parameters in the child workspace

Exactly what I was looking for!

 

Firstly, I added two Published User Parameters to the initial (child) workspace:

01 

After which I changed both SQLExecutors to SQLCreators that use the published parameters as SQL Statement.02At last, I created a parent workspace that affects the published parameters.

03 

Thanks a lot!

Reply


Helpful Members This Week

Community Stats

31,501
Posts
119,418
Replies
39,119
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.