Skip to main content
Solved

How to perform a 'for each' loop in FME?

  • December 24, 2020
  • 4 replies
  • 2500 views
F

Dear all,

 

I use FME to perform a complex comparison analysis between two large datasets. I am looking for a way to analyze various subpopulations in succession, in order to reduce memory usage and increase performance. I have defined the desired subpopulations in a Reader file, so they can feed the SQLExecutor transformers to obtain the subsets. After performing the analysis on the first subset (i.e. 'Apple') and writing the results to a report file, the next subset (i.e. 'Banana') should be analyzed and so on.

 

fme 

How can I implement such a 'for each loop' within FME?

 

Thanks in advance.

 

Remy

Best answer by ebygomm

I'd personally use a parent workspace with a workspace runner to run the workspace for each line in your file, you should be able to use send Source_1 and Source_2 as parameters in the child workspace

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

4 replies

hkingsbury
Celebrity
Forum|alt.badge.img+63
  • hkingsburyCelebrity
  • Celebrity
  • 1625 replies
  • December 24, 2020

You're going to want to look at group based custom transformers

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Transformers/Group_Based_Transformers.htm

https://community.safe.com/s/question/0D54Q0000849jVa/how-use-group-by-on-custom-transformer

 

And looping in custom transformers

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/transformers_custom_looping.htm

ebygomm
Influencer
Forum|alt.badge.img+44
  • ebygommInfluencer
  • Influencer
  • 3427 replies
  • Best Answer
  • December 29, 2020

I'd personally use a parent workspace with a workspace runner to run the workspace for each line in your file, you should be able to use send Source_1 and Source_2 as parameters in the child workspace

J
Forum|alt.badge.img+2
  • jlbaker2779
  • 194 replies
  • December 29, 2020

Workspace runner or a batch file will be your best bet.

 

If you are running something in a specific order and one at a time the runner works well.

 

If you running a comparison between the first two, comparing the result of that run to the next, etc. A batch file with a series of calls/start would let you spawn all the instances at once as independent processes. You can run up to 8 processes at a time.

 

F
  • fentanyl
  • Author
  • 3 replies
  • December 31, 2020

I'd personally use a parent workspace with a workspace runner to run the workspace for each line in your file, you should be able to use send Source_1 and Source_2 as parameters in the child workspace

Exactly what I was looking for!

 

Firstly, I added two Published User Parameters to the initial (child) workspace:

01 

After which I changed both SQLExecutors to SQLCreators that use the published parameters as SQL Statement.02At last, I created a parent workspace that affects the published parameters.

03 

Thanks a lot!

Community Stats

32,070
Posts
121,778
Replies
39,698
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.