Solved

Workspace Runner triggers too many jobs.

rmcgregor

I have a unique value logger attached to a table and a workspace runner. It took me a while to realize it, but what it does is goes through each unique value and runs the entire 'child' fme once for each unique value, but it runs all unique values each time it runs .

 

I put a test filter on the input table in the workspace runner workspace so that it only has one unique value to 'give' the runner and that solved the problem.

This seems very strange but it works. Does anyone know why this works like this and if there's a less 'hacky' way?

Thank you,

Randy McGregor

 

icon

Best answer by hkingsbury 13 July 2021, 22:17

View original
1 Attachment

11 replies

rmcgregor

From the image, you can see that only one unique value outputted (thanks to the filter) and that there is only one successful run indicated in the runner. There are four unique values and all four do in fact run. The 'child' fme seems to just know to do this even though it's only receiving a single unique value.

rmcgregor

I tried eliminating the unique value logger, but then the child fme doesn't create unique outputs for unique values. It appears as though the mere existence of the unique value logger triggers the child fme to run a unique process for each unique value even if they are not all in the input to the runner.

hkingsbury
Rank
Userlevel 5
Badge +29

That is the expected behavior - each feature will execute the workbench. Think of the features as an instruction, each one is telling FME to run a process irrespective of its attributes.

 

Another transformer to look at, instead of the UniqueValueLogger, would be either the DuplicateFilter, or Sampler (with it set to First N Features)

Check out the Abley FME Firesides on Youtube: https://www.youtube.com/@ableyltd
rmcgregor

That is the expected behavior - each feature will execute the workbench. Think of the features as an instruction, each one is telling FME to run a process irrespective of its attributes.

 

Another transformer to look at, instead of the UniqueValueLogger, would be either the DuplicateFilter, or Sampler (with it set to First N Features)

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

hkingsbury
Rank
Userlevel 5
Badge +29

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

Without knowing in detail the configuration of your WorkSpace runner and the Workspace it calls, its difficult to answer that. My guess would be that the child workspace is not configured to filter the features it reads based on the features that trigger it in the parent workspace

Check out the Abley FME Firesides on Youtube: https://www.youtube.com/@ableyltd
R
Badge

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

The child workspace is not configured for filtering. The runner filters the input table in order to get a single value to plug into the uniquevaluelogger, which sends that single unique value to the child workspace, which then runs the workspace once for each value in the field the uniquevaluelogger accesses. It's strange, but it seems to work. Thank you for you input.

jkr_wrk
Rank
Userlevel 3
Badge +18

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

Can you show the evidence of the WorkspaceRunner running more than ones. Because from what i see in your screenshot is not strange at all.

 

 

 

Sample: Every Nth Feature (N=1)
R
Badge

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

The screen shot shows how I used the testfilter to get a single value for the facility name. There are 4 unique values. Then I put the filtered table through uniquevaluelogger, only the one value comes out the other end (so to speak) and is plugged into the runner which runs a workspace with the same table in it. Somehow, it knows to create a unique output for each of the four facility names.

Without the filter, there would be four unique values coming out of the uniquevaluelogger and the child fme would run four times creating all four outputs each time.

jkr_wrk
Rank
Userlevel 3
Badge +18

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

That is the desired behavior.

 

Sometimes you want to read a featureset and want to run the workspace runner for every feature that you read. Using one or more feature attributes as input.

Sample: Every Nth Feature (N=1)
rmcgregor

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

It is only receiving one value (the one filtered for), so there is only one output for the uniquevaluelogger, but the child fme 'gets' that it's supposed to create a unique output for each of the four values in the table. It works and I don't need to drag this out. I appreciate everyone's input. I gave the 'best answer' to the gentleman who first provided the correct answer, but many have provided similar information. Thank you!

rmcgregor

Thank you. What I don't understand is how the 'child' fme (the one triggered by the runner) knows to cycle through each value and produce one output for each. My workflow appears to work, so I guess I am only asking a question at this point. It feels a bit like a hack to do it this way...

I think I see what's happening. I used 'coordinate system setter' earlier and the reprojector is telling me that it is ignoring THAT setting in favor of the coordinate system that the reprojector has been instructed to use.

Reply


Community Stats

28,729
Posts
109,527
Replies
36,565
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.