When I set the WorkspaceRunner Number of Workspaces per FME Process parameter the last batch isn't run. For example if I set the value to 10 and there are 42 items to process then only the first 40 are processed.
I have created simple Parent/Child workspaces that demonstrate the problem.
I am using the latest Mac build of 2019.
FME Desktop 2019.2.2 for 64-bit Mac OS X
Build 19817 / January 7th, 2020.
I was able to reproduce the issue. It seems that the last batch wouldn't be run if the number of processes (jobs) was less than the Number of Processes per FME Process. This could be a bug in the WorkspaceRunner.
@danatsafe, could you please take a closer look at this?
A workaround is to modify the Number of Processes per FME Process so that remainder would not happen. For example, if you set 21 to the parameter in your demo, remainder would be 0 (42 / 21 = 2 remainder 0). However, if the total number of processes is unknown when creating the workspace, set 1 to the parameter, or set Yes to the Wait for Job to Complete.
I was able to reproduce the issue. It seems that the last batch wouldn't be run if the number of processes (jobs) was less than the Number of Processes per FME Process. This could be a bug in the WorkspaceRunner.
@danatsafe, could you please take a closer look at this?
A workaround is to modify the Number of Processes per FME Process so that remainder would not happen. For example, if you set 21 to the parameter in your demo, remainder would be 0 (42 / 21 = 2 remainder 0). However, if the total number of processes is unknown when creating the workspace, set 1 to the parameter, or set Yes to the Wait for Job to Complete.
Thanks for looking into this.
Hi @takashi and @ianashley I see the problem too and have filed FMEENGINE-63048 about the WorkspaceRunner. Fortunately there seems to be an easy work-around: enable feature caching.
Hi @takashi and @ianashley I see the problem too and have filed FMEENGINE-63048 about the WorkspaceRunner. Fortunately there seems to be an easy work-around: enable feature caching.
Thanks very much.
Thanks very much.
@ianashley The problem with the WorkspaceRunner is now fixed in the latest FME 2020 betas. For FME 2019, enabling feature caching should work around it, or adding a Logger or even a Junction to the Summary port.