For my workbench I have one input dataset from an oracle spatial reader. The dataset has over 2 million features. At the moment workbench takes over 12 hours to run as the features get stuck at a featurejoiner transformer near the end of the workbench.
Solved
How can I set up a workspace runner to run 100,000 records at a time and finish when it reaches the end of the dataset(oracle spatial table)?
+1- Contributor
Best answer by markatsafe
@buckrogers I think you can probably leverage WorkspaceRunner and a BETWEEN query to your database. In the 'parent' workspace calculate how many records you have (COUNT(*)) and then divide that by your batch size. You should be able to calculate the value1 & value2 for the BETWEEN. Then pass those to WorkspaceRunner which will call the child workspace - that actually does the work. Something along those lines...
I've included a couple of example workspaces and sample data that you can use as a possible starting point.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
@buckrogers I think you can probably leverage WorkspaceRunner and a BETWEEN query to your database. In the 'parent' workspace calculate how many records you have (COUNT(*)) and then divide that by your batch size. You should be able to calculate the value1 & value2 for the BETWEEN. Then pass those to WorkspaceRunner which will call the child workspace - that actually does the work. Something along those lines...
I've included a couple of example workspaces and sample data that you can use as a possible starting point.
+1- Contributor
@Mark Stoakes that's working perfectly and has quartered the run time. Thank you so much
Login to the community
No account yet? Create an account
An FME Account is required to contribute
LoginEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.