For my workbench I have one input dataset from an oracle spatial reader. The dataset has over 2 million features. At the moment workbench takes over 12 hours to run as the features get stuck at a featurejoiner transformer near the end of the workbench.
Solved
How can I set up a workspace runner to run 100,000 records at a time and finish when it reaches the end of the dataset(oracle spatial table)?
+1- Contributor
Best answer by markatsafe
@buckrogers I think you can probably leverage WorkspaceRunner and a BETWEEN query to your database. In the 'parent' workspace calculate how many records you have (COUNT(*)) and then divide that by your batch size. You should be able to calculate the value1 & value2 for the BETWEEN. Then pass those to WorkspaceRunner which will call the child workspace - that actually does the work. Something along those lines...
I've included a couple of example workspaces and sample data that you can use as a possible starting point.
@buckrogers I think you can probably leverage WorkspaceRunner and a BETWEEN query to your database. In the 'parent' workspace calculate how many records you have (COUNT(*)) and then divide that by your batch size. You should be able to calculate the value1 & value2 for the BETWEEN. Then pass those to WorkspaceRunner which will call the child workspace - that actually does the work. Something along those lines...
I've included a couple of example workspaces and sample data that you can use as a possible starting point.
+1- Contributor
@Mark Stoakes that's working perfectly and has quartered the run time. Thank you so much
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.