Skip to main content
Hello, 

 

 

how do you batch process two readers?

 

 

eg; I have 30 very large shapefiles that I want to analyse against a tiled grid.

 

 

I have one folder that has the 30 shapefiles and another folder that has a tiled grid of a certain area, about 26 tiles, tiled because of the large volume of data (it has to be tiled, no way around it).

 

 

So, how do you batch this so that shapefile 1 will be analysed against all 26 tiles and written out to it's new shapefile, then shapefile 2 will be analysed against all 26 tiles and written out to the new shapefile etc etc. Seems to be a double batching of somesort. When there was just 1 tile in the tests, the batch wizard helped but not anymore.

 

 

thanks
Hi,

 

 

you will probably have to create a batch for this yourself, I doubt the batch wizard is able to do it for you.

 

 

Alternatively, create a new workspace and control the other workspace(s) with a WorkspaceCaller.

 

 

David
thanks, 

 

 

so if I had the 30 shapefiles on batch in a given workspace and had them feed into a WorkspaceCaller, within which called the 26 tiles on batch, that would analyse each shapefile through one at a time against all 26 tiles?
Hi,

 

 

yes, this would be a solution.

 

1) Create a workspace which reads a shape file and the grid (26 tiles), analyses the shape file against 26 tiles and writes the result.

 

2) Create another workspace which reads every shape file path strings with the Directory and File Pathnames reader. Each of features read by the reader will have a shape file path as its attribute, so you can run the first workspace for each shape file one by one, passing the file path to the 1st workspace as SourceDataset parameter through the WorkspaceRunner.

 

I think this is David's second suggestion.

 

 

Takashi
right ok, thanks for that. 

 

 

So in my '1 shapefile to every tile of the grid' - workbench1 -  i can just choose any one shapefile to act as the reader because in the WorkspaceRunner - workbench2 - i am choosing $(SourceDataset_PATH) as the source file(s) which will supercede any shapefile nominated in workbench1. have i understood correct?
That's correct 🙂
Hi,

 

 

just be aware that if you're planning on publishing your workspaces on FME Server, you should use the FMEServerWorkspaceRunner rather than the WorkspaceRunner.

 

 

David
hmm, it still doesn't run all shapefiles against all tiles, i am only getting the results written for shapefile 1 against 26 tiles. I have set the writer to fan out on a concatenation of tile name and shapefile name (so there should be 30 x 26) but i am just getting shapefile 1 results still. 

 

 

The source Esri Shape files in WorkspaceRunner are $(SourceDataset_PATH)
can anyone help as to why it is not working? the workspace runner is set as the following;

 

 

 

 

my results are only the 1st shapefile against the 26 tiles = 26 .dbf results.

 

 

It is reading in all the files though, the results log says all files have been read/run by WorksapceRunnner but only the results of shapefile 1 is being written.

 

 

Using the attribute 'path_windows' in place of $(SourceDataset_PATH) produces an error thusly; 

 

WorkspaceRunner: Failed to run workspace C:/FastProcessingSam/Wales/WorkCaller_test/tiles_polygons.fmw -- Trace was:

 

 

                   FME 2014 (20140112 - Build 14230 - WIN32)

 

                      FME Desktop ESRI Edition (floating)

 

                               Permanent License.

 

                       Machine host name is: BUD-67CXX4J

 

                Copyright (c) 1994 - 2014 by Safe Software Inc.

 

                               Safe Software Inc.

 

 

 

Reading........2500.....5000....

 

 

SHAPE Reader: Dataset `C:\\xxxxxxxxxxx\\shapefile1.dbf' lacks the required extension `shp'

 

Program Terminating

 

 


sorry, I misunderstood your previous post.

 

You should specify "path_windows" to Source Esri Shape File(s) parameter. "path_windows" is an attribute of features read by the PATH reader, which contains a shape file path string.
but using 'path_windows' creates a host of errors that i have posted above, at least $(SourceDataset_PATH) actually runs (albeit not creating anough results)

 

 

or do i restrict the PATH reader to look at *.shp in the directory as opposed to  * ?
Did you specify Path Filter parameter like this?

 

 

 


ahhhh, of all the many test runs i did of different parameters and differents settings, the only combo i did not have was 'path_windows' attribute in the Source Esri Shape File(s) parameter AND *.shp in the Path Filter parameter in the PATH reader. It worked. 

 

 

thanks so much for your help
Good to hear you got a solution 🙂

Reply