Skip to main content
Question

What is the right way to batch process with FME?

  • October 16, 2014
  • 2 replies
  • 290 views

Forum|alt.badge.img+1
I work with FME Desktop extensively, and many times, I need to run large processes on many files.  I have never come up with a satisfactory way to do so.

 

 

If the files are small and the process can be set up to do it, I will bring everything in in one reader and use Group By, but this sometimes is too taxing on the machine or doesn't work at all.

 

 

My only other option, then, is to write a batch file which invokes fme.exe multiple times, passing in different inputs and outputs.  But writing these batch files is fiddly and time-consuming.

 

 

Is there a better way?
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

  • October 16, 2014

Have you tried using a Directory and File Pathnames reader in conjunction with the WorkspaceRunner transformer?

I'm assuming you have one reader for your Large Process workbench. First, convert all your parameters to private. Then create one new published parameter, and call it something like Filepath.

 

 

Second, go to the Navigator, and open the Advanced Options portion of the reader. Select Feature Types to Read, click the drop down arrow, and link it to the Filepath parameter.

Third, create a new workbench. Put a Filepath and Directory reader into it, and select the folder where you store your many files that need to be processed.  Connect that reader to a WorkspaceRunner transformer, point that at your Large Process workbench, and feed the path_windows attribute to the published parameter.


Forum|alt.badge.img
  • October 20, 2014
If your files are all the same, (eg. same CAD layers), and the files are in the same directory then you should be able to use the File>Batch Deploy option. Also, you will probably need to change the settings of your reader General>Merge Feature Type. Check that box and then enter a * for the Merge Filter.