Skip to main content
Question

Process large number of input files in groups


I have 10000 GML files that I have read in as a single merged feature type. The task is to convert them into FBX files, maximum of 300 MB per FBX files.

 

How do I accomplish this?

 

I have tried modulocounter but doesnt seem to work.

Also, I have tried fanout, and am trying to see if I should run a script to separate those files into folders first, prior to FME.

 

Any ideas?

4 replies

caracadrian
Contributor
Forum|alt.badge.img+23
  • Contributor
  • July 20, 2023

You should run your workspace via WorkspaceRunner and limit the number of features read.


  • Author
  • July 20, 2023
caracadrian wrote:

You should run your workspace via WorkspaceRunner and limit the number of features read.

How do I do that? Sorry I am pretty new to FME


caracadrian
Contributor
Forum|alt.badge.img+23
  • Contributor
  • July 20, 2023
corsa wrote:

How do I do that? Sorry I am pretty new to FME

You should start from this article Batch Processing Using the WorkspaceRunner (safe.com)

There are a few concepts (Parameters and splitting workloads) that might be hard to set up at first, but you will get the hang of it.


  • Author
  • July 20, 2023

Thanks @caracadrian​ . I managed to run it, but my output file keeps getting overwritten. How do I dynamically append, say _index like _1, _2, to avoid overwriting?


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings