Skip to main content
Question

Process large number of input files in groups


I have 10000 GML files that I have read in as a single merged feature type. The task is to convert them into FBX files, maximum of 300 MB per FBX files.

 

How do I accomplish this?

 

I have tried modulocounter but doesnt seem to work.

Also, I have tried fanout, and am trying to see if I should run a script to separate those files into folders first, prior to FME.

 

Any ideas?

4 replies

caracadrian
Contributor
Forum|alt.badge.img+23
  • Contributor
  • July 20, 2023

You should run your workspace via WorkspaceRunner and limit the number of features read.


  • Author
  • July 20, 2023
caracadrian wrote:

You should run your workspace via WorkspaceRunner and limit the number of features read.

How do I do that? Sorry I am pretty new to FME


caracadrian
Contributor
Forum|alt.badge.img+23
  • Contributor
  • July 20, 2023
corsa wrote:

How do I do that? Sorry I am pretty new to FME

You should start from this article Batch Processing Using the WorkspaceRunner (safe.com)

There are a few concepts (Parameters and splitting workloads) that might be hard to set up at first, but you will get the hang of it.


  • Author
  • July 20, 2023

Thanks @caracadrian​ . I managed to run it, but my output file keeps getting overwritten. How do I dynamically append, say _index like _1, _2, to avoid overwriting?


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings