Question

Hello is there any steps to speed up the run time of FME workbench

  • 24 November 2023
  • 3 replies
  • 81 views

Hello is there any steps to speed up the run time of FME workbench, My workbench is taking more than 6 hours of time for execution.


3 replies

Userlevel 6
Badge +33

It depends 🙂 Workspaces can do so many different things, it is hard to say anything about performance improvement without having more info.

 

The first thing you need to do is to determine what is the current bottleneck. For example, if the source database / dataset is slow, then load time is holding everything up.

 

Questions I have:

  • What is you workspace generally doing? Raster data? Vector data? Webservices?
  • Do you see "Optimizing memory" in the log?
    • This is a red flag, it does tell you the memory if full and the process is swapping, i.e. writing memory to disk, and reading it back later on, this creates massive amounts if I/O and is slow.
    • If you have this, you probably need to redesign your process. Like tiling you datasets and process per tile.
  • Do you have FeatureCaching on during a production run?
    • This should be off as it eats memory.
  • Where are you reading from and writing to?
    • Local SSD? This is fast.
    • Network drive? This is slow.
  • Do you need all data you are loading? Or can you do the filtering in the source database? Let the db do the work is best practice.

 

Also see the article Performance Tuning FME.

 

Userlevel 5
Badge +36

In addition to all aspects mentioned above, there may be 'blocking transformers' in your workspace. Blocking transformers will hold all features until the last feature has entered. Lots of data in combination with one or more blocking transformers may considerably slow down your workspace. Blocking transformers will also store all features in memory or in a temporary file on disk.

In that case it may be possible to eliminate all or some blocking transformers by changing your workflow.

Sometimes it is also possible to speed up the workspace by using Group Processing in blocking transformers.

Userlevel 5

+1 for https://community.safe.com/s/article/performance-tuning-fme

There's also this recorded webinar: https://www.youtube.com/watch?v=t8-FfO2-MCg

Reply