Question

Does the linux version support parallel processing?

  • 28 September 2015
  • 1 reply
  • 1 view

Badge
Wondering if the Linux-version supports parallel processing. I am running some bufferings and dissolves on a national level and there should be room for parallel processing on these jobs. Using htop it seems like the processing is restricted to one core:

 

 

 

 

I have tried setting the dissolvers parallel processing level to extreme, but the job has no signifcant speed increase.

 

 

I am using Edition: FME Desktop Microsoft SQL Server Edition (floating) Version: FME(R) 2015.1.2.0 (20150911 - Build 15538 - linux-x64)

1 reply

Badge
It does. I tried the examples provided by FME (https://knowledge.safe.com/articles/Samples_and_Demos/Parallel-Processing) and found that my linux handles this expertly. The challenge remains in finding out when to use parallel processing and not.

 

 

The transformers I am using are mostly buffering, dissolve and AreaonAreaOverlayer. For the dissolve part I guess there will not be any real gain. It would however be interesting to have an overview of proven approaches for "splitting" jobs so that several cores can be used. Could for example splitting a dissollve job into several smaller jobs assigned with different cores and then having the resulting objects dissolved in a last job be a good approach? Guess it is not in place, but to me it sounds like a relevant approach which could work well on dissolve jobs:
  1. Divide features into n groups
  2. assign each group to one core
  3. use transformer
  4. collect resulting data from all groups
  5. used transformer on resulting data
I would love to see an advisory chart related to the different parallel enabled processes and preconditions having benefits from parallel computing.

Reply