Question

WorkspaceRunner and Maximum number of concurrent process limit

  • 31 May 2018
  • 1 reply
  • 20 views
J
Badge

I am wondering if there is any way to overcome the 7 concurrent FME process limitation of WorkspaceRunner? I am processing 2800 LAS files into DEM's (using a slightly modified version of the excellent example here) using a WorkspaceRunner being fed by a Directory and File Pathnames reader).

Our processing workstation has 16 cores and 64 gigs of ram... 7 concurrent instances of this workbench is only utilizing about 35-40% of my system resources, and it takes almost a day to get through all my files.

 

Is there any way (excluding upgrading to FME server for something that really doesn't need Server's capabilites) to get this running on more cores (or can someone help me figure out how to run it in a highly parallel fashion)?

Thanks!

1 reply

takashi
Rank
Userlevel 2
Badge +17

Hi @jwe, I don't think there is a way to overcome the limitation. If you could transplant the entire workflow in the current child workspace into a new custom transformer, you could utilize the Parallel Processing functionality to launch multiple (maximum = number of cores x 2) child processes to run the workflow defined in the custom transformer concurrently. See also these links.

Reply


Community Stats

28,694
Posts
109,434
Replies
36,545
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.