Skip to main content
Solved

Multiple ws with fmeobjects.FMEWorkspaceRunner()


jorge_vidinha
Contributor
Forum|alt.badge.img+2

Hello, im searching for the possibility of running multiple workspaces at the same time (not queued jobs) making use of the python fme objects fmeobjects.FMEWorkspaceRunner() .

I was kind of hoping that i could pass a list of dictionarys as 'parameters' argument to runWithParameters(workspace, parameters) ...

I was expecting the same behaviour as seen in workbench transformer WorkSpaceRunner, where you can set wait for job to complete = No and the amount of jobs to run (max 7, i suppose) run in parallel.

Im i missing something wiht this object, or im really suppose to implement some python multiprocessing technic to acomplished the parallelism needed ?

What your thoughts are ?

Regards,

Jorge

Best answer by david_r

The execution of FMEWorkspaceRunner() is synchronous, so you'll have to implement your own multithreading mechanisms if you need to run several workspaces in parallel, e.g. using the Python threading module.

On the other hand, I have no idea if the fmeobjects library is thread-safe...

View original
Did this help you find an answer to your question?

3 replies

david_r
Celebrity
  • Best Answer
  • June 13, 2018

The execution of FMEWorkspaceRunner() is synchronous, so you'll have to implement your own multithreading mechanisms if you need to run several workspaces in parallel, e.g. using the Python threading module.

On the other hand, I have no idea if the fmeobjects library is thread-safe...


sigtill
Supporter
Forum|alt.badge.img+24
  • Supporter
  • June 13, 2018

Use rather a Custom Transformer in parallell mode. And just run that workspace once. It will spin up multiple instances. https://knowledge.safe.com/articles/1211/parallel-processing.html


jorge_vidinha
Contributor
Forum|alt.badge.img+2
  • Author
  • Contributor
  • June 14, 2018
david_r wrote:

The execution of FMEWorkspaceRunner() is synchronous, so you'll have to implement your own multithreading mechanisms if you need to run several workspaces in parallel, e.g. using the Python threading module.

On the other hand, I have no idea if the fmeobjects library is thread-safe...

Thanks @david_r , somewhat taken into the incorrect assumption based on the name simililarity of FMEWorkspaceRunner() and the WorkspaceRunner transformer in Desktop. Wondering if naming the object as FMERunWorkspace() wouldnt be ok , but i dont know the background at all ...

 


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings