Skip to main content
  • EN

Dear Support,

I am currently trying to execute a workspace and I have problem with the following error:

Unable to create worker; parallelism level may be too high for the translation requested

Transformer 'pointFetcher': Unable to create worker; parallelism level may be too high for the translation requested

For a set of 15000 polygons, I try in parallel and in a custom transformer to read pointclouds in a external database and then compute statistics for those pointclouds. Still within the custom transformer and in parallel, the input polygon is written with its statistics as attributes. By doing so in parallel and in a custom transformer, I am not overwhelming my RAM. In fact, as soon as the statistics are computed, the variables containing the pointcloud and the statistics for a polygon are erased from the RAM.

When launching this process in a minimal parallel processing mode (4 parallel subprocess) and for 1000 polygons, I do not encounter any problem.

When I launch the workspace for the 15000 polygons, I am getting the upper mentioned error after reading ~2060 polygons.

I don't understand why the number of group should be a problem as 4 child process are the maximum allowed number of subprocess (minimal processing mode). If I understand correctly, the main program should send the most 4 jobs simultaneously and resend jobs as long as there are features to process.

This process has to work polygon by polygon, as aggregating them together will force me to read a huge pointcloud and thus saturating my RAM.

Is the parallel mode the right way to process these data. Why is FME complaining when the number of group is getting to high ?

In advance thanks for your helpful support,

Florian

Hello @floriangandor,

 

Would you mind sharing your workspace and the log file you produced that resulted in the error?

 

Hi @floriangandor, are you using a Mac?
Hi @floriangandor, are you using a Mac?
Hi, I am using a Windows on Win10.

 

 

Hello @floriangandor,

 

Would you mind sharing your workspace and the log file you produced that resulted in the error?

 

Hello Trent, I will send you my workspace on monday. How am I supposed to share you my workspace? Per mail ?

 

 

Hello Trent, I will send you my workspace on monday. How am I supposed to share you my workspace? Per mail ?

 

 

Hello @floriangandor

 

You can add it as an attachment to a reply here. Using the little paper clip icon at the top of the little chat window.

 

My first hunch is that windows is getting tired out on us from launching so many worker processes. We will work towards duplicating this here and either insert a retry or delay. I'm also curious if you are using a floating license or not.

Hello @floriangandor

 

You can add it as an attachment to a reply here. Using the little paper clip icon at the top of the little chat window.

 

Hereby is the workspace I use. I am still on FME 2016.1. We use floating license in our office.

 

Thanks for your quick as light support!

 

 

statistics-calculator-glaciers-b.fmw

 

 

maybe set the group by attribute to some modulo 4 counter?

to prevent the attempt to create more then 4 concurrent processes?

Reply


Community Stats

30,159
Posts
114,480
Replies
37,805
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.