Question

Batch Processing Failing

B
Badge

Hi,

I have a workbench wherein I am trying to batch process several .dwg datasets in one directory and put them into a file .gdb as follows:

I am building polygons based on a .dwg layer and am outputting to the desired .gdb.

I then have a separate workspace for utilizing the workspace runner so that I may process all these datasets as shown in "FME Workflow Management" as follows:

I receive the following error when running which I am unable to resolve:

WorkspaceRunner: Failed to run workspace C:/Users/blee/Desktop/MyDocs/Projects/PlanExtentWorkbench/Workbench/MissingPlanExtents.fmw -- Trace was:

FME 2017.0.1.1 (20170512 - Build 17291 - WIN64)

FME Desktop ESRI Edition (floating)

Permanent License.

Machine host name is: WLTS4018

Operating System: Microsoft Windows 7 64-bit Service Pack 1 (Build 7601)

Copyright (c) 1994 - 2017, Safe Software Inc.

Safe Software Inc.

I hope this is sufficent to help resolve this error.

Thanks in advance.

12 replies

oscard
Rank
Userlevel 2
Badge +19
I have also encountered that error in the past.

 

 

In my case there was a problem with the parameters of the workspace that is run with the WorskpaceRunner transformer.

 

 

mark2atsafe
Rank
Userlevel 4
Badge +25

Can you check which version of the WorkspaceRunner you are using? If necessary select Tools > FME Options > Transformers and turn on Display Transformer Version. Then add a summary annotation to your transformer (select then Shift+Ctrl+K).

If your transformer is a Version 3, can you upgrade to FME2018.1 and see if it works there? I believe there was a bug in version 3 that might cause this sort of issue.

Otherwise, can you try running the child workspace by itself, to ensure it completes? Obviously if it won't run then the WorkspaceRunner won't be able to run it.

As a final thing to try, delete and add a new WorkspaceRunner transformer.

Hopefully something there will work or give us more clues about what is going wrong.

B
Badge

Hi @Mark2AtSafe,

Thanks for the response! I actually am using FME Desktop 2017 (Build 17291) and the Workspace Runner transformer is Version 2.

The child workspace does run and produces the desired output. As well, I tried deleting and adding a new WorkspaceRunner but it eneded up with the same failed output.

The Workspace Runner ends up only outputting the last .dwg in the source folder.

B
Badge

I would also like to add that I receive this:

 

 

Reading...

Emptying factory pipeline...child killed: segmentation violation

I could not find much documentation as to why this error arises. I am looking into installing FME 2018, in order to see if it works in that environment.

Thanks.

mark2atsafe
Rank
Userlevel 4
Badge +25

I would also like to add that I receive this:

 

 

Reading...

Emptying factory pipeline...child killed: segmentation violation

I could not find much documentation as to why this error arises. I am looking into installing FME 2018, in order to see if it works in that environment.

Thanks.

That is not great at all. The "Emptying Factory Pipeline" part means that the readers have just finished in the workspace and are being closed. So that's a normal message to see. The other part... not so much. Can you get in touch with our support team directly? See safe.com/support - I suspect that to resolve this issue we'll need to try and replicate your setup or maybe even remotely connect to your machine to monitor it ourselves. That sort of error is something we'd want to take quick action about for sure.

 

 

B
Badge
That is not great at all. The "Emptying Factory Pipeline" part means that the readers have just finished in the workspace and are being closed. So that's a normal message to see. The other part... not so much. Can you get in touch with our support team directly? See safe.com/support - I suspect that to resolve this issue we'll need to try and replicate your setup or maybe even remotely connect to your machine to monitor it ourselves. That sort of error is something we'd want to take quick action about for sure.

 

 

I have contacted the support team directly. Thanks @Mark2AtSafe
D
Badge

I would also like to add that I receive this:

 

 

Reading...

Emptying factory pipeline...child killed: segmentation violation

I could not find much documentation as to why this error arises. I am looking into installing FME 2018, in order to see if it works in that environment.

Thanks.

Hi, @Mark2AtSafe Thanks for sharing this knowledge.

Even I am getting the same log- "Emptying Factory Pipeline". But, I am running my wb on AWS machine through the batch which has 2 wb inside. My actual workspace took around 11 hours to finish the process, and now it has already taken 5 hours for this log. Can we not do something about it, since it is about the per hour cost on the AWS server. Thanks.

mark2atsafe
Rank
Userlevel 4
Badge +25

Hi, @Mark2AtSafe Thanks for sharing this knowledge.

Even I am getting the same log- "Emptying Factory Pipeline". But, I am running my wb on AWS machine through the batch which has 2 wb inside. My actual workspace took around 11 hours to finish the process, and now it has already taken 5 hours for this log. Can we not do something about it, since it is about the per hour cost on the AWS server. Thanks.

Hi @dewan

Are you getting the other messages mentioned in this question, or just Emptying Factory Pipeline? As I mentioned, that message is perfectly normal and appears in every FME translation.

If the problem is that your workspace is running slower than you'd like, I suggest you create a new question for that. Other folk might have good ideas and they are unlikely to respond to this thread.

Include as much information as you can - for example the log file and either the workspace itself or a detailed screenshot of it. In general it's hard to say why something takes a long while. Usually it is a lack of memory so keep an eye on system resources to see what is being used. But if you are simply processing a large amount of data, well that can be the cause. Parallelizing the process on multiple cores or multiple Server engines would certainly help.

Regards

Mark

D
Badge

Hi @dewan

Are you getting the other messages mentioned in this question, or just Emptying Factory Pipeline? As I mentioned, that message is perfectly normal and appears in every FME translation.

If the problem is that your workspace is running slower than you'd like, I suggest you create a new question for that. Other folk might have good ideas and they are unlikely to respond to this thread.

Include as much information as you can - for example the log file and either the workspace itself or a detailed screenshot of it. In general it's hard to say why something takes a long while. Usually it is a lack of memory so keep an eye on system resources to see what is being used. But if you are simply processing a large amount of data, well that can be the cause. Parallelizing the process on multiple cores or multiple Server engines would certainly help.

Regards

Mark

Yes, that's correct. Got the solution, thanks!

ekkischeffler
Badge +1

I'm having the same issue. My WorkspaceRunner is reporting this error:

Reading....500.1000.500.2.10..1500...30.131.1500.3000..4000.500....5500..1500.7500.2000..8500..9500.2500.178.2000.2.49.2..500..10000..920...118..2..

Emptying factory pipeline....2..2child killed: segmentation violation

 

When I run the child directly, I'm getting this (not very useful) error message:

Error running translation.

 

I've upgraded all transformers to their latest versions. I've also searched on the 'Error running translation.' issue and found some weird workarounds, like turning off logging etc. Nothing seems to work. I've had this error in some completely unrelated instances before and usually the only way to fix it was to break up the workflow into separate FMWs and/or to remove some parts of it - without understanding why. Sometimes even removing some random transformer would solve it (while it's never the same type of transformer).

 

I'm working with FME Desktop 2019.2.2.0 (20200103 - Build 19817 - WIN64)

mark2atsafe
Rank
Userlevel 4
Badge +25

I'm having the same issue. My WorkspaceRunner is reporting this error:

Reading....500.1000.500.2.10..1500...30.131.1500.3000..4000.500....5500..1500.7500.2000..8500..9500.2500.178.2000.2.49.2..500..10000..920...118..2..

Emptying factory pipeline....2..2child killed: segmentation violation

 

When I run the child directly, I'm getting this (not very useful) error message:

Error running translation.

 

I've upgraded all transformers to their latest versions. I've also searched on the 'Error running translation.' issue and found some weird workarounds, like turning off logging etc. Nothing seems to work. I've had this error in some completely unrelated instances before and usually the only way to fix it was to break up the workflow into separate FMWs and/or to remove some parts of it - without understanding why. Sometimes even removing some random transformer would solve it (while it's never the same type of transformer).

 

I'm working with FME Desktop 2019.2.2.0 (20200103 - Build 19817 - WIN64)

Do you have the log file for the child process. In the other cases here it seems that a problem in that workspace was causing the first workspace to fail. This is an ugly message and not particularly useful, but by itself, it's probably not the direct cause of the issue.

ekkischeffler
Badge +1

Do you have the log file for the child process. In the other cases here it seems that a problem in that workspace was causing the first workspace to fail. This is an ugly message and not particularly useful, but by itself, it's probably not the direct cause of the issue.

Hi Mark, in this particular case I think I have tracked down the issue and reported it under C152357. Basically a feature with a geometry issue that was read from a DWG file. I don't have a log file but all it read is 'Error running translation'.

Reply


Community Stats

28,670
Posts
109,364
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.