Skip to main content
Solved

Slow performance speed in workbench containing Custom Looping Transformer

  • February 20, 2018
  • 5 replies
  • 45 views

Forum|alt.badge.img+1

Hi,

I have a FME 2016.1 workbench that contains a Custom Loop transformer and within that I also have a PythonCaller with some Python Code.

The workbench outputs the expected result but it takes over an hour to run. I need it to run ideally in less than 20 minutes.

I have attached some screen shots of the main workbench and the custom transformer. Does anyone have ideas on things I could do to speed this up?

Thanks,

Best answer by gio

looks like the iteration addition, seeming to pose as a index, is not synched with the process.

Is the loop connection not totally superfluous? Looks like you can do this with just one input.

I suspect by the looks of it, that a lot of lists are created not in sequence but simultaneously, which might account for the performance issue.

It is not actually processing "one at a time" afacs (as far as I know...) and can see.

But, you are processing groups. I don't think you need to use a loop at all.

deductive comment based on process structure analysis.

View original
Did this help you find an answer to your question?

5 replies

Forum|alt.badge.img
  • February 20, 2018
Most likely something to do with your custom loop transformer, without test data and workbench won't be able to help much

 

 


redgeographics
Celebrity
Forum|alt.badge.img+50

Like @davidrich said: without the actual workbench and test data it's hard to give much specific advice.

One thing you can try is have the custom transformer openen when you run the workspace, by looking at the live feature counts you can get an idea of where time is lost

And just so you know, it may be that "an hour" is how long this takes.


gio
Contributor
Forum|alt.badge.img+15
  • Contributor
  • Best Answer
  • February 21, 2018

looks like the iteration addition, seeming to pose as a index, is not synched with the process.

Is the loop connection not totally superfluous? Looks like you can do this with just one input.

I suspect by the looks of it, that a lot of lists are created not in sequence but simultaneously, which might account for the performance issue.

It is not actually processing "one at a time" afacs (as far as I know...) and can see.

But, you are processing groups. I don't think you need to use a loop at all.

deductive comment based on process structure analysis.


Forum|alt.badge.img+1
  • Author
  • February 21, 2018
gio wrote:

looks like the iteration addition, seeming to pose as a index, is not synched with the process.

Is the loop connection not totally superfluous? Looks like you can do this with just one input.

I suspect by the looks of it, that a lot of lists are created not in sequence but simultaneously, which might account for the performance issue.

It is not actually processing "one at a time" afacs (as far as I know...) and can see.

But, you are processing groups. I don't think you need to use a loop at all.

deductive comment based on process structure analysis.

Hi @gio Yep I am processing groups. The reason I used a loop was because I couldn't work out how to process the attributes per group, without passing the groups individually into the loop...

 


Forum|alt.badge.img+1
  • Author
  • February 21, 2018
gio wrote:

looks like the iteration addition, seeming to pose as a index, is not synched with the process.

Is the loop connection not totally superfluous? Looks like you can do this with just one input.

I suspect by the looks of it, that a lot of lists are created not in sequence but simultaneously, which might account for the performance issue.

It is not actually processing "one at a time" afacs (as far as I know...) and can see.

But, you are processing groups. I don't think you need to use a loop at all.

deductive comment based on process structure analysis.

Oh I understand now! Yes the loop is completely superfluous. I completely over-thought things and I just needed to use a ListBuilder before the PythonCaller and that ensures that each records' group (as defined by the lists I just built) isprocessed individually.

 

The time taken to process the whole file is now only 10.8 seconds!!

 

Thanks so much for pointing this out @gio!

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings