Skip to main content
Solved

Group processing by objects count and parallelize treatment


geo-x
Contributor
Forum|alt.badge.img+6
  • Contributor

Dear FME Community,

I should to optimize my job, and for that I need to make group processing with an optimization of objects numbers like that :

And after, as you can see, make a parallelize of objets for not having only a linear treatment.

 

Do you know how can I do this ?

 

Thank you very much.

Best answer by takashi

You can create a custom transformer and set transtormer parameters - Group By and Parallel Processing to implement paralell processing. See these pages to learm more.

Custom Transformers and Parallel Processing
Parallel Processing in FME

However, I cannot understand why 24 objects (4 groups, 6+3+4+11; left side of your screenshot) should become 27 objects (3 groups, 6+10+11; right side) after perfoeming "Group 10 objects" (center). Could you please explain how the "Group 10 objects" should work?

View original
Did this help you find an answer to your question?

2 replies

takashi
Influencer
  • Best Answer
  • June 13, 2025

You can create a custom transformer and set transtormer parameters - Group By and Parallel Processing to implement paralell processing. See these pages to learm more.

Custom Transformers and Parallel Processing
Parallel Processing in FME

However, I cannot understand why 24 objects (4 groups, 6+3+4+11; left side of your screenshot) should become 27 objects (3 groups, 6+10+11; right side) after perfoeming "Group 10 objects" (center). Could you please explain how the "Group 10 objects" should work?


geo-x
Contributor
Forum|alt.badge.img+6
  • Author
  • Contributor
  • June 16, 2025


Dear ​@takashi,

Thank you for your help. This post is the suit of this post :

who works, but I’ve a problem with “Insufficient memory available”

(My PC configuration is :
Processor    Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz   2.90 GHz
Installed RAM    64.0 GB (63.0 GB usable)
Storage 750 Go
System type    64-bit operating system, x64-based processor)

So I try to optimize my workbench with a group and parallel processing.

For the number of objects in a gorup, sorry, you’re right, this was a mistake I have corrected the picture.

So, I would like the process, optimize the groups of objects of 10 maximum and if a group is composed of about 11 objects, he take it in one group only.

I’m go to reading articles which you shared.

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings