Skip to main content
Question

Alternative to Aggregator

  • November 24, 2020
  • 2 replies
  • 364 views

markcoopersdc
Contributor
Forum|alt.badge.img+6

Hi. I am using the Aggregator transformer to summarise some tabular data by a reference (there may be multiple records originally for each reference). I am merging the attributes and concatenating some attributes.

 

This is technically doing exactly what I want but with nearly 1m records, it is very slow. I know there are usually multiple ways to achieve something in FME so I wondered there an alternative to the Aggregator to do what I need?

 

Many thanks

 

Thanks

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

fdw
Contributor
Forum|alt.badge.img+7
  • Contributor
  • November 24, 2020

You could try using a few list transformers: listbuilder (groupby), listconcatenator, listhistogrammer, etc. Not sure if that would be any faster though.


hkingsbury
Celebrity
Forum|alt.badge.img+65
  • Celebrity
  • November 24, 2020

I initially jumped straight to statisticsCalculator, but because you're wanting to concatenate attrs that wont work.

 

You could look at python, that may offer some speed improvements.

 

Assuming that your aggregator will be using groupby, you could embed the aggregator in a custom transformer. Then set the custom transformer to groupby the same attribute and enable parallel processing.