Skip to main content
Question

Alternative to Aggregator

  • November 24, 2020
  • 2 replies
  • 313 views

markcoopersdc
Contributor
Forum|alt.badge.img+6

Hi. I am using the Aggregator transformer to summarise some tabular data by a reference (there may be multiple records originally for each reference). I am merging the attributes and concatenating some attributes.

 

This is technically doing exactly what I want but with nearly 1m records, it is very slow. I know there are usually multiple ways to achieve something in FME so I wondered there an alternative to the Aggregator to do what I need?

 

Many thanks

 

Thanks

2 replies

fdw
Contributor
Forum|alt.badge.img+6
  • Contributor
  • November 24, 2020

You could try using a few list transformers: listbuilder (groupby), listconcatenator, listhistogrammer, etc. Not sure if that would be any faster though.


hkingsbury
Celebrity
Forum|alt.badge.img+55
  • Celebrity
  • November 24, 2020

I initially jumped straight to statisticsCalculator, but because you're wanting to concatenate attrs that wont work.

 

You could look at python, that may offer some speed improvements.

 

Assuming that your aggregator will be using groupby, you could embed the aggregator in a custom transformer. Then set the custom transformer to groupby the same attribute and enable parallel processing.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings