Skip to main content
Released

It's time for Safe to rebuild StatsCalculator!

Related products:FME Form
  • August 28, 2019
  • 4 replies
  • 40 views

dataninja
Forum|alt.badge.img

Hi everyone,


This is basically what I am trying to do in FME and even though my computer has 32GB of RAM, it's basically too much for FME to handle (It takes forever and out-of-memory eventually).


But I did the same thing in Alteryx and it finished in 2:19 minutes...

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

4 replies

antoine
Enthusiast
Forum|alt.badge.img+7
  • Enthusiast
  • September 2, 2019

Hi,

I agree on this but I would rather compare to already "free" integrated numpy or R solutions instead (in your case PointCloudStatisticsCalculator should do the trick ). I guess some people might even have done some custom transformers on this. Btw Safe is working a lot on "feature table" features lately so it should soon be faster ! (https://knowledge.safe.com/questions/49482/what-transformers-have-feature-table-technology-fo.html)


fmelizard
Safer
Forum|alt.badge.img+22
  • Safer
  • September 6, 2019

Sorry it has taken us this long but indeed even as I type this the venerable StatsCalc is being re-imagined and redone. We'll post here when it is in an FME 2020 beta. Thanks for providing additional confirmation that we're headed in the right direction.


mark2atsafe
Safer
Forum|alt.badge.img+59

Does the update for 2020 solve the issues here?

According to our development team, version 9 of the transformer is on average 88% faster than version 8, with some cases being over 99% faster and almost all cases being over 70% faster. Some cases (string statistics with feature table input) are over 100x faster!


debbiatsafe
Safer
Forum|alt.badge.img+22

This idea has been implemented in FME 2020.0--the StatisticsCalculator has been rewritten and the transformer interface updated as well!

Should this feature or its implementation not address your concerns, related to this Idea posting, please let us know by leaving a comment.