Hello!
I've been having some issues with the cumulative output from StatisticsCalculator - it seems to be giving me the complete sum rather than cumulative sum of the chosen attribute.
Simplified example:
Anyone come across similar issues? Suspect this might be a bug.
Cheers!
Best answer by fmelizard
Sorry about this one. Backport can't happen...but a fix still can. In FME 2020 we've redone the StatisticsCaculator to work in bulk mode and so you should expect a pretty good speedup (final #s still being calculated).
In FME 2019.x, there is an unfortunate interaction between bulk mode and the unique way StatisticsCalculator was written that affects the Cumulative port. A fix is in the works -- watch this space. In the meantime putting a PythonCaller (or AttributeKeeper) in front to blow up the feature table will do the trick.
We'll post back here when the fix is made. Thanks for letting us know, sorry again.
+56To be honest I just tried it and everything looked fine.
Could it be to do with caching? Maybe it's reading from an old cache instead of new data? Perhaps try Shift+F5 to run, erasing all the caches.
What version of FME do you have? Then I can look it up to see if it's a known issue.
To be honest I just tried it and everything looked fine.
Could it be to do with caching? Maybe it's reading from an old cache instead of new data? Perhaps try Shift+F5 to run, erasing all the caches.
What version of FME do you have? Then I can look it up to see if it's a known issue.
Hi Mark, cheers.
Erasing caches didn't work unfortunately.
I'm running 2019.1 (Build 19608). Strange thing is it works on my colleague's version without a hitch - he's on the 2018 version
+56To be honest I just tried it and everything looked fine.
Could it be to do with caching? Maybe it's reading from an old cache instead of new data? Perhaps try Shift+F5 to run, erasing all the caches.
What version of FME do you have? Then I can look it up to see if it's a known issue.
Hmmm. I also wonder if it's something to do with the new bulk mode. Can you put a Sampler transformer before it, and just set it to randomize sampling? That should (I think) break up the bulk mode. Then if the StatsCalc works as expected then we'll know.
To be honest I just tried it and everything looked fine.
Could it be to do with caching? Maybe it's reading from an old cache instead of new data? Perhaps try Shift+F5 to run, erasing all the caches.
What version of FME do you have? Then I can look it up to see if it's a known issue.
No joy there either! Still getting the final sum
+56No joy there either! Still getting the final sum
I think the bulk mode is what's causing it though. If I create some random features then it works fine, but if I read from a format that supports bulk mode, I get the same issue. I'll query the developers, but in the meantime I'll try and find a way to drop out of bulk mode to prove that's the issue and give us a workaround.
I think the bulk mode is what's causing it though. If I create some random features then it works fine, but if I read from a format that supports bulk mode, I get the same issue. I'll query the developers, but in the meantime I'll try and find a way to drop out of bulk mode to prove that's the issue and give us a workaround.
Ah okay, does sound like that's the suspect. Thanks for looking into it for me! Much appreciated. A workaround would do just fine for the moment :)
+44This looks to be the same issue
https://knowledge.safe.com/questions/102581/statisticscalculator-suddenly-not-yielding-the-sam.html
If you add a python caller prior to the statistics calculator, it seems to sort it out (you don't appear to need any code other than the default)
+44To be honest I just tried it and everything looked fine.
Could it be to do with caching? Maybe it's reading from an old cache instead of new data? Perhaps try Shift+F5 to run, erasing all the caches.
What version of FME do you have? Then I can look it up to see if it's a known issue.
This looks to be the same issue
https://knowledge.safe.com/questions/102581/statisticscalculator-suddenly-not-yielding-the-sam.html
If you add a python caller prior to the statistics calculator, it seems to sort it out (you don't appear to need any code other than the default)
+56This looks to be the same issue
https://knowledge.safe.com/questions/102581/statisticscalculator-suddenly-not-yielding-the-sam.html
If you add a python caller prior to the statistics calculator, it seems to sort it out (you don't appear to need any code other than the default)
Perfect. I found Aggregator/Deaggregator, but PythonCaller is much easier. So that's the workaround for now @evanrichards_. It seems like this is fixed in FME2020 but I've asked if we can get a backport to a 2019 release (2019.2.x or something) so you can get it working sooner.
+56Ah okay, does sound like that's the suspect. Thanks for looking into it for me! Much appreciated. A workaround would do just fine for the moment :)
Yeah, sadly, for technical reasons, I don't think a backport can happen, so we'll be stuck with the workaround until 2020. But if it's working for you right now with that workaround, then that's good.
Perfect. I found Aggregator/Deaggregator, but PythonCaller is much easier. So that's the workaround for now @evanrichards_. It seems like this is fixed in FME2020 but I've asked if we can get a backport to a 2019 release (2019.2.x or something) so you can get it working sooner.
Perfect! The PythonCaller works for me. Thanks both for the help :)
+20Sorry about this one. Backport can't happen...but a fix still can. In FME 2020 we've redone the StatisticsCaculator to work in bulk mode and so you should expect a pretty good speedup (final #s still being calculated).
In FME 2019.x, there is an unfortunate interaction between bulk mode and the unique way StatisticsCalculator was written that affects the Cumulative port. A fix is in the works -- watch this space. In the meantime putting a PythonCaller (or AttributeKeeper) in front to blow up the feature table will do the trick.
We'll post back here when the fix is made. Thanks for letting us know, sorry again.
