Question

Statistics Calculator Python "Overflow error" for setAttribute()

  • 15 December 2017
  • 3 replies
  • 2 views
A

When passing approx. 1.300.000 features to the statistics calculator for calculating max features within groups of features, I receive this error:

StatisticsCalculator_InputProcessor(TeeFactory): Cloned 1282424 input feature(s) into 1282424 output feature(s)<br>StatisticsCalculator_UnneededInputNuker(TeeFactory): Cloned 1282424 input feature(s) into 0 output feature(s)
StatisticsCalculator_CompleteOutputter(SortFactory): Output 0 feature(s) without doing any sorting
Python Exception <TypeError>: Overflow error for setAttribute().
@Python failed to execute function `StatisticsCalculator.summarizeStatistics'
StatisticsCalculator_SummaryCreator(CreationFactory): StatisticsCalculator_SummaryExploder: @Python execution failed

Could it be a memory failure? And if so, can I circumvent this be eg. dividing my features into the groups that I already am trying to fetch the max value for? And how would i do that?

Edit: 

When I lower the features to, say 800.000, the statistics calculator runs smoothly. At the moment I will try to stringformat the attributes from my feature source used in the calculator, to see if this will do the trick.

3 replies

david_r
Rank
Userlevel 4

This can happen if one of the results in your StatisticsCalculator exceeds the maximum value of a 64-bit unsigned integer, which equals 264-1 or 18,446,744,073,709,551,615. It could also occur if one of the results are less than the minimum value of a 64-bit signed integer, which equals -(263) or -9,223,372,036,854,775,808.

Try removing the output attribute names for the calculations you do not need (in particular those that could potentially suche huge numbers) and see if that makes the problem go away, for example:

A

This can happen if one of the results in your StatisticsCalculator exceeds the maximum value of a 64-bit unsigned integer, which equals 264-1 or 18,446,744,073,709,551,615. It could also occur if one of the results are less than the minimum value of a 64-bit signed integer, which equals -(263) or -9,223,372,036,854,775,808.

Try removing the output attribute names for the calculations you do not need (in particular those that could potentially suche huge numbers) and see if that makes the problem go away, for example:

0684Q00000ArKZ5QAN.png

I did remove both all the excess attributes from the input features and opted only for the calculation of the max value for a group, before running StatisticsCalculator and getting the same error. I am fairly new to FME, but not SQL, so I went on from the StatisticsCalculator and used the InlineQuerier, which gave me what I was looking for. Thanks for the troubleshooting!

SELECT "gdkbyg_id", MAX("barriere_hoejde") AS max_barriere_hoejde
FROM Output_2
GROUP BY "gdkbyg_id"
david_r
Rank
Userlevel 4

I did remove both all the excess attributes from the input features and opted only for the calculation of the max value for a group, before running StatisticsCalculator and getting the same error. I am fairly new to FME, but not SQL, so I went on from the StatisticsCalculator and used the InlineQuerier, which gave me what I was looking for. Thanks for the troubleshooting!

SELECT "gdkbyg_id", MAX("barriere_hoejde") AS max_barriere_hoejde
FROM Output_2
GROUP BY "gdkbyg_id"
That's a good workaround, thanks for sharing. 

 

It would be great if Safe could isolate and fix this bug, however. Could you share a screenshot of how your StatisticsCalculator has been configured as well as the resulting value for "max_barriere_hoejde"?

Reply


Community Stats

28,669
Posts
109,357
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.