I have a very simple workbench: An excel file as input and straight after it a statistics calculator. Only: the results make no sense. Example: one column has a sum of 78000 in excel and an average of 68.
In FME the sum is 1500 and the average 1.8. It is very clear that the results in excel are the right results. What is happening?
Hard to say without data to reproduce... If you share a (stripped) file we can try and help.
I simplified the workbench and input
and the workbench
I can't reproduce you issue, using 2020.1.3.0. The mean I get is 69.03787878787878.
What version do you use?
Does adding a blank pythoncaller just before the statisticscalculator solve you issue?
I too don't see the issue with the incorrect sum, the mean will probably not be what you expect unless you map the empty attributes to zeros however. Although excel will ignore the empty cells in the same way if using '=AVERAGE'
The statistics calculator is a bit misleading here, because the count includes all records, but the mean only includes records with numerical values
I can't reproduce you issue, using 2020.1.3.0. The mean I get is 69.03787878787878.
What version do you use?
Does adding a blank pythoncaller just before the statisticscalculator solve you issue?
this is weird Even after a complete restart I still get 1.28:
Adding an empty pythoncaller makes no difference.
FME 2020.0.1.0
I too don't see the issue with the incorrect sum, the mean will probably not be what you expect unless you map the empty attributes to zeros however. Although excel will ignore the empty cells in the same way if using '=AVERAGE'
The statistics calculator is a bit misleading here, because the count includes all records, but the mean only includes records with numerical values
what u describe is exactly what I want ;)
I haven't looked at the workspace but sometimes if you use the same excel reader with a different file you might get unexpected results. If you have defined the schema by going through the parameters wizard in the Excel reader FME will scan the file and also figure out how many rows to read. If you then change the Input file to something with more rows, it can be that FME will read the same amount of rows as previously defined.
This could explain why others get different results. If you delete the reader and re add it then you might get the result you expect.
I haven't looked at the workspace but sometimes if you use the same excel reader with a different file you might get unexpected results. If you have defined the schema by going through the parameters wizard in the Excel reader FME will scan the file and also figure out how many rows to read. If you then change the Input file to something with more rows, it can be that FME will read the same amount of rows as previously defined.
This could explain why others get different results. If you delete the reader and re add it then you might get the result you expect.
I made a new excel-file and a new workbench for this thread/topic/forum (no copy-pasting). It had the same behaviour.
Upgrading to a newer FME-version did fix it though. I am still a buit bumped out, I really need to be able to trust the outcome of transformers.
I made a new excel-file and a new workbench for this thread/topic/forum (no copy-pasting). It had the same behaviour.
Upgrading to a newer FME-version did fix it though. I am still a buit bumped out, I really need to be able to trust the outcome of transformers.
Like @virtualcitymatt wrote I think it has to do with the reader only reading a specific range. Do you still got the old workspace? Did it read all the rows? No Feature Caching? No caching in the reader?
I made a new excel-file and a new workbench for this thread/topic/forum (no copy-pasting). It had the same behaviour.
Upgrading to a newer FME-version did fix it though. I am still a buit bumped out, I really need to be able to trust the outcome of transformers.
It shouldnt be since I made a completely new workbench especially for this thread without copying transformers
I made a new excel-file and a new workbench for this thread/topic/forum (no copy-pasting). It had the same behaviour.
Upgrading to a newer FME-version did fix it though. I am still a buit bumped out, I really need to be able to trust the outcome of transformers.
Did you also try to rename the excel file? I think maybe you've found a bug - It would be really good for safe support see this I think.
What you're describing sounds really familiar to me. I feels like I has the same issue a few years ago. I really feels like there is some weird caching of some schema definition going on somewhere.