Skip to main content
Open

Support percentile list in StatisticsCalculator

Related products:Transformers

bruceharold
Contributor

For numeric statistics, support a percentile list output in the Summary for integer attributes. The list elements would be the value in the distribution which is less than p% where p = 1,2,3,...99.

3 replies

DanAtSafe
Safer
Forum|alt.badge.img+18
  • Safer
  • August 30, 2023

Hi @bruceharold​ I've linked this idea to FMEENGINE-78093


bruceharold
Contributor
Forum|alt.badge.img+17
  • Author
  • Contributor
  • August 30, 2023

Thanks!


bruceharold
Contributor
Forum|alt.badge.img+17
  • Author
  • Contributor
  • August 30, 2023

A little background FYI dear reader, this idea sprang from my asking OpenAI what transformer to use to calculate percentile values, and it hallucinated with a suggestion to use the percentile statistic option - which doesn't exist. I didn't save the prompt and response so just resubmitted it, and look what I got!

 

What FME transformer will return the percentile of an integer attribute?

 

  1. I apologize for the confusion in my previous response. FME does not have a built-in transformer specifically designed to calculate percentiles of an integer attribute. However, you can achieve this by using custom transformers or scripting within FME. One approach is to use the StatisticsCalculator transformer along with some additional transformers and calculations:Use the StatisticsCalculator transformer to calculate statistics on your integer attribute (e.g., minimum, maximum, count).
  2. Use the AttributeCreator transformer to create new attributes for each desired percentile.
  3. Calculate the percentile values using formulas based on the statistics obtained from step 1 and assign them to the newly created attributes using AttributeManager or ExpressionEvaluator transformers.
  4. Here's an example of how you can calculate percentiles manually using these steps:Connect your input feature(s) containing the integer attribute to a StatisticsCalculator transformer.
  5. Open the parameters for the StatisticsCalculator transformer and select your integer attribute under "Attribute".
  6. Checkmark "Minimum", "Maximum", and "Count" under "Calculate Statistics".
  7. Run your workspace.
  8. Next, add an AttributeCreator transformer after the StatisticsCalculator:In AttributeCreator, create new attributes for each desired percentile (e.g., p25, p50, p75).
  9. Connect a LineOnLineOverlayer transformer after AttributeCreator.

In LineOnLineOverlayer: 7a: Set up one line feature with two points representing (0%, min value) and (100%, max value). 7b: Set up another line feature with two points representing (25%, 0) and (75%, 0). Finally, connect an Intersector or NeighborFinder after LineOnLineOverlayer: 8: Configure it to intersect/associate with your original features based on spatial relationships. 9: Use an ExpressionEvaluator or AttributeManager transformer to calculate percentiles based on intersections/associations between lines and features. Please note that this approach requires additional configuration beyond standard FME transformers but allows you to compute percentiles within FME workflows without relying on built-in percentile calculation functionality.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings