Skip to main content
Solved

Most efficient way to count multiple distinct values


ebygomm
Influencer
Forum|alt.badge.img+38

I have a workspace producing various outputs, I'd like to produce a further output which details the number of distinct customers and number of distinct products for each output but this appears to be either convoluted involving multiple steps and statistics calculators or slow involving a count distinct in the inlinequerier. Wondering if I'm missing a trick

Best answer by larry

This custom transformer should do the trick.

View original
Did this help you find an answer to your question?

13 replies

gio
Contributor
Forum|alt.badge.img+15
  • Contributor
  • October 13, 2016

You can use listbuilder followed by a , listhistogrammer

GroupBy the Attribute you want to have distinct element counted for.


ebygomm
Influencer
Forum|alt.badge.img+38
  • Author
  • Influencer
  • October 13, 2016

I don't think that will work, the two things are independent.

e.g. in the following I would want the customer value to be 2 and the product value to be 3


itay
Supporter
Forum|alt.badge.img+17
  • Supporter
  • October 13, 2016

Python to the rescue? or is it R time?


Forum|alt.badge.img+7

Hi @egomm,

When I am in this situation I make use of a PythonCaller in order to create statistics. If you would need some help with that, just let me know!


Forum|alt.badge.img
  • Best Answer
  • October 13, 2016

This custom transformer should do the trick.


itay
Supporter
Forum|alt.badge.img+17
  • Supporter
  • October 13, 2016
larry wrote:

This custom transformer should do the trick.

nice, so it was python to the rescue after all....

 

 


Forum|alt.badge.img
  • October 13, 2016
larry wrote:

This custom transformer should do the trick.

Just uploaded it to the hub: AttributeValueCounter

 

 

 


takashi
Influencer
  • October 14, 2016

I also think that Python scripting would be an efficient solution, but if the destination dataset is a database, SQL querying might also be worth to try.


ebygomm
Influencer
Forum|alt.badge.img+38
  • Author
  • Influencer
  • October 14, 2016
larry wrote:

This custom transformer should do the trick.

 

Perfect,thanks

gio
Contributor
Forum|alt.badge.img+15
  • Contributor
  • October 14, 2016

@egomm

Your example can easily be solved by..

Matcher followed by a statisticscalculator grouped by fme_feature_type...

Put in a custom for ease of use...

When using custom explode the features so you can use attr_name as input

No need for python or whatever.

And if its in a database then of course you would use sql.


larry wrote:

This custom transformer should do the trick.

I am trying to use this transformer to count all the unique values in a column. but does seem to work. need some help. My data as below.

 

A

 

1

 

1

 

1

 

2

 

3

 

0

 

0

 

 

Output should look like:

 

0 -> 2

 

1 -> 3

 

2 -> 1

 

3 -> 1

 

 


takashi
Influencer
  • May 2, 2018
sobanmughal wrote:
I am trying to use this transformer to count all the unique values in a column. but does seem to work. need some help. My data as below.

 

A

 

1

 

1

 

1

 

2

 

3

 

0

 

0

 

 

Output should look like:

 

0 -> 2

 

1 -> 3

 

2 -> 1

 

3 -> 1

 

 

Hi @sobanmughal, The AttributeValueCounter counts the number of distinct values for each attribute, I don't think it's suitable to your requirement. You can just use the StatisticsCalculator here. Set these parameters and check the resulting features output from the Summary port.

 

  • Group By: A
  • Attributes to Analyze: A
  • Total Count Attribute: <an attribute name which will store desired count value>

takashi wrote:
Hi @sobanmughal, The AttributeValueCounter counts the number of distinct values for each attribute, I don't think it's suitable to your requirement. You can just use the StatisticsCalculator here. Set these parameters and check the resulting features output from the Summary port.

 

  • Group By: A
  • Attributes to Analyze: A
  • Total Count Attribute: <an attribute name which will store desired count value>
Thanks takashi, I have found another custom transformer that can do the same thing.

 

UniqueValueLogger

 

 

https://hub.safe.com/transformers/uniquevaluelogger

 

 

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings