Skip to main content
Solved

Creating an attribute that accumulates count of duplicate values (sequentially)

salvaleonrp
Enthusiast
Forum|alt.badge.img+15

I was able to do a cumulative count of duplicate values (where duplicate is “-1” only) using a Group By on a Counter transformer shown below.

 

(Added note: I should have added this to remove ambiguity - SORRY. The output as shown below in the Preview is the desired output. I only care to know the number of consecutive sequential values when the value is “-1”. If the pattern changes and another series of duplicate “-1” appears, the count starts with “1” again. I don’t care if the pattern for duplicate values are greater than “0”, like “2420” on the Preview.)

 

 

Since I have more than a dozen of these attributes, I’d like to use an AttributeCreator instead but I can’t seem to use the CurrentAttribute() to accumulate duplicate values.

In addition, everytime a sequence of “-1”s begin (as shown in row 143 above), count starts at 1 again.


Any suggestions?

Best answer by hkingsbury

You can use a AttributeCreator with Adjacent Features enabled

Then use a conditional statement to check if the previous features value equals the same as the current feature. Then the value will be the previous features count plus one if they’re the same. If the are not the same, set the count value to one

 

The one down side of this approach, is that FME thinks the count attribute doesn’t exist (it kinda does/doesn’t).

To fix this, you can put an AttributeExposer in front of the AttributeCreator to expose the attribute

 

View original
Did this help you find an answer to your question?

10 replies

todd_davis
Supporter
Forum|alt.badge.img+22
  • todd_davisSupporter
  • Supporter
  • September 26, 2024

The error you are getting with the counter restarting is due to your setting “When Group changes”. It needs to be “when all features received” because you groups are not in order. E.g. -1 is one of the first groups, then a new group of 2420 arrives, and then a “new group” or -1 arrives.

 

That is the first bit, I’ll have a think about the second. I do already have a quicker way, but like you say “dozen of attributes”, so there has to be a better way than my initial thought.

 

salvaleonrp
liamfez
2 people upvoted this
  • salvaleonrp
    salvaleonrp
  • liamfez
    liamfez
todd_davis
Supporter
Forum|alt.badge.img+22
  • todd_davisSupporter
  • Supporter
  • September 26, 2024

Actually, this should be the same logic used in the GroupCounter transformer. That would be a relatively tidy way of implementing this. I am also assuming that the data is not huge (e.g. not millions or billions of features), as if was, then this would be far better as a pointcloud analysis.

Not knowing the reasons for this, I have started to wonder if you are wanting to get the maximum count of each group. If this was the case, then the statisticscalculator would be better.

 

 

salvaleonrp
DanAtSafe
2 people upvoted this
  • salvaleonrp
    salvaleonrp
  • DanAtSafe
    DanAtSafe
liamfez
Influencer
Forum|alt.badge.img+34
  • liamfezInfluencer
  • Influencer
  • September 26, 2024
todd_davis wrote:

Actually, this should be the same logic used in the GroupCounter transformer. That would be a relatively tidy way of implementing this. I am also assuming that the data is not huge (e.g. not millions or billions of features), as if was, then this would be far better as a pointcloud analysis.

Not knowing the reasons for this, I have started to wonder if you are wanting to get the maximum count of each group. If this was the case, then the statisticscalculator would be better.

 

 

I was thinking the same, that perhaps the statisticscalculator may do what is desired.

not a bot
salvaleonrp
1 person upvoted this
  • salvaleonrp
    salvaleonrp
hkingsbury
Celebrity
Forum|alt.badge.img+53
  • hkingsburyCelebrity
  • Celebrity
  • September 26, 2024

Are you able to share some example data?

salvaleonrp
1 person upvoted this
  • salvaleonrp
    salvaleonrp
salvaleonrp
Enthusiast
Forum|alt.badge.img+15
  • salvaleonrpEnthusiast
  • Author
  • Enthusiast
  • September 26, 2024
todd_davis wrote:

The error you are getting with the counter restarting is due to your setting “When Group changes”. It needs to be “when all features received” because you groups are not in order. E.g. -1 is one of the first groups, then a new group of 2420 arrives, and then a “new group” or -1 arrives.

 

That is the first bit, I’ll have a think about the second. I do already have a quicker way, but like you say “dozen of attributes”, so there has to be a better way than my initial thought.

 

Explained in my added notes that the Table preview is the desired outcome.

salvaleonrp
Enthusiast
Forum|alt.badge.img+15
  • salvaleonrpEnthusiast
  • Author
  • Enthusiast
  • September 26, 2024
todd_davis wrote:

The error you are getting with the counter restarting is due to your setting “When Group changes”. It needs to be “when all features received” because you groups are not in order. E.g. -1 is one of the first groups, then a new group of 2420 arrives, and then a “new group” or -1 arrives.

 

That is the first bit, I’ll have a think about the second. I do already have a quicker way, but like you say “dozen of attributes”, so there has to be a better way than my initial thought.

 

Explained in my Added Notes that the Table preview shows the desired outcome. Really looking for an alternate solution using AttributeCreator that will produce the same result as the Counter for many attributes.  

salvaleonrp
Enthusiast
Forum|alt.badge.img+15
  • salvaleonrpEnthusiast
  • Author
  • Enthusiast
  • September 26, 2024
hkingsbury wrote:

Are you able to share some example data?

Here’s the sample data. There is no need to sort the TestDate.

 

 

1 Attachments
hkingsbury
1 person upvoted this
  • hkingsbury
    hkingsbury
hkingsbury
Celebrity
Forum|alt.badge.img+53
  • hkingsburyCelebrity
  • Celebrity
  • Best Answer
  • September 26, 2024

You can use a AttributeCreator with Adjacent Features enabled

Then use a conditional statement to check if the previous features value equals the same as the current feature. Then the value will be the previous features count plus one if they’re the same. If the are not the same, set the count value to one

 

The one down side of this approach, is that FME thinks the count attribute doesn’t exist (it kinda does/doesn’t).

To fix this, you can put an AttributeExposer in front of the AttributeCreator to expose the attribute

 

nielsgerrits
todd_davis
salvaleonrp
3 people upvoted this
  • nielsgerrits
    nielsgerrits
  • todd_davis
    todd_davis
  • salvaleonrp
    salvaleonrp
todd_davis
Supporter
Forum|alt.badge.img+22
  • todd_davisSupporter
  • Supporter
  • September 26, 2024

Like this??

This might not be exactly what you want, but I believe you can achieve what you want with adjacent features in attributecreator or attributemanager

1 Attachments
salvaleonrp
1 person upvoted this
  • salvaleonrp
    salvaleonrp
salvaleonrp
Enthusiast
Forum|alt.badge.img+15
  • salvaleonrpEnthusiast
  • Author
  • Enthusiast
  • September 26, 2024
hkingsbury wrote:

You can use a AttributeCreator with Adjacent Features enabled

Then use a conditional statement to check if the previous features value equals the same as the current feature. Then the value will be the previous features count plus one if they’re the same. If the are not the same, set the count value to one

 

The one down side of this approach, is that FME thinks the count attribute doesn’t exist (it kinda does/doesn’t).

To fix this, you can put an AttributeExposer in front of the AttributeCreator to expose the attribute

 

I tried this approach orginally but I was comparing attribute values (cached) instead of the attribute.

 

The Incomplete Transformer for feature[-1].count is an issue I’ve reported before. Good reminder for me to check on that issue.

 

Thanks to all responses.

Reply


Helpful Members This Week

Community Stats

31,111
Posts
117,959
Replies
38,867
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.