FME Separate duplicate values from attributes

As a work around, I've written the duplicate output to an excel file then with a second Attribute Filter, imported this list of duplicate values to filter them out of the Road network. I'm sure there's a better way out there though.

Hi @eltigre333, the Counter might help you. If you set the 'Road Name' attribute to the 'Counter Name' parameter of the Counter, it generates sequential numbers starting with an identical number (Count Start) for each set of features grouped by the road name.

Then you can use the combination of road name + count number as the unique ID for each segment.

Thanks @takashi ! I'd tried the counter but clearly I hadn't entered the counter name correctly…

Further to this, is there a way to join these commonly named roads to unique Asset ID in an excel file?

I've been using Joiner without success so far, even with Attribute Creator before hand.

Here's an example of the spreadsheet I'd like to join to the Road Lines.

The input road line features would have for example two unique Roads both called 'Road A'. How can I tell FME which Road joins to Asset_No A1 or A2?

I'm guessing it can't without some sort of unique ID's on the feature class?

Asset_IDAsset_NoRIDSegmentA1-10A1Road A10A1-20A1Road A20A1-30A1Road A30A2-10A2Road A10A2-20A2Road A20A2-30A2Road A30

I'm not sure what the 'join' means. Can you post your desired table that should be created from the sample?

Apologies if not clear. The above table is the desired output once the road polylines are joined to the excel schema. Two roads called "Road A" must be joined to the attributes in table above and be given the attributes of either Asset_No A1 or A2.

Thanks P

Seems that the Asset_No is the first part of the Asset_ID. If the Asset_ID is given by the source dataset, you can split the value (A1-10, ... A2-10, ...) by hyphen and assign the first part (A1, A2) to the Asset_No field. The AttributeSplitter may be a quick way to split the Asset_ID.

Thanks but not what I need. As per original post, road feature class has just the one attribute, road name. Trying to merge roads with excel schema as in table above

it's necessary to clarify these points to get the solution.

• The rules for differentiating roads belonging to an asset group from roads belonging to another asset group
• The naming rules for the Asset_ID and Asset_No

Thanks for your persistence with my question.

The only commonality between the feature class and the schema is the "Road Name" attribute, hence why I don't think it can be done?!

Literally, two (or more) Roads with the same name need to be paired to different Asset_ID attributes where no rules other than Road Name exist.

I tried the Joiner transformer to Join on road name first, using Match ALL and using the "fields to add" feature - added the Asset_ID field, then added another Joiner transformer using both Road Name and Asset_ID to join on and Match First.

I used this approach successfully with the unique road names (just reversing the cardinality options between the first and second Joiner transformers, i.e., Joiner 1 - Match First, Joiner 2 - Match ALL) but with the duplicate road names I get of course multiple matches of both Asset_ID's.

Here's an example:

joiner-multi-segment.png

joiner-multi-segment-2.png

dual-joiner-output.png

Perhaps from here I can simply use a Duplicate Filter?!

All else fails I'm going to have to manually amend road names and excel schema to create some differentiation. For example, Road A North, Road A South.

The road segments belonging to an identical Asset_No have the same name and also form a continuous polyline? Perhaps is this the rule to make groups of road segments? If so, there should be a spatial approach to resolve this.

That could be the case where road name exists in two towns but there are definitely many instances where a road is broken by a park or other feature so therefore not a continuous poly line and creating two instances of the road name in feature class, i.e. Road A (1) merge to segment 10 and 20, Road A (2) merge to segment 30, of the same Asset_No

Hi, ... honestly I'm confused.

In the example, the six road segments have the same name 'Road A', and they don't have any attribute other than name. I therefore cannot find any way (reason) to classify them.

Why the first 3 segments should be differentiated from the subsequent 3 segments?

Yes, as per my previous statement "The only commonality between the feature class and the schema is the "Road Name" attribute, hence why I don't think it can be done?!"

Q. "Why the first 3 segments should be differentiated from the subsequent 3 segments?"

A. Because in some instances they are different (unique) roads and require merging with a unique Asset_ID with accompanying attributes, like road surface type, segment lengths, etc.

I can see now that I'll probably have to further split these commonly named roads into two groups and process separately. Group 1 - unique roads with same name. Group 2 - roads that have multi part poly lines that are named alike.

Thanks for your help so far. I'll do some more experimenting and post here if I figure anything out...

@eltigre333, I would recommend to first pre-process your road table to assign unique IDs based on road alignments or routes, so that segments of Road A in one jurisdiction can be differentiated with Road A segments in the other.

Many transportation agencies uses LRS to manage their data using the route-measure concept.