Skip to main content
Solved

Remove duplicate points if they belong to the same group ID

  • June 28, 2018
  • 2 replies
  • 131 views
J

I have a point file containing instructions along routes, lets call this the major decision points. Each collection of points belong to a route and has a route id. This data is all contained within one file, which can be made of points belong to numerous routes.

I then have another file containing points for all junctions along the route, again this data has a route ID that is the same as the previous file for the given route. Lets call these the minor decision points and again this file is made up of points belonging to different route IDs.

The problem is that I need to remove the minor decision points when they are duplicates for major decision points, however, these should only be removed if they belong to the same route ID. To complicate maters some of these routes will overlap.

My initial idea was to create a loop transformer that processes features that belong to the same route ID, and then remove the duplicate minor decision points this way. However, I am stuck on exactly how I would achieve such a solution. Does anyone have any ideas or other potential solutions?

Best answer by takashi

Hi @jedimule, since the major decision points and the minor decision points both have a common attribute called "route ID", you can detect duplicate points for each route with the PointOnPointOverlayer grouping them by the route ID.

Enter all the "major" and "minor" points into a PointOnPointOverlayer, set "route ID" to the Group By parameter and set an appropriate tolerance to the Point Tolerance parameter (0 if you want to detect "exact" match).

The output points will have overlap count attribute. If its value was 1 or more, it indicates the point duplicates in the group.

After separating the resulting points into "major" and "minor" by seeing an attribute which indicates point type (e.g. "fme_feature_type"), you can filter out duplicate "minor" points according to the overlap count attribute.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

takashi
Celebrity
  • takashiCelebrity
  • 7843 replies
  • Best Answer
  • June 28, 2018

Hi @jedimule, since the major decision points and the minor decision points both have a common attribute called "route ID", you can detect duplicate points for each route with the PointOnPointOverlayer grouping them by the route ID.

Enter all the "major" and "minor" points into a PointOnPointOverlayer, set "route ID" to the Group By parameter and set an appropriate tolerance to the Point Tolerance parameter (0 if you want to detect "exact" match).

The output points will have overlap count attribute. If its value was 1 or more, it indicates the point duplicates in the group.

After separating the resulting points into "major" and "minor" by seeing an attribute which indicates point type (e.g. "fme_feature_type"), you can filter out duplicate "minor" points according to the overlap count attribute.

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
J
  • jedimule
  • Author
  • 7 replies
  • June 29, 2018

Hi @jedimule, since the major decision points and the minor decision points both have a common attribute called "route ID", you can detect duplicate points for each route with the PointOnPointOverlayer grouping them by the route ID.

Enter all the "major" and "minor" points into a PointOnPointOverlayer, set "route ID" to the Group By parameter and set an appropriate tolerance to the Point Tolerance parameter (0 if you want to detect "exact" match).

The output points will have overlap count attribute. If its value was 1 or more, it indicates the point duplicates in the group.

After separating the resulting points into "major" and "minor" by seeing an attribute which indicates point type (e.g. "fme_feature_type"), you can filter out duplicate "minor" points according to the overlap count attribute.

So simple! I had obviously been looking at this issue for too long and over complicating it.

Community Stats

32,085
Posts
121,845
Replies
39,726
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.