Skip to main content
Question

cleaning GPS line work

  • August 3, 2016
  • 22 replies
  • 121 views
mapper
Contributor
Forum|alt.badge.img+2

Hello

I am trying to clean up some GPS line work before it is loaded into our database. THe lines need to be free of self intersections. I have come close to accomplishing this by buffering the line and then using the centreline replacer to create a smoothed line. The problem is that the centreline replacer is creating short dangles along the length of the line. The actual centreline it is creating is good enough for my purposes I just need to remove the dangles. If anyone has any ideas on how to do this it would be much appreciated.

In this screen shot the Green line is the original line with self intersections and the red line is the result of the centreline replacer.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

22 replies

ciarab
Contributor
Forum|alt.badge.img+9
  • ciarabContributor
  • Contributor
  • 167 replies
  • August 3, 2016

Have you tried the GeometryValidator already? That should correct your self intersecting lines if you set the attempt repair to yes..

mapper
Contributor
Forum|alt.badge.img+2
  • mapperContributor
  • Author
  • Contributor
  • 22 replies
  • August 3, 2016

Have you tried the GeometryValidator already? That should correct your self intersecting lines if you set the attempt repair to yes..

Yes I tried Geometry validator. It didn't catch all self intersection and failed to repair several that it did catch

david_r
Celebrity
  • david_rCelebrity
  • 8394 replies
  • August 4, 2016

Maybe try the Generalizer? There is also the SpikeRemover.

mark2atsafe
Safer
Forum|alt.badge.img+56
  • mark2atsafeSafer
  • Safer
  • 2554 replies
  • August 4, 2016

You could run the centreline through a Deaggregator and then try to get rid of the dangles in some way. Maybe test their length and throw away features less than x in length. Reducing the buffer size would probably help in this scenario.

But still, I would like to clean up the original data, not create a new centreline from it. But I think we'd need the data to do that. I'd like to know why the GeometryValidator isn't identifying or removing all of the self-intersections.

FME Evangelist to the Rich and Famous!!
mapper
Contributor
Forum|alt.badge.img+2
  • mapperContributor
  • Author
  • Contributor
  • 22 replies
  • August 4, 2016

I was able to get the Geometry Validator to catch all the self intersections by all checking off "Contains NAN" and "Degenerate or Corrupt Geometries" as well as "self intersects in 2D" but it still doesn't seem to be repairing the self intersections. The screenshot below shows the same feature as the first screenshot as it came out of the geom validator repaired port.

mapper
Contributor
Forum|alt.badge.img+2
  • mapperContributor
  • Author
  • Contributor
  • 22 replies
  • August 4, 2016

You could run the centreline through a Deaggregator and then try to get rid of the dangles in some way. Maybe test their length and throw away features less than x in length. Reducing the buffer size would probably help in this scenario.

But still, I would like to clean up the original data, not create a new centreline from it. But I think we'd need the data to do that. I'd like to know why the GeometryValidator isn't identifying or removing all of the self-intersections.

When I use the deaggregator The "main" centreline gets split into many short segments many of which are shorter than the dangles so filter the dangle out by length isn't working. Is there any way to get the centreline replacer to only generate one line down the centre of the polygon?

 

 

itay
Supporter
Forum|alt.badge.img+18
  • itaySupporter
  • Supporter
  • 1442 replies
  • August 4, 2016

Have you tried several iterations of the GeometryValidator?

mapper
Contributor
Forum|alt.badge.img+2
  • mapperContributor
  • Author
  • Contributor
  • 22 replies
  • August 4, 2016

Have you tried several iterations of the GeometryValidator?

I just tried running it through 2 geometry validators. On the second iteration they all pass the validation check even though there is still an visually obvious self intersection in the line.

 

 

ciarab
Contributor
Forum|alt.badge.img+9
  • ciarabContributor
  • Contributor
  • 167 replies
  • August 4, 2016

Have you tried the generalizer as suggested by david_r ? I suppose it depends on how much you want to compromise your original data but I think it would work and remove the 2d intersects by straightening out the line shown in your original dataset in green.

You could do minimal generalisation on all data and then extreme generalisation on those that fail 2d intersection. Or only generalise the lines that fail?

mark2atsafe
Safer
Forum|alt.badge.img+56
  • mark2atsafeSafer
  • Safer
  • 2554 replies
  • August 5, 2016
When I use the deaggregator The "main" centreline gets split into many short segments many of which are shorter than the dangles so filter the dangle out by length isn't working. Is there any way to get the centreline replacer to only generate one line down the centre of the polygon?

 

 

Right, I was thinking if you made the buffer really small then the dangles would be smaller than any short segments in the main centreline. Anyway, I'm going to file some enhancement requests with our developers - they won't help right now, but hopefully will help others in the future.

 

 

FME Evangelist to the Rich and Famous!!
mapper
Contributor
Forum|alt.badge.img+2
  • mapperContributor
  • Author
  • Contributor
  • 22 replies
  • August 5, 2016

Have you tried the generalizer as suggested by david_r ? I suppose it depends on how much you want to compromise your original data but I think it would work and remove the 2d intersects by straightening out the line shown in your original dataset in green.

You could do minimal generalisation on all data and then extreme generalisation on those that fail 2d intersection. Or only generalise the lines that fail?

This method is working for most of my gps lines. I ran the features that failed validation through a generalizer to remove self intersections and then a second generalizer with smoothing algorithm. The smoothing algorithm makes the line work a bit more cartographically pleasing as it is very jagged after the first generalizer. the last step is to run the features through another validator to check for any remaining self intersections, any features that fail the second validation are edited by hand. This process results in very few manual edits and the generalized line work is still with in the error of the original GPS line work.

 

 

ciarab
Contributor
Forum|alt.badge.img+9
  • ciarabContributor
  • Contributor
  • 167 replies
  • August 5, 2016
This method is working for most of my gps lines. I ran the features that failed validation through a generalizer to remove self intersections and then a second generalizer with smoothing algorithm. The smoothing algorithm makes the line work a bit more cartographically pleasing as it is very jagged after the first generalizer. the last step is to run the features through another validator to check for any remaining self intersections, any features that fail the second validation are edited by hand. This process results in very few manual edits and the generalized line work is still with in the error of the original GPS line work.

 

 

Great ! Sounds like success

 

 

itay
Supporter
Forum|alt.badge.img+18
  • itaySupporter
  • Supporter
  • 1442 replies
  • August 6, 2016
I just tried running it through 2 geometry validators. On the second iteration they all pass the validation check even though there is still an visually obvious self intersection in the line.

 

 

Any chance it is a 3D intersection?

 

 

N

You could run the centreline through a Deaggregator and then try to get rid of the dangles in some way. Maybe test their length and throw away features less than x in length. Reducing the buffer size would probably help in this scenario.

But still, I would like to clean up the original data, not create a new centreline from it. But I think we'd need the data to do that. if you need help related to the powerpoint go to the Ms powerpoint customer service

mark2atsafe
Safer
Forum|alt.badge.img+56
  • mark2atsafeSafer
  • Safer
  • 2554 replies
  • July 4, 2018

I wanted to add a new answer to this question. In 2019 the CentreLineReplacer has a new mode for creating the approximate centre line. With the data I tried it on, the difference looks like this:

So no more odd branching when replacing the centreline of a long, twisting polygon.

NB: it's in 2019, so needs the beta build of FME for now.

FME Evangelist to the Rich and Famous!!
david_r
Celebrity
  • david_rCelebrity
  • 8394 replies
  • July 4, 2018

I wanted to add a new answer to this question. In 2019 the CentreLineReplacer has a new mode for creating the approximate centre line. With the data I tried it on, the difference looks like this:

So no more odd branching when replacing the centreline of a long, twisting polygon.

NB: it's in 2019, so needs the beta build of FME for now.

That's a significant improvement, bravo!
takashi
Celebrity
  • takashiCelebrity
  • 7843 replies
  • July 4, 2018

I wanted to add a new answer to this question. In 2019 the CentreLineReplacer has a new mode for creating the approximate centre line. With the data I tried it on, the difference looks like this:

So no more odd branching when replacing the centreline of a long, twisting polygon.

NB: it's in 2019, so needs the beta build of FME for now.

Amazing! Thanks for the enhancement.

 

 

Why not inspect features with Visual/Data Preview and Feature/Record Information before writing them into a destination dataset?
gio
Contributor
Forum|alt.badge.img+15
  • gioContributor
  • Contributor
  • 2252 replies
  • July 5, 2018

@mapper

This works on simple selfintersections (if you intend to remove the offending bit)

gio
Contributor
Forum|alt.badge.img+15
  • gioContributor
  • Contributor
  • 2252 replies
  • July 5, 2018

@mapper

This works on simple selfintersections (if you intend to remove the offending bit)

 

her is a pic

 

gio
Contributor
Forum|alt.badge.img+15
  • gioContributor
  • Contributor
  • 2252 replies
  • July 5, 2018

I wanted to add a new answer to this question. In 2019 the CentreLineReplacer has a new mode for creating the approximate centre line. With the data I tried it on, the difference looks like this:

So no more odd branching when replacing the centreline of a long, twisting polygon.

NB: it's in 2019, so needs the beta build of FME for now.

hmm, got to try it. See if it performs on large and complex sets.

 

 

I made my own cl replacer, using only fme transformers. But it is big, has 4 levels of customs and takes a lot o process time and memory. (but at least does not crash and create those skelly lines)

 

 

jwe
Participant
Forum|alt.badge.img
  • jweParticipant
  • Participant
  • 7 replies
  • August 7, 2018

I wanted to add a new answer to this question. In 2019 the CentreLineReplacer has a new mode for creating the approximate centre line. With the data I tried it on, the difference looks like this:

So no more odd branching when replacing the centreline of a long, twisting polygon.

NB: it's in 2019, so needs the beta build of FME for now.

Hi @Mark2AtSafe, that looks very promising. I know there's been a few comments that the beta should have been available at the end of July, is there any update as to when this will be released?

 

mark2atsafe
Safer
Forum|alt.badge.img+56
  • mark2atsafeSafer
  • Safer
  • 2554 replies
  • August 7, 2018
Hi @Mark2AtSafe, that looks very promising. I know there's been a few comments that the beta should have been available at the end of July, is there any update as to when this will be released?

 

We're trying for sometime this week (so by the end of 10th August). I can't guarantee, but it's almost certainly just a few days rather than weeks. We wanted it to come after 2018.1 released, and that got delayed, hence the delay in the 2019 beta.

 

FME Evangelist to the Rich and Famous!!

Community Stats

32,084
Posts
121,841
Replies
39,724
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.