Question

Slow Clipper Issue

  • 4 September 2018
  • 3 replies
  • 9 views

Badge

I have a dataset of around 85k road segments and 130k non-road segments where I want to cut off small parts of the roads around the non-road segments: There's a buffer around each of the non-road segments that I use as clipper on the roads.

I've applied all the tips given in this thread ( https://knowledge.safe.com/questions/62461/why-the-clipper-transformer-is-too-slow.html ). I forced the clipper to wait until all clippers were available, I dissolved both datasets before clipping and it works fine on smaller datasets. Still, it takes around 2 seconds per feature in the dissolved dataset (total 12k). I've even discarded all attributes.

Algorithmically, clipping polygons should take nowhere near as long when everything fits in memory. [M. de Berg et al. Computational Geometry. Ch2 ]

Is there anything I've missed? Why is clipping so slow?


3 replies

Badge
Contrary to the advice given in the other thread, I've found that not dissolving the features before the clipper is faster. It now took ~90 seconds, instead of hours.

 

This may be because now the locality is preserved, so it isn't compared to all other polygons, though that is only my hypothesis, I can't see the inner workings of FME.

 

 

Userlevel 3
Badge +13
Contrary to the advice given in the other thread, I've found that not dissolving the features before the clipper is faster. It now took ~90 seconds, instead of hours.

 

This may be because now the locality is preserved, so it isn't compared to all other polygons, though that is only my hypothesis, I can't see the inner workings of FME.

 

 

Hi @arjanboogaart That other thread is already dated as we're overhauling the Dissolver now. Which build of FME are you using?

 

Userlevel 3
Badge +13

I'd also consider pumping the non-dissolved buffered non-road folks into the LineOnAreaOverlay as the areas, and the roads in as lines. That method will employ a good spatial index underneath. You shoudl be able to tell the lines that were in vs. out by checking their overlap count afterwards.

Reply