Skip to main content
  • EN
Hello,

 

 

I have 3 millions points that I have to find in what districts they are localized. There is almost 200 districts to test. I am using the pointOnAreaOverlayer because it is faster than the spatialRelator.

 

 

However, it still takes a lot of time. I wonder if using the bounding box of districts in a first POAO and then a listExploder on all matchs and finally a second POAO with a GroupBy on the districts that BBOX matched, to test in what district the point is really localized could be faster?

 

 

This strategy would be similar to how spatial query are done in database when using the spatial index. Doest the POAO already use such strategy internally?

 

 

Any info about that? For now my tests are not really promising...
Hi,

 

 

Have you tried the Clipper? In my experiences, the Clipper is faster than any other transformers, to perform spatial querying for points against areas.

 

 

Takashi
I think the numbers you need to test simply is a lot.

 

So any spatial strategy wil take a lot of time.

 

 

You are referring to quad-tree indexing i assume.

 

If you have all in a spatial database  you can use it, though it is depricated. "users are encouraged to use R-tree indexing")

 

 

Maybe you should tile the proces and run it per tile using a workspacecaller.

 

 

I tried oredering them first adn using grid. FIrst bit took 1m28sec for 65407points.

 

Second bit samish. That still would be more than 1,5 hours of processing (if i scale it linearily..).

 

 
HIi,

 

Anything going into FME gets translated to the internal format and therefore is also spatially indexed. I would try the Clipper as Takashi suggested, but also the POAO with areas going in first (drag the areas reader to the top) is worth trying.

 

Doc states: ' This can reduce memory use of the PointOnAreaOverlayer if you have many Point features to overlay.'
Thank you for your answers!

 

 

It seems the deceleration is mostly caused by too much use of RAM when manipulating a large amount of features in a complex workspace containing sorters or aggregators wich must accumulate all features before processing.

 

 

Takashi: thank you for the Clipper advice. Its true that he is slightly faster than the POAO. However, you cant merge attributes in a list when the points intersect more than one polygon. I had to use the many Clippers in cascade to achieve my goal and then the speed was reduced.

 

 

Gio: Quad-tree depracted? Why and where is it depracated? In my opinion, both have their advantages depending in what goal they are created .

 

 

 
You are right. The Clipper cannot detect intersections between a point and multiple areas.  As you mentioned, the PoinOnLineOverlayer is suitable if there may be overlapped districts.
"deprecated" ....ask Oracle...it is a quote from their site.

 

 

I think it is since version 11.
Flashed. If you overlayed districts with the AreaOnAreaOverlayer beforehand, the Clipper can also be used. But, it's unknown whether the performance is better than POAO unless measuring. Possibly worse...

 

Jusa an idea.

Reply


Community Stats

30,108
Posts
114,295
Replies
37,744
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.