Question

spatial relator alternative

  • 27 February 2014
  • 8 replies
  • 23 views
bubblebeb
Rank
Badge +1
I have a layer of around 21m features and I want to group them geographically using a region layer which has around 10 regions. I have used the spatial relator which I think is working but it has been running now for 17 hours and has only processed 687,000. 

 

Is there a better way of doing this with a process that does not take so much time to run given that I have a large number of features to process. 

8 replies

fmelizard
Rank
Userlevel 4
Badge +13
Hi,

 

If your data resides in a database, you should make use of it and its spatial capabilities. So for example make use of the regions features to trigger the database (via a FeatureReader) to return features only intersecting the region trigger.

 

Optionally make use of a bounding box to limit the amount of features entering the database.

 

 

The issue with reading so much data into FME and spatially relating it is that FME creates a spatial index to claculate the spatial relationship, and with a great number of reatures it can result in a very long process.

 

 

bubblebeb
Rank
Badge +1
Many thanks for this. 

 

I need to process all the features but tag them with the region name so I can then fanout the resulting shapefile by region name. 

 

I dont need to filter out features that are not in a single region. All 21m features will sit in one of the 10 or so regions. They are all in the UK and the regions divide the UK into regions. 
david_r
Rank
Userlevel 4
Hi,

 

 

it always an interesting challenge to optimize performance when such huge datasets are involved...

 

 

For polylines, I'd consider strong generalization of the geometry before the SpatialRelator. For polygons, I'd try the HullReplacer to simplify them.

 

 

You can use the GeometryExtractor and GeometryReplacer to "backup" your original geometry before the generalization and restore it after.

 

 

David
fmelizard
Rank
Userlevel 4
Badge +13
The FeatureReader, will tag it for you, have a look at the attribute settings.
bubblebeb
Rank
Badge +1
I managed to resolve it by using a CentrePointReplacer --> PointOnAreaOverlayer using the regions as the Area --> FeatureMerger using the starting polygons as the requester and the new points as the supplier. 

 

Many thanks for you suggestions. 
takashi
Rank
Userlevel 2
Badge +17
Hi,

 

 

Just be aware that the point created by the CenterPointReplacer may be outside of the original feature.

 

http://docs.safe.com/fme/html/FME_Transformers/Default.htm#Transformers/centerpointreplacer.htm

 

 

According to your solution, I guess that relationship of the regions and the target features can be assumed as 1 : N. If so, maybe you can use the SpatialFilter effectively without geometry replacing and feature merging. In some cases, the SpatialFilter could be much faster than the SpatialRelator.

 

 

Takashi
david_r
Rank
Userlevel 4
Takashi is right, the CenterPointReplacer might(!) give wrong results for some geometries.

 

 

Another option could be to use the CoordinateExtractor to get one of the polyline vertices into a 2DPointReplacer and use that instead.

 

 

Regarding performance for the SpatialFilter vs the SpatialRelator, look at the help text for the SpatialFilter for hints about when to use which.

 

 

David
fmelizard
Rank
Userlevel 4
Badge +13
Hi,

 

 

Both the Spatial relator and Spatial Filter requiere all features to be read into the ws and to spatially relate (build index), with the FeatureReader the features are read and related at the same time, also the underlaying database is doing the work instead of FME.

Reply


Community Stats

28,669
Posts
109,358
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.