Is there a better way of doing this with a process that does not take so much time to run given that I have a large number of features to process.
I have a layer of around 21m features and I want to group them geographically using a region layer which has around 10 regions. I have used the spatial relator which I think is working but it has been running now for 17 hours and has only processed 687,000.
Page 1 / 1
Hi,
If your data resides in a database, you should make use of it and its spatial capabilities. So for example make use of the regions features to trigger the database (via a FeatureReader) to return features only intersecting the region trigger.
Optionally make use of a bounding box to limit the amount of features entering the database.
The issue with reading so much data into FME and spatially relating it is that FME creates a spatial index to claculate the spatial relationship, and with a great number of reatures it can result in a very long process.
Many thanks for this.
I need to process all the features but tag them with the region name so I can then fanout the resulting shapefile by region name.
I dont need to filter out features that are not in a single region. All 21m features will sit in one of the 10 or so regions. They are all in the UK and the regions divide the UK into regions.
Hi,
it always an interesting challenge to optimize performance when such huge datasets are involved...
For polylines, I'd consider strong generalization of the geometry before the SpatialRelator. For polygons, I'd try the HullReplacer to simplify them.
You can use the GeometryExtractor and GeometryReplacer to "backup" your original geometry before the generalization and restore it after.
David
The FeatureReader, will tag it for you, have a look at the attribute settings.
I managed to resolve it by using a CentrePointReplacer --> PointOnAreaOverlayer using the regions as the Area --> FeatureMerger using the starting polygons as the requester and the new points as the supplier.
Many thanks for you suggestions.
Hi,
Just be aware that the point created by the CenterPointReplacer may be outside of the original feature.
http://docs.safe.com/fme/html/FME_Transformers/Default.htm#Transformers/centerpointreplacer.htm
According to your solution, I guess that relationship of the regions and the target features can be assumed as 1 : N. If so, maybe you can use the SpatialFilter effectively without geometry replacing and feature merging. In some cases, the SpatialFilter could be much faster than the SpatialRelator.
Takashi
Takashi is right, the CenterPointReplacer might(!) give wrong results for some geometries.
Another option could be to use the CoordinateExtractor to get one of the polyline vertices into a 2DPointReplacer and use that instead.
Regarding performance for the SpatialFilter vs the SpatialRelator, look at the help text for the SpatialFilter for hints about when to use which.
David
Hi,
Both the Spatial relator and Spatial Filter requiere all features to be read into the ws and to spatially relate (build index), with the FeatureReader the features are read and related at the same time, also the underlaying database is doing the work instead of FME.