HI, I'm trying to use Spatial Filter Transformer to clipp the MasterMap data against the outer boundary, while running the FME workbench to clipp some 3 million records Desktop is running out of memory. Unlike Clipper, Spatial Filter does not pass the candidates to the output while its reading data from reader, Spatial Filter tries to read and cache the full data into memory which results in peak utilization @1spatialdave, I could use Clipper Transformer instead of Spatial Filter but on the output side(.DWG) i'm getting the Outer Boundary Box for each 1KM GRID(ex: SD5626.DWG) which i don't need. Could you please help me with this request.
Userlevel 4
+25
- Safer
- 2399 replies
-
11 July 2017
The SpatialFilter transformer should pass candidates to the output, but only if you set the Filter Type to Filters First and (of course) make sure that the filters really are first in the translation. It should work the same way as Clippers First in the Clipper. If it's not, then it's something we at Safe should look into.
Also, is there just a single boundary and is it just a small area of the MasterMap? If so, we'd really want to filter out the components that can't possibly match, before doing the SpatialFilter.
For example, use the Clipper to get rid of most of the data, then do a SpatialFilter on what is left. Or use a bounding box to simplify the boundary for a first pass. In the past I've fetched the bounding box coordinates and filtered data using an attribute test (eg is feature max x < boundary min x) rather than a spatial test.
Userlevel 2
+16
- Contributor
- 2180 replies
-
11 July 2017
If the Mastermap data is in a database, the FeatureReader could be used to only read the data that interacts with the boundary (or boundaries) and ignore the other data.
This way the database can do part of the filtering.
Reply
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.