I am fairly new to FME and I have been trying to find a workflow to find points within polygons. My point data is dense and contains over 84 million points. I am trying to find all the points that fall within a given polygon for polygons. I have tried using PointOnAreaOverlay, FeatureReader, SpatialFilter, and Clipper. They all take a couple of hours to run. If I use ArcGIS Desktop and do a Select By Location it completes in less than 30 seconds. Is there a methodology in FME that could produce fairly quick results? The data is currently in a File Geodatabase but could be converted.
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
+17- Contributor
I would use this approach:
Read the polygon.
Assuming it is a complex polygon, replace with a bounding box (BoundingBoxReplacer).
Use the BBox as the initiator for the FeatureReader to read the points.
Then finish with the Clipper (using the original polygon as Clipper and the read points as Clippee).
Changing the reader did the trick. I'll do some testing with different formats to see if it makes any difference.
+6- Contributor
markatsafe wrote:
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
I am trying to use a Geodatabase reader (to read the polygons) and a FeatureReader (to read the points) Then use PointOnAreaOverlay to get the count of points in each polygon, however I am getting all rejected. It works when using Geodatabase reader to read the points also, but extremely slow.
Both points and polygons are in same coord system. There are 6 million points and 4000 polygons.
Any idea? I have also tried to use an SQLCreator and VertexCreator, but the output from PointOnAreaOverlay goes to Rejected in this case also.
sunsilk11 wrote:
@MarkAtSafe
I am trying to use a Geodatabase reader (to read the polygons) and a FeatureReader (to read the points) Then use PointOnAreaOverlay to get the count of points in each polygon, however I am getting all rejected. It works when using Geodatabase reader to read the points also, but extremely slow.
Both points and polygons are in same coord system. There are 6 million points and 4000 polygons.
Any idea? I have also tried to use an SQLCreator and VertexCreator, but the output from PointOnAreaOverlay goes to Rejected in this case also.
If you use the PointOnareaOverlayer, your essentially doing the spatial query twice - once in th FeatureReader and again in the PointOnAreaOverlayer
+6- Contributor
markatsafe wrote:
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
Do you mean the polygon fc is the initiator for the FeatureReader? I had used a Creator.
I need different counts based on attributes queries on the point fc. I have used PointOnAreaOverlay and listHistogram to get different counts. But your suggestion seems simpler once the PolyId is appended to point fc. Would this perform better?
On the otherhand SqlCreator work to read the point data.
+6- Contributor
markatsafe wrote:
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
I am getting reader format GEODATABASE_SDE' does not support querying by spatial predicate, but supports envelope queries. Any ideas?
I am using Geodatabase SDE format and reading from a spatial view in Oracle.
fme-featreader.png
sunsilk11 wrote:
@MarkAtSafe
I am getting reader format GEODATABASE_SDE' does not support querying by spatial predicate, but supports envelope queries. Any ideas?
I am using Geodatabase SDE format and reading from a spatial view in Oracle.
fme-featreader.png
Reply
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.