I am fairly new to FME and I have been trying to find a workflow to find points within polygons. My point data is dense and contains over 84 million points. I am trying to find all the points that fall within a given polygon for polygons. I have tried using PointOnAreaOverlay, FeatureReader, SpatialFilter, and Clipper. They all take a couple of hours to run. If I use ArcGIS Desktop and do a Select By Location it completes in less than 30 seconds. Is there a methodology in FME that could produce fairly quick results? The data is currently in a File Geodatabase but could be converted.
Question
Performance Finding Points Within Polygons
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
+26- Contributor
I would use this approach:
Read the polygon.
Assuming it is a complex polygon, replace with a bounding box (BoundingBoxReplacer).
Use the BBox as the initiator for the FeatureReader to read the points.
Then finish with the Clipper (using the original polygon as Clipper and the read points as Clippee).
Changing the reader did the trick. I'll do some testing with different formats to see if it makes any difference.
+6- Contributor
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
I am trying to use a Geodatabase reader (to read the polygons) and a FeatureReader (to read the points) Then use PointOnAreaOverlay to get the count of points in each polygon, however I am getting all rejected. It works when using Geodatabase reader to read the points also, but extremely slow.
Both points and polygons are in same coord system. There are 6 million points and 4000 polygons.
Any idea? I have also tried to use an SQLCreator and VertexCreator, but the output from PointOnAreaOverlay goes to Rejected in this case also.
@MarkAtSafe
I am trying to use a Geodatabase reader (to read the polygons) and a FeatureReader (to read the points) Then use PointOnAreaOverlay to get the count of points in each polygon, however I am getting all rejected. It works when using Geodatabase reader to read the points also, but extremely slow.
Both points and polygons are in same coord system. There are 6 million points and 4000 polygons.
Any idea? I have also tried to use an SQLCreator and VertexCreator, but the output from PointOnAreaOverlay goes to Rejected in this case also.
If you use the PointOnareaOverlayer, your essentially doing the spatial query twice - once in th FeatureReader and again in the PointOnAreaOverlayer
+6- Contributor
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
Do you mean the polygon fc is the initiator for the FeatureReader? I had used a Creator.
I need different counts based on attributes queries on the point fc. I have used PointOnAreaOverlay and listHistogram to get different counts. But your suggestion seems simpler once the PolyId is appended to point fc. Would this perform better?
On the otherhand SqlCreator work to read the point data.
+6- Contributor
A combination of the Geodatabase reader (to read the polygons) and FeatureReader (to read the points) should do the trick. Your point data needs to be in a FME format that supports a spatial index. these include:
- Esri Geodatabase (File Geodb ArcObjects) reader - but not the Esri Geodatabase (File Geodb API) reader
- Shape file, with a spatial index, may also improve the performance of the point data read as reading Shape is generally faster than the Esri Geodatabase (File Geodb ArcObjects) reader
- Loading you point data into Oracle, SQL Server or PostGIS with a suitable spatial index.
I am getting reader format GEODATABASE_SDE' does not support querying by spatial predicate, but supports envelope queries. Any ideas?
I am using Geodatabase SDE format and reading from a spatial view in Oracle.
fme-featreader.png
@MarkAtSafe
I am getting reader format GEODATABASE_SDE' does not support querying by spatial predicate, but supports envelope queries. Any ideas?
I am using Geodatabase SDE format and reading from a spatial view in Oracle.
fme-featreader.png