I have 1 area Featureclass with 700.000 square of 5mX5m and 1 featureclass with 70.000 lines.
I want to know for each square how many times it is touched bij a line. I have tryed [SpatialRelator] but after 14 hours FME was stil comparing, amd i tryed [LineOnAreaOverlayer] but FME gives a memory error. Is there maybe another way? Someone suggested [InLineQuerier] but i see no options to check if square is touched by a line.
Thanks for your time.
Best answer by fmelizard
Hi @kees_v -- I gave this a try with simulated data with the volumes you mentioned and had quite good performance with the SpatialRelator on FME 2018. I'm curious what build of FME you were using, and how many attributes are on the feature classes? I used the Grid features as the Requestor.
I also turned off all attribute accumulation in the SpatialRelator -- the count you seek is the _related_candidates attribute that gets added. I used "Intersects" as the predicate.
If you are on 32 bit and can't go to 64, I can imagine this could cause issues. A workaround is to:
a) translate your lines to FFS -- with a spatial index
b) make a workspace with your grid as the input. Run each feature through a FeatureReader saying "Intersects" as the spatial predicate and pointing to the FFS file you just made. The "Initiator" port is what you want to catch on the output side == it will have the count of features that matched added.
c) run it with a small data volume to test -- limit the grids by adjusting the max features to read parameter in the navigator https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/reader_parameters_featuresToRead.htm (don't forget to unset this once everything looks good).
Hope this helps. Really good question here and one we want ot make ever better moving forward.
+49The SpatialRelator is what I'd try too, but it sounds like you're running into memory problems.
Can you divide the data into smaller chunks and run each of those seperately?
+28While normally I would recommend the SpatialRelator, as an alternative I would look at having the area grid features trigger a featureReader of the lines with the spatial constraints set to intersect and merging attributes, then either an Aggregator (attributes only) or StatisticsCalculator grouped by the ID of the grid feature will give you the count of the lines for that grid feature. Make sure Input is Ordered is set to yes, otherwise you'll end up the same memory errors.
+15spatial relator with calculate cardinality.
list concatenate card_list-point, no separator. Count nrs. of 1's in string. You can use string search in regexp mode for instance with regexp={1} and submatches , nr of list elements is nr of touches.
Best to tile the process as you appear to have capacity problems. Do not start tiling on a square vertex if tile size is modulus of your 5x5.
+18Hi @kees_v -- I gave this a try with simulated data with the volumes you mentioned and had quite good performance with the SpatialRelator on FME 2018. I'm curious what build of FME you were using, and how many attributes are on the feature classes? I used the Grid features as the Requestor.
I also turned off all attribute accumulation in the SpatialRelator -- the count you seek is the _related_candidates attribute that gets added. I used "Intersects" as the predicate.
If you are on 32 bit and can't go to 64, I can imagine this could cause issues. A workaround is to:
a) translate your lines to FFS -- with a spatial index
b) make a workspace with your grid as the input. Run each feature through a FeatureReader saying "Intersects" as the spatial predicate and pointing to the FFS file you just made. The "Initiator" port is what you want to catch on the output side == it will have the count of features that matched added.
c) run it with a small data volume to test -- limit the grids by adjusting the max features to read parameter in the navigator https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/reader_parameters_featuresToRead.htm (don't forget to unset this once everything looks good).
Hope this helps. Really good question here and one we want ot make ever better moving forward.
Hi @kees_v -- I gave this a try with simulated data with the volumes you mentioned and had quite good performance with the SpatialRelator on FME 2018. I'm curious what build of FME you were using, and how many attributes are on the feature classes? I used the Grid features as the Requestor.
I also turned off all attribute accumulation in the SpatialRelator -- the count you seek is the _related_candidates attribute that gets added. I used "Intersects" as the predicate.
If you are on 32 bit and can't go to 64, I can imagine this could cause issues. A workaround is to:
a) translate your lines to FFS -- with a spatial index
b) make a workspace with your grid as the input. Run each feature through a FeatureReader saying "Intersects" as the spatial predicate and pointing to the FFS file you just made. The "Initiator" port is what you want to catch on the output side == it will have the count of features that matched added.
c) run it with a small data volume to test -- limit the grids by adjusting the max features to read parameter in the navigator https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/reader_parameters_featuresToRead.htm (don't forget to unset this once everything looks good).
Hope this helps. Really good question here and one we want ot make ever better moving forward.
I have FME version 2015, translating the lines to FFS with spatial index did the tric.
Thank you very much for your help!
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
