I am in the process of aggregating some polygons, then joining these simplified polygons (geometry) with sensor data (no geometry) on a common field. It doesn't matter whether I am using the FeatureJoiner, or the FeatureMerger my process is taking way longer than what I know FME is capable of doing. I purged the temp data, as well as turned off feature caching to no avail. Any ideas or suggestions would be appreciated.
+44- Influencer
- 3427 replies
Where is the non geometry sensor data coming from and how big is it? Perhaps there is an option to use a databasejoiner to avoid having to read all the data in?
+17- Contributor
- 58 replies
Where is the non geometry sensor data coming from and how big is it? Perhaps there is an option to use a databasejoiner to avoid having to read all the data in?
It's only about 15k records. I pushed it from SQL Server into MS Access. I had another process doing that for multiple sensors. It is running, just not at the speed I would assume it should. It is joining one polygon and making 15k polygons, so I change it to a temporal layer. I have tested the process, and the result works, but it is just a dog on a system with 64Gb of RAM. FME shouldn't even blink at this process. I also upgraded the FeatureMerger transformer to the latest version, but it didn't help the situation.
+17- Contributor
- 58 replies
Are you using suppliers first?
No. I could get it to work, but I had to break the processing into separate workbench files, or I should say a fresh workbench file. It was very odd. Either the data was corrupted, or the workbench file, I think. Adding more time based filtering at some point soon. I will have to walk a little slower in my testing.
+23- Enthusiast
- 83 replies
It's only about 15k records. I pushed it from SQL Server into MS Access. I had another process doing that for multiple sensors. It is running, just not at the speed I would assume it should. It is joining one polygon and making 15k polygons, so I change it to a temporal layer. I have tested the process, and the result works, but it is just a dog on a system with 64Gb of RAM. FME shouldn't even blink at this process. I also upgraded the FeatureMerger transformer to the latest version, but it didn't help the situation.
Out of curiousity, what version of FME are you using?