I am in the process of aggregating some polygons, then joining these simplified polygons (geometry) with sensor data (no geometry) on a common field. It doesn't matter whether I am using the FeatureJoiner, or the FeatureMerger my process is taking way longer than what I know FME is capable of doing. I purged the temp data, as well as turned off feature caching to no avail. Any ideas or suggestions would be appreciated.
+43- Influencer
Where is the non geometry sensor data coming from and how big is it? Perhaps there is an option to use a databasejoiner to avoid having to read all the data in?
+17- Contributor
ebygomm wrote:
Where is the non geometry sensor data coming from and how big is it? Perhaps there is an option to use a databasejoiner to avoid having to read all the data in?
It's only about 15k records. I pushed it from SQL Server into MS Access. I had another process doing that for multiple sensors. It is running, just not at the speed I would assume it should. It is joining one polygon and making 15k polygons, so I change it to a temporal layer. I have tested the process, and the result works, but it is just a dog on a system with 64Gb of RAM. FME shouldn't even blink at this process. I also upgraded the FeatureMerger transformer to the latest version, but it didn't help the situation.
+17- Contributor
jdh wrote:
Are you using suppliers first?
No. I could get it to work, but I had to break the processing into separate workbench files, or I should say a fresh workbench file. It was very odd. Either the data was corrupted, or the workbench file, I think. Adding more time based filtering at some point soon. I will have to walk a little slower in my testing.
+23- Enthusiast
wisegis wrote:
It's only about 15k records. I pushed it from SQL Server into MS Access. I had another process doing that for multiple sensors. It is running, just not at the speed I would assume it should. It is joining one polygon and making 15k polygons, so I change it to a temporal layer. I have tested the process, and the result works, but it is just a dog on a system with 64Gb of RAM. FME shouldn't even blink at this process. I also upgraded the FeatureMerger transformer to the latest version, but it didn't help the situation.
Out of curiousity, what version of FME are you using?
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.