I am in the process of aggregating some polygons, then joining these simplified polygons (geometry) with sensor data (no geometry) on a common field. It doesn't matter whether I am using the FeatureJoiner, or the FeatureMerger my process is taking way longer than what I know FME is capable of doing. I purged the temp data, as well as turned off feature caching to no avail. Any ideas or suggestions would be appreciated.
Userlevel 6
+32
- Evangelist
- 2324 replies
-
10 February 2021
Just to be sure, is FeatureCaching on or off?
Userlevel 1
+10
- Participant
- 3078 replies
-
10 February 2021
Where is the non geometry sensor data coming from and how big is it? Perhaps there is an option to use a databasejoiner to avoid having to read all the data in?
+14
- Contributor
- 56 replies
-
10 February 2021
Where is the non geometry sensor data coming from and how big is it? Perhaps there is an option to use a databasejoiner to avoid having to read all the data in?
It's only about 15k records. I pushed it from SQL Server into MS Access. I had another process doing that for multiple sensors. It is running, just not at the speed I would assume it should. It is joining one polygon and making 15k polygons, so I change it to a temporal layer. I have tested the process, and the result works, but it is just a dog on a system with 64Gb of RAM. FME shouldn't even blink at this process. I also upgraded the FeatureMerger transformer to the latest version, but it didn't help the situation.
+14
- Contributor
- 56 replies
-
10 February 2021
Are you using suppliers first?
No. I could get it to work, but I had to break the processing into separate workbench files, or I should say a fresh workbench file. It was very odd. Either the data was corrupted, or the workbench file, I think. Adding more time based filtering at some point soon. I will have to walk a little slower in my testing.
Userlevel 1
+11
- Enthusiast
- 60 replies
-
11 February 2021
It's only about 15k records. I pushed it from SQL Server into MS Access. I had another process doing that for multiple sensors. It is running, just not at the speed I would assume it should. It is joining one polygon and making 15k polygons, so I change it to a temporal layer. I have tested the process, and the result works, but it is just a dog on a system with 64Gb of RAM. FME shouldn't even blink at this process. I also upgraded the FeatureMerger transformer to the latest version, but it didn't help the situation.
Out of curiousity, what version of FME are you using?
Reply
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.