Our data is in the cloud and there is possibility for increasing the performance of our network disks on the cloud. However doing so, will lead to much higher costs. So if the performances of the fme scripts are mostly determined by internal memory of the computer, then maybe we wouldn't need to increase the performance of the cloud services.
Question
To which extent are performances of FME scripts determined by network disks or internal memory of the computer?
+63
It's hard to say without knowing more about the kind of workspaces (and data) you're using.
Reading and writing large datasets can certainly be a performance bottleneck and those kinds of processes would certainly benefit from faster drives. On the other hand, heavy geometry processes (especially spatial analysis) and large data joins or statistics calculations could benefit more from having more RAM.
One thing you can look for is at the end of every log file: FME reports the maximum amount of RAM it needed for that process (Peak Process Memory), if that is well below the amount of RAM available on that machine there's no immediate need to increase it. Also check for the log message Optimizing Memory Usage, that indicates FME needs to start swapping data to temporary files which will slow things down.
On FME Server you'll see that it maintains statistics per workspace about the average running time and memory use, again that may be useful to determine if that needs to be increased or not.
+15- Enthusiast
Thank you for the elaborative answer. it's true what you said, it's determined by several factors: the data you have (size of it) and what kind of transformations you do.
I will make use of the tools you mentioned to see what kind of bottlenecks we face.
+73- Celebrity
It's hard to say without knowing more about the kind of workspaces (and data) you're using.
Reading and writing large datasets can certainly be a performance bottleneck and those kinds of processes would certainly benefit from faster drives. On the other hand, heavy geometry processes (especially spatial analysis) and large data joins or statistics calculations could benefit more from having more RAM.
One thing you can look for is at the end of every log file: FME reports the maximum amount of RAM it needed for that process (Peak Process Memory), if that is well below the amount of RAM available on that machine there's no immediate need to increase it. Also check for the log message Optimizing Memory Usage, that indicates FME needs to start swapping data to temporary files which will slow things down.
On FME Server you'll see that it maintains statistics per workspace about the average running time and memory use, again that may be useful to determine if that needs to be increased or not.
One thing we've found with storage, is its not necessarily the sustained write speed that matters (to an extent) but more the number of IOPs (input/output operations per second) the storage can perform
Login to the community
No account yet? Create an account
An FME Account is required to contribute
LoginEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.