Using truncate and append with arcsde geodatabase writer . The SDE is on an orace database.Inserting 3milion rows. FME on prem takes 1 hour. but when on azure cloud it takes 8 hours. Any idea how to improve this.Any parameters that I can tweek.
Question
FME FORM in Azure cloud
+57- Influencer
Turn off the Feature Caching. Instead use inspector transformers in places you wish to inspect the outputs
Don’t expose attributes on the readers unless you have transformers that require them
Learn about other optimisation tips: Performance Tuning FME – FME Support Center
+68- Celebrity
There are a lot of considerations to take into account in a situation like this, not only FME, but also infrastructure.
Some infrastructure things to consider are:
What are the specs of each machine? (on prem/Azure) If the Azure machine is lower spec then it would be expected to take longer
What are the IOPs of the Azure disk? When FME runs out of RAM (quite possible with 3 million rows) it utilises the disk. the slower the disk, the slower FME runs
Where is the data being read from? If the source data is on prem, it’s going to be slower having to read from Azure
What is the network connectivity between Azure and the Oracle DB? If the netwrok is throttled then thats going to effect the rate that FME can write at
+47- Celebrity
Another thing to consider is the user/permissions - from memory if a particular user doesn’t have permission to truncate / perform transactions then the write may fall back to an edit session which can be dramatically slower.
I haven’t been able to find anything on that which I can point you to directly. But if you’re trying to access from a different machine I could see something like this happening.
https://support.safe.com/hc/en-us/articles/25407680189453-Improving-Performance-when-working-with-Esri-Geodatabases