Skip to main content

I have a workbench that I regularly use to read from an Oracle Locator database and write to an ESRI file geodatabase. It writes about 55 feature classes to this geodatabase (approximately 8 million features total) and takes about 90 minutes to complete.

I have just had an SDE made available to me so I made a copy of my original file gdb workbench and have been changing it to write to the SDE instead. The problem I'm having is the workbench now writes extremely slowly; it took 90 minutes to write a single feature class with 168,000 features in it (at that rate I estimate it would take two or three days to write all 8 million features to the SDE).

Has anybody experienced anything like this or have any idea what I can do to speed it up? If it makes a difference I believe the SDE is running on some version of SQL Server.

Thanks,

Mike

Hi @michael_m,

 

I'm not sure if you still have this problem, however, there is this article which might be useful: https://knowledge.safe.com/articles/165/improving-performance-when-working-with-esri-geoda.html

 

 

In General I would say SDE is a bit slower than File Geodatabase, however, what you are describing is painful. One thing which you can check in the logfile is the difference between CPU usage vs FME Session duration. If the CPU is very low compared to Session Duration then it means FME is waiting. This could be due to slow network or a particular configuration of the SDE itself (e.g., versioning) or the writer configuration (transaction type).

 

 

Another thing which I've seen is due to limited permissions - As I remember there were cases where the Feature type had it's table handling to Truncate Existing, however, the user didn't have permission to truncate so and edit session was started and each feature was deleted one by one....This doesn't sound like you though because from the sounds of it you are just trying to create them and fill them up.

 

 

If you are really not making progress then I would get as much information about your environment as possible form IT (FME version, 64 bit vs 32 bit, ESRI version, underlying Database version etc) and send a reproduction to Safe Software along with the logfiles and workspace you have. They can try and reproduce your issue or, at least, rule FME out of the picture.

 

 

Good luck.

Hi @michael_m,

 

I'm not sure if you still have this problem, however, there is this article which might be useful: https://knowledge.safe.com/articles/165/improving-performance-when-working-with-esri-geoda.html

 

 

In General I would say SDE is a bit slower than File Geodatabase, however, what you are describing is painful. One thing which you can check in the logfile is the difference between CPU usage vs FME Session duration. If the CPU is very low compared to Session Duration then it means FME is waiting. This could be due to slow network or a particular configuration of the SDE itself (e.g., versioning) or the writer configuration (transaction type).

 

 

Another thing which I've seen is due to limited permissions - As I remember there were cases where the Feature type had it's table handling to Truncate Existing, however, the user didn't have permission to truncate so and edit session was started and each feature was deleted one by one....This doesn't sound like you though because from the sounds of it you are just trying to create them and fill them up.

 

 

If you are really not making progress then I would get as much information about your environment as possible form IT (FME version, 64 bit vs 32 bit, ESRI version, underlying Database version etc) and send a reproduction to Safe Software along with the logfiles and workspace you have. They can try and reproduce your issue or, at least, rule FME out of the picture.

 

 

Good luck.

Thank you, @virtualcitymatt. It did turn out to be the setting for "Transaction Type". I changed that from "Versioned Edit Session" to "Transaction" and the overall write time fell back to about two hours.


Reply