Question

How best to 'Append' using the Esri Geodatabse (ArcSde GDB) Writer

  • 18 August 2021
  • 2 replies
  • 36 views
P

Append_exampleHi Guys, I have two feature classes in SDE and I want to append featureclass A to featureclass B and do a few additional its like dropping and recreating indexes and setting a value on newly appended data.

I have this working but very very slowly, so i assume I have taken the wrong approach. ?

I have an ESRI SDE r eader, connected to the fetaureclass A, and sets max features to read per feature type 3000000, I have Sql to run before read- which sets an attribute with date and time of now so I can track the records later.

I then use coordinate setter and attribut manager and connect to a ESRI SDE writer, I have set this on Transact mode with 0 Transaction Number and 100000 Transaction and Features Per Transaction to be 100000 and Commit at end of Write to be NO . I drop indexes in Sql Before write and recreate them afer write .. potentially for faster loading. It is incredibly slow - around 100000 feature every half hour. (2.4 million rows to process) I can only assume this is not the way to do it? or have I set parameters wrong ? Any help is hugely appreciated .. I've tried this so many ways nw I'm loosing track. My database is Oracle and my Arcgis level is Pro 2.7.1 Advanced.

Paul

2 replies

virtualcitymatt
Rank
Userlevel 4
Badge +26

From memory it's just kinda slow, but I think it shouldn't be this slow. One thing that I have heard is make sure you have appropriate permissions to perform changes to the feature class using transactions. I have heard that there can be issues of permissions which lead to FME switching to an edit session instead of using transactions.

 

It's good that you're using SQL to drop and then recreate the indexes, this should speed it up. For performance larger transaction sizes I think should be better.

 

One test for performance comparison would be to use FME to write these features to a totally new FeatureClass on the same SDE server to see if you get the same speed. If you get vastly different performance then there is likely something you can do. If your performance remains slow then you might be pretty limited in what you can do.

P

sound thanks for posting ... I've dropped the spatial index and its faster ..... but I'm unsure if I can drop the ObjectID index and recreate it as sde uses this ?

I tried it to a 'New' fetaureclass with no records this morning and it performed faster that the featureclass I'm trying to write to now... I'll keep plugging away .. again many thanks for the help

Reply


Community Stats

28,670
Posts
109,366
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.