I have revised an old workbench to use the "new" db_operation variable to do add/modify/delete updates in one writer instead of setting three writers with separate modes. I wonder what was done? It looks like the updates are cached and applied in a single query. The time has dramatically reduced from 15 minutes down to 1 min 17 seconds! I am using a csv file to update a file geodatabase. The geodatabase is indexed on the key. The database is 7 GB and the main table contains 24.6 million records.
I write the end of the log into an annotation to give me an easy persistent change over time:
(The log shows the updates written/deleted but not the full table being updated)
Features Written Summary
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Title_Memorial 86576
Title_memorial 586
changeset 87162
====================================================================
Total Features Written 174324
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
CSV Reader: Done reading CSV files in directory `E:\crs\currentgdb/\'
Translation was SUCCESSFUL with 208 warning(s) (174324 feature(s) output)
FME Session Duration: 19 minutes 2.0 seconds. (CPU: 40.2s user, 13.7s system)
END - ProcessID: 9008, peak process memory usage: 357652 kB, current process memory usage: 179524 kB
Translation was SUCCESSFUL
11 March 2015
Title_Memorial 101382
Title_memorial 913
changeset 102295
=====================================================================
Total Features Written 204590
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
CSV Reader: Done reading CSV files in directory `E:\crs\currentgdb/\'
Translation was SUCCESSFUL with 323 warning(s) (204590 feature(s) output)
FME Session Duration: 15 minutes 28.1 seconds. (CPU: 43.7s user, 14.8s system)
END - ProcessID: 10980, peak process memory usage: 409740 kB, current process memory usage: 377948 kB
Translation was SUCCESSFUL
12 April 2015
Version 2 using db_operations. FME 2018.1 has changed something
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Title_Memorial 133920
changeset 133920
==============================================================================
Total Features Written 267840
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Translation was SUCCESSFUL with 909 warning(s) (267840 feature(s) output)
FME Session Duration: 1 minute 17.5 seconds. (CPU: 60.1s user, 14.1s system)
END - ProcessID: 16104, peak process memory usage: 375528 kB,
current process memory usage: 117120 kB
Translation was SUCCESSFUL
7 Jan 2019
