Question

FME Desktop - Truncate append

  • 22 April 2019
  • 2 replies
  • 25 views

I am running FME Desktop 2018.1 in a Windows 10. My source data is in a SQLServer database (2016 R2) and the destination feature class is in an ESRI Enterprise Geodatabase (ver 10.5.1) also using SQLServer (2016 R2) as the backend database.

The destination feature class is in a Enterprise spatial data warehouse, business decisions are make on the contents in the destination feature class.

I have a FME Workspace that reads a table with latitude and longitude and writes to a feature class in a ESRI Enterprise Geodatabase.

The writer is set to TABLE Exists: 'yes', TRUNCATE Table First: 'yes'

 

It appears that FME:

1) Truncates data in the destination feature class, then

2) Reads the source table, then

3) Writes to the destination feature class.

Is this correct or have I misinterpreted the FME log files? If this is correct, then there can be considerable time when the destination feature class is empty or being updated.

 

If FME does truncate the destination data first, then can I change the order so that FME:

1) Read the source table, then

2) Truncate the destination feature class. then

3) Write to the destination feature class.

 

The problem is that truncating a feature class with 500,000 points is very quick in my environment (3 seconds), reading the source table can take 2-3 minutes, writing to the destination feature class can take 1 minute. Changing the order may speed up the update process considerable.

 


2 replies

Userlevel 2
Badge +16

Reading from a database source can be slower.

Suggested workflow:

Read data from source and store in FFS file (internal format).

Use a second workspace to read from FFS and write (including truncate).

Reading from FFS is faster and therefor your table will be empty shorter.

Hope this helps.

Badge +6

You can also write to a secondary table in the output database, I personally use a random number generator, then using ArcPy I you can delete \\ rename the new table. Using a template feature class (the original one possibly) FME will recreate the same table schema including domains \\ aliases \\ etc. The only issue is permissions but that can also be included in ArcPy or a "run after migration" SQL statement.

Reply