Question

Performance Issue in File Geodatabase closing

  • 9 August 2016
  • 4 replies
  • 5 views

Badge +1

Hello,

I am writing several thousand polygon shapefile features to a File Geodatabase using Feature Merge type as "All" in a batch process using TCL script (generated from batch deploy method). While the reading and writing features seem to be consistent. Upon looking at log file it seems like closing Geodatabase writer takes about 15 minutes. I do not understand why the two innocuous steps is taking such a long time. My file Geodatabase writer is ArcObj writer.

Please let me what might be the possible reasons for such occurrence.

I have attached the snippet of my log file below. See the time and line in bold.

2016-08-09 12:30:53| 916.8| 0.0|INFORM|Shape Reader: Bounding box of shapefile(s) selected from folder `C:\\_LOCALdata\\Cadastral\\titles/' is: (-113.426429531,52.9452830456),(-113.329698464,53.0034416118)

 

2016-08-09 12:30:54| 917.4| 0.6|INFORM|Reading source feature # 1790000

 

2016-08-09 12:30:55| 918.5| 1.1|INFORM|Reading source feature # 1792500

 

2016-08-09 12:30:55| 918.9| 0.4|INFORM|MULTI_READER(ESRISHAPE_1): Done reading 1793436 features from 6620 readers

 

2016-08-09 12:30:55| 918.9| 0.0|INFORM|Emptying factory pipeline

 

2016-08-09 12:30:55| 918.9| 0.0|STATS |Router and Unexpected Input Remover(RoutingFactory): Tested 1793436 input feature(s), wrote 1793436 output feature(s): 1793436 matched merge filters, 1793436 were routed to output, 0 could not be routed.

 

2016-08-09 12:30:55| 918.9| 0.0|STATS |t_401001 (ESRISHAPE_1) Splitter(TeeFactory): Cloned 1793436 input feature(s) into 1793436 output feature(s)

 

2016-08-09 12:30:56| 919.5| 0.6|STATS |Destination Feature Type Routing Correlator(RoutingFactory): Tested 1793436 input feature(s), wrote 1793436 output feature(s): 0 matched merge filters, 1793436 were routed to output, 0 could not be routed.

 

2016-08-09 12:30:56| 919.5| 0.0|STATS |Final Output Nuker(TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

 

2016-08-09 12:30:56| 919.5| 0.0|STATS |GeodatabaseRelationshipFeaturesPipeline::GeodatabaseRelationshipFeatures(SortFactory): Sorted 0 feature(s)

 

2016-08-09 12:30:56| 919.5| 0.0|INFORM|Transaction #18 (final transaction) was successfully committed

 

2016-08-09 12:46:38|1856.4|936.9|INFORM|Closing the Geodatabase writer

 

2016-08-09 12:58:12|2539.6|683.2|INFORM|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

2016-08-09 12:58:12|2539.6| 0.0|INFORM|Feature output statistics for `GEODATABASE_FILE' writer using keyword `GEODATABASE_FILE_1':


4 replies

Userlevel 2
Badge +16

The Arc Objects GDB writer uses the ESRI client software. It does not look like you need that in this workflow. Have you tried using the API GDB writer. I have good experience with that (and good performance).

Badge +1

The Arc Objects GDB writer uses the ESRI client software. It does not look like you need that in this workflow. Have you tried using the API GDB writer. I have good experience with that (and good performance).

Thanks, I will try that but I still need to know that why the Closing Geodatabase Writer taking so long because some of the workflow need Arc Objects GDB.

Thanks

Vivek

Badge +16

Any chance the writer is doing some validation?

Another option would be to change the logging setting to provide (possibly) more meaningful information

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_Workbench/Workbench/Viewing_the_Log.htm

Badge +9

Hi @videh451,

Thanks for sharing this on the community, would you be able to provide a reproduction? We would need your workspace, script and data and then we can test this and correct the problem. Please submit a support ticket at: https://support.safe.com and we will be happy to look into this further.

 

 

Kind regards,

 

Brian Pont

 

Reply