Question

Can multiple workbench write to a file geodatabase at the same time?

  • 7 October 2022
  • 3 replies
  • 19 views

Badge +1

I would like to know if it is possible to run several fme workbench that will write in parallel to a file geodatabase at the same time with each workbench writing to different feature classes?

 

I tried this but it is not working and I don't know why.

 

@Takashi Iijima​ and others, any thoughts  on this?


3 replies

Userlevel 3
Badge +26

While I've never tried multiple workbenches writing to the same database simultaneously, I think it would be best practice to avoid it. I would look into stringing the workbenches together using a series of WorkspaceRunners instead of running them at the same time.

Userlevel 5
Badge +25

This sounds like a bad idea.

Badge +5

Hi @arthy​ ,

 

You'll find that when an FME workbench writes to a geodatabase it should create a lock on it, stopping any other processes from doing the same (at least I think anyway).

 

I recently had to solve this issue and found the following work around:

  1. Split the workbench into multiple workbenches for each writer - All of these workbenches will run concurrently.
  2. Duplicate the file geodatabase and have each workbench writing to its own file geodatabase.
  3. Once all of these workbenches finish, trigger a new workbench that runs a python caller. See the append_gdb code attached. You will need to change the output geodatabases to the ones you want.

As you are inserting values to this via python, you'll find that you can't truncate so you'll need to use the append script to clear down your geodatabase before you run the process again. It's a manual work around but I've found it allows parallel processing of workspaces quite well 🙂

 

Truncate:

import fmeobjects
import arcpy  
from arcpy import env
 
class FeatureCreator(object):
    """Template Class Interface:
    When using this class, make sure its name is set as the value of the 'Class
    to Process Features' transformer parameter.
    """
    
    def __init__(self):
        """Base constructor for class members."""
        pass
        
    def input(self, feature):
 
#Ignore this count. It was used to clear down all of my geodatabases via a loop - gdb1->gdb2 etc
        count = ['']
        for num in count:
            env.workspace = r"C:\yourgeodatabase"+num".gdb"
                             
            fc_list = arcpy.ListFeatureClasses()
            tables = arcpy.ListTables()
            ds_list = arcpy.ListDatasets()
 
            #feature classes
            for fc in fc_list:
                arcpy.TruncateTable_management(fc)
                print("table truncated - "+fc)
            #tables
            for t in tables:
                if t == 'WINGS_RELATIONSHIPS':
                    arcpy.TruncateTable_management(t)
                    print("table truncated - "+t)
            #data sets
            for ds in ds_list:
                
                for FC in arcpy.ListFeatureClasses("*","",ds):
                    arcpy.TruncateTable_management(FC)
                    print("table truncated - "+FC)
 
            print("GDB "+ num + " Finished")
 
        newFeature = fmeobjects.FMEFeature()
        self.pyoutput(newFeature)
        
    def close(self):
        """This method is called at the end of the class."""
        pass

 

Reply