Question

How to pass through a lot of files with python defies error message from fme workspace?


Hi everyone

 

I have to create shapefiles from SWISS Interlis data for over 1000 SWISS Interlis files.

 

How can I pass through all SWISS Interlis files with help of Python code even if I get from some SWISS Interlis files error message like this:

 

 

 

 

 

Traceback (most recent call last):

 

  File "C:\\temp\\AV\\FMEWorkspaceAusfuehren.py", line 73, in <module>

 

    runner.runWithParameters(workspace, parameters)

 

FMEException: FMEException: 1: Failure running workspace 'C:\\temp\\PyTEST\\ITF2Shape.fmw'

 

 

 

 

 

 

 

 

 

 

 

My python code see below :

 

 

 

import os

 

import os.path

 

import sys

 

sys.path.append("C:\\\\Program Files (x86)\\\\FME\\\\fmeobjects\\\\python27")

 

import fmeobjects

 

 

 

# Source path

 

source = 'C:\\\\temp\\\\PyTEST'

 

 

 

# Change to source path

 

os.chdir(source)

 

print "source = " + source

 

 

 

# Init FMEWorkspaceRunner Class

 

runner = fmeobjects.FMEWorkspaceRunner()

 

 

 

# FME workspace path

 

workspace = 'C:\\\\temp\\\\PyTEST\\\\ITF2Shape.fmw'

 

 

 

# List SWISS Interlis files

 

ListITF = []

 

for daten in os.listdir(os.getcwd()):

 

     if daten.endswith('.itf') or daten.endswith('.ITF'):

 

          ListITF.append(daten)

 

          print "ITF file " + daten + " listed."

 

print "ITF-Files listed."

 

 

 

# FME Workspace prepare

 

for itfOrdner in ListITF:

 

     #SWISS Interlis basename

 

     count = 0

 

     newSourceDataset = ListITF[count]

 

     newDestDataset = os.path.splitext(os.path.basename(itfOrdner))[0]

 

     # FME workspace parameters

 

     parameters = {}

 

     parameters['SourceDataset_ch.ehi.fme.Main'] = source + '\\\\' + str(newSourceDataset)

 

     parameters['DestDataset_SHAPE'] = source + '\\\\' + str(newDestDataset)

 

     print "SourceParameters: " + parameters['SourceDataset_ch.ehi.fme.Main']

 

     print "DestDataset: " + parameters['DestDataset_SHAPE']

 

     count = count + 1

 

 

 

     # Start FME Workspace

 

     print "Workspace started."

 

     runner.runWithParameters(workspace, parameters)

 

    

 

     print "Workspace finished."

 

    

 

# FME process rid

 

runner = None

 

 

 

Thanks Zoran

6 replies

Userlevel 4
Hi

 

 

what does the file "C:\\temp\\PyTEST\\ITF2Shape.log" say?

 

 

David
Userlevel 4
If you just need to ignore errors from the runWithParameters command, try the following Python idiom:

 

 

-----

 

try:

 

    runner.runWithParameters(workspace, parameters)

 

except:

 

    pass

 

-----

 

 

David
Hi David

 

The logfile says same as python error message:

 

2015-06-08 13:54:06|  31.4|  0.0|STATS |DM01AVCH24D.TSEinteilung.Toleranzstufe (DM01AVCH24D.TSEinteilung.Tole        2

 

2015-06-08 13:54:06|  31.4|  0.0|STATS |DM01AVCH24D.TSEinteilung.ToleranzstufePos (DM01AVCH24D.TSEinteilung.T        2

 

2015-06-08 13:54:06|  31.4|  0.0|STATS |DM01AVCH24D.TSEinteilung.Toleranzstufe_Geometrie_LT (DM01AVCH24D.TSEi        3

 

2015-06-08 13:54:06|  31.4|  0.0|STATS |DM01AVCH24D.TSEinteilung.Toleranzstufe_MT (DM01AVCH24D.TSEinteilung.T        2

 

2015-06-08 13:54:06|  31.4|  0.0|STATS |==============================================================================

 

2015-06-08 13:54:06|  31.4|  0.0|STATS |Total Features Written                                                   85638

 

2015-06-08 13:54:06|  31.4|  0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

 

2015-06-08 13:54:06|  31.4|  0.0|ERROR |java.lang.Exception: INTERLIS 2 reader failed

 

2015-06-08 13:54:06|  31.4|  0.0|INFORM|FME Session Duration: 39.7 seconds. (CPU: 28.5s user, 2.5s system)

 

2015-06-08 13:54:06|  31.4|  0.0|INFORM|END - ProcessID: 6036, peak process memory usage: 144220 kB, current process memory usage: 138340 kB

 

 

Zoran
Hi David

 

I tried already dio options with 

 

try:

 

  runner.runWithParameters(workspace, parameters)

 

except:

 

   pass

 

 

The problem is that after error message (run FME workspace) the pythone code shalls ignore the FME workspace error message and  process the next SWISS Interlis file in the list (ListITF) and not jump to the except: code block.

 

 

Zoran

 

 
Userlevel 4
That's going to be difficult, if not impossible.

 

 

Is it at all possible for you to call the workspace once per ITF file?

 

 

David
Hi David

 

Thank you very much for your support.

 

I solved the problem!!!!!!

 

 

First I defined a function to run the FME workspace.

 

The function guaranteed the processing of all files also if the FME run fail.

 

def start_workspace(works,param):

 

    failed = False

 

    result = None

 

    try:

 

        result = runner.runWithParameters(works, param)

 

 

 

    except:

 

        failed = True

 

 

 

    return failed, result

 

 

And then lower in my python code I start FME Workspace:

 

print "Workspace started."

 

     failed = start_workspace(workspace,parameters)

 

     result = start_workspace(workspace,parameters)

 

     if failed:

 

        continue

 

 

 

Complete python code see below:

 

 

import os

 

import os.path

 

import sys

 

sys.path.append("C:\\\\Program Files (x86)\\\\FME\\\\fmeobjects\\\\python27")

 

import fmeobjects

 

 

 

# Source path

 

source = 'C:\\\\temp\\\\PyTEST'

 

 

 

# Change to source path

 

os.chdir(source)

 

print "source = " + source

 

 

 

# Init FMEWorkspaceRunner Class

 

runner = fmeobjects.FMEWorkspaceRunner()

 

 

 

# FME workspace path

 

workspace = 'C:\\\\temp\\\\PyTEST\\\\ITF2Shape.fmw'

 

 

 

# List SWISS Interlis files

 

ListITF = []

 

for daten in os.listdir(os.getcwd()):

 

     if daten.endswith('.itf') or daten.endswith('.ITF'):

 

          ListITF.append(daten)

 

          print "ITF file " + daten + " listed."

 

print "ITF-Files listed."

 

 

 

# Function to run FME workspace

 

# The function guaranteed the processing of all files also if the FME run fail

 

def start_workspace(works,param):

 

    failed = False

 

    result = None

 

    try:

 

        result = runner.runWithParameters(works, param)

 

 

 

    except:

 

        failed = True

 

 

 

    return failed, result

 

 

 

# Init counter

 

count = 0

 

 

 

# FME Workspace prepare

 

for itfOrdner in ListITF:

 

     #SWISS Interlis basename

 

      newSourceDataset = ListITF[count]

 

     newDestDataset = os.path.splitext(os.path.basename(itfOrdner))[0]

 

     # FME workspace parameters

 

     parameters = {}

 

     parameters['SourceDataset_ch.ehi.fme.Main'] = source + '\\\\' + str(newSourceDataset)

 

     parameters['DestDataset_SHAPE'] = source + '\\\\' + str(newDestDataset)

 

     print "SourceParameters: " + parameters['SourceDataset_ch.ehi.fme.Main']

 

     print "DestDataset: " + parameters['DestDataset_SHAPE']

 

     count = count + 1

 

 

 

     # Start FME Workspace

 

     print "Workspace started."

 

     failed = start_workspace(workspace,parameters)

 

     result = start_workspace(workspace,parameters)

 

     if failed:

 

        continue

 

    

 

 

 

    

 

# FME process rid

 

runner = None

 

 

 

print "Workspace finished."

 

 

 

Zoran

 

 

Reply