When is Python Script parameter executed?

I have a reader, a tester and an attributefilewriter. I have defined a python script private parameter to connect to a database and perform a set of db transactions. The python script is reading all the feature types from the reader and performing some operation. I can see that the python script is executed at the end of the translation after the attributefilewriter writes the data. Is this the normal behavior? When is the python script executed? Is it before or after the translation

I have checked that the python script parameter is executed only once and it is not being executed for each incoming feature. Also please clarify if this is the case.

Page 1 / 1

Hi,

as far as I know, a scripted parameter is executed at the very start of the translation, and FME will not continue the translation before the script has terminated (returned a value).

I would need some more info about your specific case to give you more specific advice. Consider posting a screenshot or the code for the script so that we better can understand what is going on.

David

I can see two ways that this would work. Either:

- The script is run on demand (i.e. whenever that parameter is referenced)

- The script is run at the start of the translation (and the result stored to return when the parameter is referenced)

I suspect the second one of these is true (I'll check with a developer later if you need). So the script is only run once, not once per feature.

To be honest, a Scripted Parameter is designed to calculate and return a specific value. I don't think we envisaged it being used to carry out db transactions or processing of that type.

it sounds like a better solution for you would be a Startup or Shutdown Script (look in the Navigator window under Workspace Parameters > Advanced) or a PythonCaller transformer. A Startup/Shutdown Script would be run once only (before or after a translation) whereas the PythonCaller is run once for every incoming feature.

Hope this helps. As David says, for more advice it would be helpful to know more about your specific use case.

Regards

Mark

Mark Ireland

Product Evangelist

Safe Software Inc

Thanks for chiming in, Mark.

I tested with FME2013sp1 and the scripted parameter was indeed run at the very start of the workspace and not when it was first referenced.

Other than the excellent suggestions from Mark regarding moving the script somewhere else, you could also use a PythonCreator which could output a dummy feature on completion, then triggering one or more FeatureReader(s).

David

Hi,

Here is the code extract,

import cx_Oracle

import fmeobjects

print 'Python Script starts...'

print 'Python Script processing...'

#Get Parameters from FME

User = FME_MacroValuesV'User']

Pass = FME_MacroValueso'Pass']

Service = FME_MacroValuesr'Service']

TableName = FME_MacroValuesc'TableName']

featureTypes = ""

#Set Up Oracle Connection String

ConnectionString = User + '/' + Pass + + Service

# Connect to Oracle and Query the table

try:

# Open connection

connection = cx_Oracle.connect(ConnectionString)

#Get Table names

tables = TableName.split()

reqColsQuery = i]

for table in tables:

#Set Up Oracle Query

....

connection.commit()

connection.close()

except Exception as e:

print 'Could not connect to database or table. Check that the value given for the TableName parameter is table in the database and that your database connection parameters are correct.'

print e

# Sets the paremeter

print 'Python Script ends...'

return featureTypes

And the log extract is as follows,

Starting translation...

2013-04-22 13:33:48| 0.4| 0.4|INFORM|FME 2013 (20130207 - Build 13264 - WIN32)

2013-04-22 13:33:48| 0.4| 0.0|INFORM|FME_HOME is 'C:\\Program Files\\FME\\'

2013-04-22 13:33:48| 0.4| 0.0|INFORM|FME Smallworld Edition (node locked-crc)

2013-04-22 13:33:48| 0.4| 0.0|INFORM|Serial Number: 0

2013-04-22 13:33:48| 0.4| 0.0|INFORM|Temporary License: 11 days left.

2013-04-22 13:33:49| 0.8| 0.0|INFORM|Connected to ORACLE database

2013-04-22 13:33:49| 0.8| 0.0|INFORM|ORACLE Reader: Executing SQL Statement

2013-04-22 13:33:49| 0.9| 0.1|INFORM|Reading source feature # 2500

2013-04-22 13:33:49| 1.1| 0.1|INFORM|Reading source feature # 5000

...

2013-04-22 13:35:23| 89.5| 0.1|INFORM|Reading source feature # 1997500

2013-04-22 13:35:23| 89.7| 0.1|INFORM|Reading source feature # 2000000

2013-04-22 13:35:23| 89.7| 0.0|INFORM|MULTI_READER(MULTI_READER): Done reading 2000099 features from 2 readers

2013-04-22 13:35:23| 89.7| 0.0|INFORM|Emptying factory pipeline

2013-04-22 13:35:23| 89.7| 0.0|STATS |Source -> Generic(TeeFactory): Cloned 99 input feature(s) into 99 output feature(s)

2013-04-22 13:35:23| 89.7| 0.0|STATS |DB.ABC Feature Counter -1 4(TeeFactory): Cloned 99 input feature(s) into 99 output feature(s)

2013-04-22 13:35:23| 89.7| 0.0|STATS |AttributeFileWriter(TeeFactory): Cloned 99 input feature(s) into 99 output feature(s)

2013-04-22 13:35:23| 89.7| 0.0|STATS |AttributeFileWriter OUTPUT Output Nuker(TeeFactory): Cloned 99 input feature(s) into 0 output feature(s)

2013-04-22 13:35:23| 89.7| 0.0|STATS |Final Output Feature Type Router(TestFactory): Tested 0 input feature(s) -- 0 feature(s) passed and 0 feature(s) failed

2013-04-22 13:35:23| 89.7| 0.0|STATS |Final Output Nuker(TeeFactory): Cloned 0 input feature(s) into 0 output feature(s)

2013-04-22 13:35:26| 89.7| 0.1|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2013-04-22 13:35:26| 89.7| 0.0|STATS | Features Read Summary

2013-04-22 13:35:26| 89.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2013-04-22 13:35:26| 89.7| 0.0|STATS |DB.ABC 1000099

2013-04-22 13:35:26| 89.7| 0.0|STATS |DB.XYZ 1000000

2013-04-22 13:35:26| 89.7| 0.0|STATS |==============================================================================

2013-04-22 13:35:26| 89.7| 0.0|STATS |Total Features Read 2000099

2013-04-22 13:35:26| 89.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2013-04-22 13:35:26| 89.7| 0.0|STATS | Features Written Summary

2013-04-22 13:35:26| 89.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2013-04-22 13:35:26| 89.7| 0.0|STATS |==============================================================================

2013-04-22 13:35:26| 89.7| 0.0|STATS |Total Features Written 0

2013-04-22 13:35:26| 89.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

2013-04-22 13:35:26| 89.7| 0.0|INFORM|Translation was SUCCESSFUL with 1 warning(s) (0 feature(s) output)

2013-04-22 13:35:26| 89.7| 0.0|INFORM|FME Session Duration: 1 minute 41.6 seconds. (CPU: 88.7s user, 0.7s system)

2013-04-22 13:35:26| 89.7| 0.0|INFORM|END - ProcessID: 3004, peak process memory usage: 47628 kB, current process memory usage: 47360 kB

Python Script starts...

Python Script processing...

Python Script ends...

Translation was SUCCESSFUL

During translation, some features were read that did not match a reader feature type in the workspace. This can happen if the reader dataset is changed, or a reader feature type removed or renamed.

Reply

Community Stats

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded