Skip to main content
Solved

Provide dataset path to reader through a Python script

R
Forum|alt.badge.img

Hi,

I want to trigger a workspace using a python script and with that, also want to set the reader dynamically as a parameter. I did look at a few posts previously on this subject like using the workspace runner or the Creator but wasn't sure if that would match my requirement.

My requirement is that - I have a set of workbenches that already have a Reader transformer added but I want to override those readers through Python script and provide dataset paths through python script.

It would be great help if I am advised whether I am trying this the wrong way and if I should be getting rid of the Readers from all my fmw files and take some other method that would allow me to provide the path in a way that the workbench could be triggered just by a python script and there would be no need to open the fmw files in FME Desktop

Best answer by david_r

You can use Python scripted parameters for this, I do it quite often and it works really well.

Look at this post for some ideas: https://knowledge.safe.com/questions/34975/moving-workspaces-between-fme-installs-tips-and-tr.html

View original
Did this help you find an answer to your question?
This post is closed to further activity.
It may be a question with a best answer, an implemented idea, or just a post needing no comment.
If you have a follow-up or related question, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

8 replies

david_r
Celebrity
  • david_rCelebrity
  • Best Answer
  • October 21, 2016

You can use Python scripted parameters for this, I do it quite often and it works really well.

Look at this post for some ideas: https://knowledge.safe.com/questions/34975/moving-workspaces-between-fme-installs-tips-and-tr.html

R
Forum|alt.badge.img
  • raghunaren
  • Author
  • October 25, 2016
david_r wrote:

You can use Python scripted parameters for this, I do it quite often and it works really well.

Look at this post for some ideas: https://knowledge.safe.com/questions/34975/moving-workspaces-between-fme-installs-tips-and-tr.html

 

Thanks @david_r. I got a head start with the post you referred !

 

 

I tried the following code with the same fmw as in the post and it works perfectly fine but if I pass the same to a 'Reader' as a private parameter I get an error that says 'ConfigParser.NoSectionError: No section: 'DATA_INPUT' '

 

 

import osfrom ConfigParser import SafeConfigParser # Built-in Python moduleini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'# Initialize parserconfig = SafeConfigParser()# Read the ini fileconfig.read(ini_file)# Return option 'PER' under section 'DATA_INPUT'# return config.get('DATA_INPUT', 'wbo')# Return file namefile_path = config.get('DATA_INPUT', 'wbo')for file in os.listdir(file_path): if file.endswith("PLACES.csv"): return file_path+'\\\\'+file

 

 

Any idea on what's wrong here.

 

 

Thanks much for your help !

 

david_r
Celebrity
  • david_rCelebrity
  • October 25, 2016
raghunaren wrote:

 

Thanks @david_r. I got a head start with the post you referred !

 

 

I tried the following code with the same fmw as in the post and it works perfectly fine but if I pass the same to a 'Reader' as a private parameter I get an error that says 'ConfigParser.NoSectionError: No section: 'DATA_INPUT' '

 

 

import osfrom ConfigParser import SafeConfigParser # Built-in Python moduleini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'# Initialize parserconfig = SafeConfigParser()# Read the ini fileconfig.read(ini_file)# Return option 'PER' under section 'DATA_INPUT'# return config.get('DATA_INPUT', 'wbo')# Return file namefile_path = config.get('DATA_INPUT', 'wbo')for file in os.listdir(file_path): if file.endswith("PLACES.csv"): return file_path+'\\\\'+file

 

 

Any idea on what's wrong here.

 

 

Thanks much for your help !

 

Can you please repost your code inside a code block (to retain formatting) as well as post your ini file?
R
Forum|alt.badge.img
  • raghunaren
  • Author
  • October 25, 2016
david_r wrote:
Can you please repost your code inside a code block (to retain formatting) as well as post your ini file?
config.txt

 

 

R
Forum|alt.badge.img
  • raghunaren
  • Author
  • October 25, 2016
raghunaren wrote:
config.txt

 

 

 

python-code.txt

 

R
Forum|alt.badge.img
  • raghunaren
  • Author
  • October 25, 2016
raghunaren wrote:
config.txt

 

 

Sorry about the code paste. I have attached a txt file which hopefully should retain the indentation.

 

 

david_r
Celebrity
  • david_rCelebrity
  • October 25, 2016

Thanks for the code and the ini-file, that helps a lot.

Two things:

  1. The error mesage "No section: 'DATA_INPUT'" means that the ConfigParser couldn't find your ini-file. Since you're using the FME_MF_DIR macro which references the directory where your workspace was saved, you also have to make sure that you have saved your workspace to the right directory relative to the ini file. If your workspace is unsaved, it won't work.
  2. Your function should return a string in all cases except if there is a critical error. As it is, you only return a file path if it finds a filename ending with "PLACES.csv". Ideally, you should also return something if the file wasn't found.

Here's my attempt at cleaning up the code a bit and making it a bit more robust:

import os
from ConfigParser import SafeConfigParser # Built-in Python module

ini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'
if not os.path.isfile(ini_file):
    raise Exception('INI file not found: %s' % ini_file)

# Initialize parser
config = SafeConfigParser()

# Read the ini file
config.read(ini_file)

# Return option 'PER' under section 'DATA_INPUT'
file_path = config.get('DATA_INPUT', 'wbo')

# Check that the directory exists
if not os.path.isdir(file_path):
    raise Exception('Directory not found: %s' % file_path)

# Look inside directory for places.csv    
for file in os.listdir(file_path):
    # Case-insensitive filename comparison
    if file.lower().endswith("places.csv"):
        # We found the file, return filename
        return file_path+'\\'+file

# Couldn't find the file
return 'File places.csv not found.'

Hope it works for you.

R
Forum|alt.badge.img
  • raghunaren
  • Author
  • October 25, 2016
david_r wrote:

Thanks for the code and the ini-file, that helps a lot.

Two things:

  1. The error mesage "No section: 'DATA_INPUT'" means that the ConfigParser couldn't find your ini-file. Since you're using the FME_MF_DIR macro which references the directory where your workspace was saved, you also have to make sure that you have saved your workspace to the right directory relative to the ini file. If your workspace is unsaved, it won't work.
  2. Your function should return a string in all cases except if there is a critical error. As it is, you only return a file path if it finds a filename ending with "PLACES.csv". Ideally, you should also return something if the file wasn't found.

Here's my attempt at cleaning up the code a bit and making it a bit more robust:

import os
from ConfigParser import SafeConfigParser # Built-in Python module

ini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'
if not os.path.isfile(ini_file):
    raise Exception('INI file not found: %s' % ini_file)

# Initialize parser
config = SafeConfigParser()

# Read the ini file
config.read(ini_file)

# Return option 'PER' under section 'DATA_INPUT'
file_path = config.get('DATA_INPUT', 'wbo')

# Check that the directory exists
if not os.path.isdir(file_path):
    raise Exception('Directory not found: %s' % file_path)

# Look inside directory for places.csv    
for file in os.listdir(file_path):
    # Case-insensitive filename comparison
    if file.lower().endswith("places.csv"):
        # We found the file, return filename
        return file_path+'\\'+file

# Couldn't find the file
return 'File places.csv not found.'

Hope it works for you.

Oh dear..I didn't have the workbench saved at all ! Thanks @david_r

 

Helpful Members This Week

Community Stats

31,294
Posts
118,683
Replies
38,878
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.