Solved

Provide dataset path to reader through a Python script

  • 21 October 2016
  • 8 replies
  • 6 views
R
Badge

Hi,

I want to trigger a workspace using a python script and with that, also want to set the reader dynamically as a parameter. I did look at a few posts previously on this subject like using the workspace runner or the Creator but wasn't sure if that would match my requirement.

My requirement is that - I have a set of workbenches that already have a Reader transformer added but I want to override those readers through Python script and provide dataset paths through python script.

It would be great help if I am advised whether I am trying this the wrong way and if I should be getting rid of the Readers from all my fmw files and take some other method that would allow me to provide the path in a way that the workbench could be triggered just by a python script and there would be no need to open the fmw files in FME Desktop

icon

Best answer by david_r 21 October 2016, 12:43

View original

8 replies

david_r
Rank
Userlevel 4

You can use Python scripted parameters for this, I do it quite often and it works really well.

Look at this post for some ideas: https://knowledge.safe.com/questions/34975/moving-workspaces-between-fme-installs-tips-and-tr.html

R
Badge

You can use Python scripted parameters for this, I do it quite often and it works really well.

Look at this post for some ideas: https://knowledge.safe.com/questions/34975/moving-workspaces-between-fme-installs-tips-and-tr.html

 

Thanks @david_r. I got a head start with the post you referred !

 

 

I tried the following code with the same fmw as in the post and it works perfectly fine but if I pass the same to a 'Reader' as a private parameter I get an error that says 'ConfigParser.NoSectionError: No section: 'DATA_INPUT' '

 

 

import osfrom ConfigParser import SafeConfigParser # Built-in Python moduleini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'# Initialize parserconfig = SafeConfigParser()# Read the ini fileconfig.read(ini_file)# Return option 'PER' under section 'DATA_INPUT'# return config.get('DATA_INPUT', 'wbo')# Return file namefile_path = config.get('DATA_INPUT', 'wbo')for file in os.listdir(file_path): if file.endswith("PLACES.csv"): return file_path+'\\\\'+file

 

 

Any idea on what's wrong here.

 

 

Thanks much for your help !

 

david_r
Rank
Userlevel 4

 

Thanks @david_r. I got a head start with the post you referred !

 

 

I tried the following code with the same fmw as in the post and it works perfectly fine but if I pass the same to a 'Reader' as a private parameter I get an error that says 'ConfigParser.NoSectionError: No section: 'DATA_INPUT' '

 

 

import osfrom ConfigParser import SafeConfigParser # Built-in Python moduleini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'# Initialize parserconfig = SafeConfigParser()# Read the ini fileconfig.read(ini_file)# Return option 'PER' under section 'DATA_INPUT'# return config.get('DATA_INPUT', 'wbo')# Return file namefile_path = config.get('DATA_INPUT', 'wbo')for file in os.listdir(file_path): if file.endswith("PLACES.csv"): return file_path+'\\\\'+file

 

 

Any idea on what's wrong here.

 

 

Thanks much for your help !

 

Can you please repost your code inside a code block (to retain formatting) as well as post your ini file?
R
Badge
Can you please repost your code inside a code block (to retain formatting) as well as post your ini file?
config.txt

 

 

R
Badge
config.txt

 

 

 

python-code.txt

 

R
Badge
config.txt

 

 

Sorry about the code paste. I have attached a txt file which hopefully should retain the indentation.

 

 

david_r
Rank
Userlevel 4

Thanks for the code and the ini-file, that helps a lot.

Two things:

  1. The error mesage "No section: 'DATA_INPUT'" means that the ConfigParser couldn't find your ini-file. Since you're using the FME_MF_DIR macro which references the directory where your workspace was saved, you also have to make sure that you have saved your workspace to the right directory relative to the ini file. If your workspace is unsaved, it won't work.
  2. Your function should return a string in all cases except if there is a critical error. As it is, you only return a file path if it finds a filename ending with "PLACES.csv". Ideally, you should also return something if the file wasn't found.

Here's my attempt at cleaning up the code a bit and making it a bit more robust:

import os
from ConfigParser import SafeConfigParser # Built-in Python module

ini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'
if not os.path.isfile(ini_file):
    raise Exception('INI file not found: %s' % ini_file)

# Initialize parser
config = SafeConfigParser()

# Read the ini file
config.read(ini_file)

# Return option 'PER' under section 'DATA_INPUT'
file_path = config.get('DATA_INPUT', 'wbo')

# Check that the directory exists
if not os.path.isdir(file_path):
    raise Exception('Directory not found: %s' % file_path)

# Look inside directory for places.csv    
for file in os.listdir(file_path):
    # Case-insensitive filename comparison
    if file.lower().endswith("places.csv"):
        # We found the file, return filename
        return file_path+'\\'+file

# Couldn't find the file
return 'File places.csv not found.'

Hope it works for you.

R
Badge

Thanks for the code and the ini-file, that helps a lot.

Two things:

  1. The error mesage "No section: 'DATA_INPUT'" means that the ConfigParser couldn't find your ini-file. Since you're using the FME_MF_DIR macro which references the directory where your workspace was saved, you also have to make sure that you have saved your workspace to the right directory relative to the ini file. If your workspace is unsaved, it won't work.
  2. Your function should return a string in all cases except if there is a critical error. As it is, you only return a file path if it finds a filename ending with "PLACES.csv". Ideally, you should also return something if the file wasn't found.

Here's my attempt at cleaning up the code a bit and making it a bit more robust:

import os
from ConfigParser import SafeConfigParser # Built-in Python module

ini_file = FME_MacroValues['FME_MF_DIR'] + '/config/config.ini'
if not os.path.isfile(ini_file):
    raise Exception('INI file not found: %s' % ini_file)

# Initialize parser
config = SafeConfigParser()

# Read the ini file
config.read(ini_file)

# Return option 'PER' under section 'DATA_INPUT'
file_path = config.get('DATA_INPUT', 'wbo')

# Check that the directory exists
if not os.path.isdir(file_path):
    raise Exception('Directory not found: %s' % file_path)

# Look inside directory for places.csv    
for file in os.listdir(file_path):
    # Case-insensitive filename comparison
    if file.lower().endswith("places.csv"):
        # We found the file, return filename
        return file_path+'\\'+file

# Couldn't find the file
return 'File places.csv not found.'

Hope it works for you.

Oh dear..I didn't have the workbench saved at all ! Thanks @david_r

 

Reply


Community Stats

28,665
Posts
109,338
Replies
36,534
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.