Skip to main content
Released

Python-function to extract absolute path

Related products:FME Form

martinkoch
Supporter

I use scripted parameters from a configuration-file, similar to explained in many blog-posts.

Te location of the script is passed through a public parameter of the type 'file(existing)'.

This parameter sometimes contains an absolute path, when the config-file is stored somewhere on the filesystem, but sometimes part of it is a FME parameter such as $(FME_MF_DIR) when the config-file is stored close to the workspace. Also, on server, I deliberately use a server-parameter such as $(FME_SHAREDRESOURCE_DATA).

To cope with this, I have had to hack a quite nasty looking bit of python

import fme import ConfigParser import os import re fmeParam = re.compile("A$(FME_[A-Z]+_[A-Z]+)") fmeParamName = re.compile("FME_[A-Z]+_[A-Z]+") def findAbsolutePath(fileParamName):     paramValue = fme.macroValues[fileParamName]     fmeParamList = fmeParam.findall(paramValue)     if fmeParamList != []:         fmeParamKey = fmeParamName.findall(fmeParamList[0])[0]         fmeBasePath = fme.macroValues[fmeParamKey]         relFilePath = fmeParam.split(paramValue)[1]         absFilePath = os.path.join(fmeBasePath,relFilePath)     else:         absFilePath = paramValue     return absFilePath filePath = findAbsolutePath("conffile") conf = ConfigParser.ConfigParser() conf.optionxform = lambda option: option conf.read(filePath) return conf.getint("Parameters","MyBeautifulNumber") 

testing for the presence of FME-parameters, filtering out the key I need to get them from macroValues, and glueing it all together for an absolute path where I can find my config-file.

Could there be a function like fme.getAbsolutePath('[key_of_filepath-parameter]')?

Kind regards,

Martin

<strong>This post is closed to further activity.</strong><br /> It may be a question with a best answer, an implemented idea, or just a post needing no comment.<br /> If you have a follow-up or related question, please <a href="https://community.safe.com/topic/new">post a new question or idea</a>.<br /> If there is a genuine update to be made, please contact us and request that the post is reopened.

14 replies

david_r
Celebrity
  • February 19, 2016

Hi

I'm not quite sure I'm understanding your use-case, but have you tried using something like:

myFile = fmeobjects.FMESession().decodeFromFMEParsableText('$(FME_MF_DIR)/test.txt')

It seems to work for me.

David


david_r
Celebrity
  • February 19, 2016

Hi

I've written the following two helper functions to accomplish exactly this:

import re
import os.path
 
def expand_fme_macros(input):
    # Expands all FME macro values in input string
    matches = re.findall("\$\((\w+)\)", input)
    for group in matches:
        input = input.replace("$(%s)" % group, FME_MacroValues[group])
    return input
 
def get_fully_qualified_path(input):
    # Expands input string into an absolute path
    input = os.path.abspath(expand_fme_macros(input))
    dir = os.path.dirname(input)
    if not dir:
        input = os.path.join(FME_MacroValues['FME_MF_DIR'], input)
    return os.path.abspath(input)

 

Use expand_fme_macros() for strings in general, and get_fully_qualified_path() for path and file names.

Usage, given that the parameter value "FME_MF_DIR" = "c:\my_data_dir\"

  • expand_fme_macros("The value of FME_MF_DIR is $(FME_MF_DIR)") returns "The value of FME_MF_DIR is c:\my_data_dir\"
  • get_fully_qualified_path("my_data.mdb") returns "c:\my_data_dir\my_data.mdb"
  • get_fully_qualified_path("temp\my_data.mdb") returns "c:\my_data_dir\temp\my_data.mdb"
  • get_fully_qualified_path("temp\..\my_data.mdb") returns "c:\my_data_dir\my_data.mdb"
  • get_fully_qualified_path("d:\another\dir\my_data.mdb") returns "d:\another\dir\my_data.mdb"

This works for any value found in FME_MacroValues, notably:

  • FME internal parameters, such as FME_MF_DIR, FME_JOB_ID, FME_DATA_REPOSITORY, FME_SERVER_NAME, etc.
  • Published parameters
  • Private parameters

Hope this helps.

David


martinkoch
Supporter
Forum|alt.badge.img+18
  • Author
  • Supporter
  • February 19, 2016

Thanks, quite an elegant solution for replacing every parameter-like value surrounded by $( ). Will probably use your findall and .replace("$(%s)") to tidy-up at least the double reg-ex and list-indices in my code.

Still leaves the clear sign more people are dealing with this matter using regex-wizardry.

Makes me a bit uneasy, as there might be a real value which meets the regex, but is no FME-parameter, making the workspace barf at a "key not found" error or something alike when searching the fme.macroValue dictionary.

Would like Safe software to take that code-responsibility. Found the cause of this problem this afternoon, cooked up above code, but have to get used to it a little before I trust it in a high-load production environment.

Kind regards,

Martin

edit: Made my reply vegetarian by having 'values meet' in stead of 'meat'.


fmelizard
Safer
Forum|alt.badge.img+19
  • Safer
  • February 21, 2016

Feels like we should include @david_r 's functions in the Python FME API. I'll discuss with the appropriate authorities.


david_r
Celebrity
  • February 22, 2016

I agree that expand_fme_macros() could use some error checking:

def expand_fme_macros(input):
    # Expands all FME macro values in input string
    matches = re.findall("\$\((\w+)\)", input)
    for group in matches:
        try:
            input = input.replace("$(%s)" % group, FME_MacroValues[group])
        except:
            pass
    return input
 

 

This way it'll just silently ignore any unknown macros $(...)

David


fmelizard
Safer
Forum|alt.badge.img+19
  • Safer
  • February 22, 2016

The authorities like it. PR filed as 68362.


Forum|alt.badge.img
  • February 23, 2016

Hi,

Perhaps I am missing something here, but when I run the following in a Startup Python Script:

print "The value of FME_MF_DIR is $(FME_MF_DIR)."

The output I get is:

The value of FME_MF_DIR is C:\\Users\\SWu\\AppData\\Local\\Temp/.

What build of FME is being run that doesn't auto-evaluate the $(FME_MF_DIR)?

Thanks,

 

Stephen

david_r
Celebrity
  • February 25, 2016

Yes, I've also noticed that, FME tries to be pretty clever about macros in hard coded strings in your code.

But if you read a string containing an FME macro from e.g. a configuration file, it does not get expanded automatically. That is a potential use case for the function above.


Forum|alt.badge.img
  • February 25, 2016

Thanks for the clarification David!


helmoet
Forum|alt.badge.img+8
  • August 24, 2016

Hi, I tried to implement @david_r solution into a Pythoncaller, sadly the resulting workspace won't run and complains about undefined macro %s. I needed the solution since I read a parameter set containing FME Parameter references and use one of them to pass to a FeatureReader (for windows paths) as a filename. If I pass one of the parameter file strings, it complains that it cannot find the file, all though the file verily exist..


david_r
Celebrity
  • August 24, 2016
Can you post the code here? Or even better, a workspace to reproduce the issue?

 

 


Forum|alt.badge.img
  • August 24, 2016
The helper functions @david_r presented have been added to the FME 2017.0 beta as:

 

 

fme.resolveFMEMacros(value) fme.getAbsolutePath(fileName)

 

Documentation can be found at http://docs.safe.com/fme/2017.0/html/FME_Desktop_Documentation/FME_Workbench/Configuration/FME_BEGIN_PYTHON.htm

 


Forum|alt.badge.img+1

This has been released as part of FME 2017.0 . Visit http://www.safe.com/downloads to download it and give it a spin.


paalped
Contributor
Forum|alt.badge.img+5
  • Contributor
  • November 13, 2017
I do it this way:

 

path = FME_MacroValues['your_param_containing_path'] if re.search('\\$\\(',path): _path = path.split('(')[1].split(')')[0] _path_rest = path.split(')')[1] path = os.path.join(FME_MacroValues[_path],_path_rest).replace('/','\\\\') Paal Pedersen

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings