Solved

PythonCaller issue: How to correctly use file directory user parameter in python caller


Badge

I have a workspace in FME server, that includes PythonCaller.

The PythonCaller includes a script that needs a directory as one of its input parameters.

The directory that i want it to use, is provided to the PythonCaller as hidden user parameter. The definition of the user parameter is just text, and the value also consists of other user parameter values, that were provided from a desktop workspace which triggers the server workspace.

I have tried everything. I've changed the parameter definition and i've to put r and quote marks around the value. Still the workspace fails.

This error message for example comes, if i just try to input the user parameter value as just plain directory (D:\\Data\\citydb_temp\\...)

" Python Exception <OSError>: [WinError 123] The filename, directory name, or volume label syntax is incorrect: 'D:<backslash>Data<backslash>citydb_temp< ..."

 

So how do i put a user parameter directory as an input parameter in to the PythonCaller correctly?

 

 

 

 

icon

Best answer by jerez 9 May 2019, 15:51

View original

9 replies

Userlevel 6
Badge +32

I use this to define this paths once and let the workspace define the path based on FME_ENGINE. This way no changes are needed on the paths through our DTAP.

What works for me is:

if FME_MacroValues.get('FME_ENGINE', '') == '':    return FME_MacroValues.get('WasWordtPathDev')elif 'test' in FME_MacroValues.get('FME_ENGINE'):    return FME_MacroValues.get('WasWordtPathTest')else:    return FME_MacroValues.get('WasWordtPathProd')

2016 workspace: filedirectoryuserparameter.fmwt

Also see this question: Scripted Parameter

Keep in mind the order of the Parameters in the Navigator is important. Parameters used in the Scripted Parameter must be placed above the Scripted Parameter. This is because the order in the Navigator determines the order of reading.

Userlevel 4

In addition to accessing the private and published parameters through the FME_MacroValues dict as demonstrated by @nielsgerrits, you can also use the ParameterFetcher to retrieve the parameter value into a feature attribute.

That way you can access the value using the feature.getAttribute() method.

Notably, this is the preferred way of accessing user parameters inside custom transformers.

Userlevel 6
Badge +32

In addition to accessing the private and published parameters through the FME_MacroValues dict as demonstrated by @nielsgerrits, you can also use the ParameterFetcher to retrieve the parameter value into a feature attribute.

That way you can access the value using the feature.getAttribute() method.

Notably, this is the preferred way of accessing user parameters inside custom transformers.

@david_r Does the ParameterFetcher differ from an AttributeCreator?

Userlevel 4

@david_r Does the ParameterFetcher differ from an AttributeCreator?

Good point. No, it should not make a difference. You could even go mad and use the AttributeManager :-)

The issue is, that the FME_MacroValues dict behaves in a (to most non-Safe users) fairly unexpected way inside a custom transformer. That's why it's recommended go convert it to a feature attribute first.

Userlevel 6
Badge +32

Good point. No, it should not make a difference. You could even go mad and use the AttributeManager :-)

The issue is, that the FME_MacroValues dict behaves in a (to most non-Safe users) fairly unexpected way inside a custom transformer. That's why it's recommended go convert it to a feature attribute first.

Thanks that explains, just checking if I miss anything.

I really like to use Parameters over Attributes for all not-individual-feature-related actions (like target file paths) because they are accessible anywhere in the workspace instead of having to merge them to features and let them travel through the entire workflow (feels like this eats performance).

I even started using VariableSetters and -Retrievers for this kind of stuff, which work quite well if order of streams is well organised.

Badge

In addition to accessing the private and published parameters through the FME_MacroValues dict as demonstrated by @nielsgerrits, you can also use the ParameterFetcher to retrieve the parameter value into a feature attribute.

That way you can access the value using the feature.getAttribute() method.

Notably, this is the preferred way of accessing user parameters inside custom transformers.

@david_r I already did try that, but then the python does not recognize feature as name.

 

Userlevel 4

@david_r I already did try that, but then the python does not recognize feature as name.

 

That may indicate a more general error in the function or class definition. Can you please post the code here.

Badge

That may indicate a more general error in the function or class definition. Can you please post the code here.

The code also includes custom packagce https://github.com/wo1fsea/PyTexturePacker

Goal is to create texture atlas out of hundreds of pictures (for better viewing performance to textured 3D city model), and the PyTexturePacker package needs input directory for pictures, and output directory for the texture atlas. I also discovered that the PyTexturePacker did not work correctly if i didn't create the output directory first, so thats why there is os.mkdir before the texture packer pack_test function.

It is working perfectly fine if i just type in the directories as text like r"C:Directory\...", but when inputted as user parameter, or feature attributes, the errors comes in. 

Another strange thing is, that i also have a workspace that only has PythonCaller with same script, with no other transformers (despite from creator) and with private user parameters as inputs for the script. If i trigger that from FMEServer, it works completely fine, even with FMEMacrovalues["parameter_name"], but not when triggered in FMEDesktop. Just can't figure out what is causing the problem in the other workspace, (and why it is not working with desktop).

Here's the code below:

import fme
import fmeobjects
import os
from PyTexturePacker import Packer
# Template Class Interface:
# When using this class, make sure its name is set as the value of
# the 'Class or Function to Process Features' transformer parameter
class FeatureProcessor(object):
    def __init__(self):
        pass
    def input(self,feature):
        self.pyoutput(feature)
      
    def close(self):
        pass
        
    path = feature.getAttribute('_atlas_path')
    os.mkdir(path)
    
def pack_test(in_path, out_path):
    # create a MaxRectsBinPacker
    packer = Packer.create(enable_rotated= 0, max_width=8192, max_height=8192, bg_color=(0, 0, 0, 255), texture_format='.jpg', border_padding= 0, shape_padding= 0)
    result = packer.pack(in_path, out_path)
  
pack_test(feature.getAttribute('_input_appearances'),feature.getAttribute('_textureAtlasOutput'))
Badge

@nielsgerrits @david_r I managed to locate and fix the problem.

It was actually due to wrong parameter type. It should have been just text but instead it was "text (multiline) or number". Due to this, FME changed all the special characters, f.e "/" to "<backslash>" which obviously is something python can't understand as directory. I just wasn't careful enough to double check my settings, silly me.

Anyways, thank you so much both for your quick answers.

Reply