Question

Access parameter from PythonCaller in Custom Transformer

  • 19 March 2013
  • 8 replies
  • 11 views
K
Rank
Badge +6
I'm trying to create a custom transformer 'FeatureCountLogger' which logs the number of features inputted into this transformer.

 

 

The custom transformer contains a PythonCaller with the following code:

 

 

import fmeobjects class FeatureProcessor(object):    def __init__(self):        self.featureList = []    def input(self,feature):        self.featureList.append(feature)    def close(self):        logger = open(FME_MacroValues['Summarylog'], "a")        message = FME_MacroValues['Message'] + str(len(self.featureList)) + '\\n'        logger.write (message)        logger.close()

 

 

 

I want to use the logger multiple times, so I created a published parameter 'Message' in the custom transformer, so I can submit the text logged to each instance of the transformer.

 

 

When I'm running this I get the following error: Python Exception <KeyError>: 'Message'

 

 

It seems like the PythonCaller can't access the parameter defined in the custom transformer. It can access parameters in the Main (eg. 'Summarylog')

 

 

Any idea what the problem is here?

8 replies

david_r
Rank
Userlevel 4
Hi,

 

 

the user parameters behave a little bit differently inside a custom transformer. They are in fact prefixed with the name of the custom transformer to avoid collisions with the main workspace parameters. Example: if you have a custom transformer called "custtransf" with a published parameter called "custparam", you will have to refer to it like this.

 

 

value = FME_MacroValues["custtransf_custparam"]

 

One other hand, I see that you populate a list of all the features in your input() method above. Unless you really need this, I would try to avoid it as it really fills up memory. I would consider doing something like this instead:

 

 

import fmeobjects class FeatureProcessor(object):    def __init__(self):        self.featureCount = 0    def input(self,feature):        self.featureCount += 1    def close(self):        message = FME_MacroValues['Message'] + str(self.featureCount) + '\\n'   It should be both quicker and more efficient.

 

 

Hope this helps.

 

 

David
K
Rank
Badge +6
Thanks for your response David.

 

 

Do you know how I can access the right parameter from python. I tried using the FeatureCountLogger_WORKSPACE_NAME, but that doesn't seem te work. For each instance of the custom transformer I get the same result; the parameter of one of them.

 

 

import fmeobjects class FeatureProcessor(object):    def __init__(self):        self.featureCount = 0    def input(self,feature):        self.featureCount += 1    def close(self):        logger = open(FME_MacroValues['Summarylog'], "a")        logMessage = FME_MacroValues[FME_MacroValues['FeatureCountLogger_WORKSPACE_NAME'] + '_Message']        message = logMessage + str(self.featureCount) + '\\n'        logger.write (message)        logger.close()

 

david_r
Rank
Userlevel 4
Hi,

 

 

this handy little snippet will print all the available key values in FME_MacroValues for you -- you are bound to find the key you search for there.

 

 

Insert it into a PythonCaller and it will add it all, nicely formatted, to your FME log window:

 

 

-----

 

import fmeobjects import pprint   def showAllMacroValues(feature):     pp = pprint.PrettyPrinter(indent=4)     pp.pprint(FME_MacroValues)

 

-----

 

 

David

 

 
K
Rank
Badge +6
Hi David,

 

 

I have three instances of my custom transformer:

 

LoggerA, LoggerB & LoggerC.

 

 

When I print the FME_MacroValues I get all parameters three times for each transformer. So how do I know inside my custom transformer python code which parameter I must use, since to code is the same for each instance.

 

 

Inside LoggerA:

 

 LoggerA_Message: "Total number of features"

 

 LoggerB_Message: "Number of filtered features"

 

 LoggerC_Message: "Number of unfiltered features"

 

 

Inside LoggerB:

 

 LoggerA_Message: "Total number of features"

 

 LoggerB_Message: "Number of filtered features"

 

 LoggerC_Message: "Number of unfiltered features"

 

 

Inside LoggerC:

 

 LoggerA_Message: "Total number of features"

 

 LoggerB_Message: "Number of filtered features"

 

 LoggerC_Message: "Number of unfiltered features"
david_r
Rank
Userlevel 4
Hi,

 

 

it will depend on what you're trying to accomplish.

 

 

If you need the total number of features in all of your workspace, I would use a Python shutdown script where you will already have a variable called "FME_TotalFeaturesWriten" (more info here) that you can print directly. 

 

 

If you need a count per transformer, I would do only the counting in your custom transformer, setting a global Python variable with your counters. The counters could be based on a dictionary where your published parameter (name of counter, if you will) is used as the dictionary key. You could then output them in your workbench shutdown script.

 

 

David
jstanger
Badge +1

Hi Koen,

I came across a similar problem with accessing custom transformer parameters from a PythonCaller node inside said custom transformer. I solved this with the following:

transformerName = "CUSTOM_TRANSFORMER_NAME"

 

instanceName = fme.macroValues[transformerName + '_WORKSPACE_NAME']

 

keys = [x for x in fme.macroValues.keys() if x.find(instanceName) >=0]

 

params = {}

 

for key in keys:

 

params[key] = fme.macroValues[key]

 

cfg_CUSTOMPARAMNAME = params[instanceName + '_CUSTOMPARAMNAME']

 

It needs to know the name you gave the custom transformer when you created it. It then uses this to query the workspace to find out what its instance name is. Then it looks for all parameters for that instance and puts it into a nice easy to find dict.

J
Badge

Hi @koen

I have had the same issue already a couple of times. For me the best workaround was to use a ParameterFetcher (or AttributeCreator / -Manager) and store the value of the parameter in an attribute. Doing so will allow you to use it easily in the PythonCaller. And everything will still work when you rename the CustomTransformer.

Good luck!

lifalin2016
Rank
Userlevel 1
Badge +22

Hi @koen

I have had the same issue already a couple of times. For me the best workaround was to use a ParameterFetcher (or AttributeCreator / -Manager) and store the value of the parameter in an attribute. Doing so will allow you to use it easily in the PythonCaller. And everything will still work when you rename the CustomTransformer.

Good luck!

This is the sensible approach. The parameter naming inside custom transformers is horrible. Fortunately one can easily clean up all the internal attributes when outputting features from the transformer.

-- Cheers, Lars I.

Reply


Community Stats

28,661
Posts
109,319
Replies
36,528
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.