Skip to main content
Solved

How to manipulate attribute values in python caller?

S
Forum|alt.badge.img+1

Can somebody please help me understand how to use the python caller? I have looked at tutorials, older code and even help for the error i get which is - Python Exception <NameError>: global name '_file_contents' is not defined

I am unable to resolve this. I need to match the coordinates in my shapefile to the coordinates in my point cloud file and extract the point cloud classification data. I have a number of unresolved issues with my logic but my important question is the following

How do I import and manipulate attribute values in a python caller in fme?

In my workspace, I have a shapefile with 12 points. I am extracting the coordinates of it and storing it as a string in the _file_contents variable [in format (x,y),(x,y), ]. I need to perform a lot of manipulations on this but to understand if I am using the transformer correctly, I am performing this first basic function of removing the last comma from my string. However, I keep getting the error I have posted above.

I have attached my template file -

Best answer by david_r

So here's the Python code that you posted:

class FeatureProcessor(object):
    def __init__(self):
        self.bgt_point = None
    def input(self,feature):
        self.bgt_point = feature.getAttribute('_coordinates')
        self.bgt_point = feature.setAttribute("_coordinates", _coordinates[:-1])
    def close(self):
        self.pyoutput(self.bgt_point)

The problem is that on line 6 there's a reference to the feature attribute "_coordinates" as if it was a Python object, but "_coordinates" hasn't been defined in a Python context anywhere, which explains the exception. In short: Python objects (variables) and feature attributes don't mix automatically, you'll have to do  that manually using methods such as e.g. setAttribute() and getAttribute().

Secondly, the last lines is also problematic:

self.pyoutput(self.bgt_point)

Syntactically it's not wrong, but the pyoutput() method expects an object of type FMEFeature, but in your case self.bgt_point is always undefined because setAttribute() doesn't have a return value (line 6 above).

Here's how you could rewrite it:

class FeatureProcessor(object):
    def __init__(self):
        self.bgt_point = '' # Initialize as empty string
    def input(self,feature):
        # Append _coordinates attribute to end of self.bgt_point
        self.bgt_point += feature.getAttribute('_coordinates')
    def close(self):
        # Create a new feature
        new_feature = FMEFeature()
        # Assign self.bgt_point minus last character to feature attribute _coordinates
        new_feature.setAttribute("_coordinates", self.bgt_point[:-1])
        self.pyoutput(new_feature)

Here's the output of the single feature exiting the PythonCaller at the end of the translation:

_coordinates (encoded: utf-8): (98858.662,437724.223),(98545.853,437751.119),(98736.242,437554.716),(98758.669,437714.289),(98826.957,437616.275),(98539.194,437634.46),(98860.333,437724.647),(98540.577,437682.626),(98638.13,437563.122),(98564.843,437818.05),(98750.72,437726.321),(98750.23,437727.173)

Hope that makes sense, if not let me know.

View original
Did this help you find an answer to your question?

11 replies

david_r
Evangelist
  • david_rEvangelist
  • Best Answer
  • October 30, 2017

So here's the Python code that you posted:

class FeatureProcessor(object):
    def __init__(self):
        self.bgt_point = None
    def input(self,feature):
        self.bgt_point = feature.getAttribute('_coordinates')
        self.bgt_point = feature.setAttribute("_coordinates", _coordinates[:-1])
    def close(self):
        self.pyoutput(self.bgt_point)

The problem is that on line 6 there's a reference to the feature attribute "_coordinates" as if it was a Python object, but "_coordinates" hasn't been defined in a Python context anywhere, which explains the exception. In short: Python objects (variables) and feature attributes don't mix automatically, you'll have to do  that manually using methods such as e.g. setAttribute() and getAttribute().

Secondly, the last lines is also problematic:

self.pyoutput(self.bgt_point)

Syntactically it's not wrong, but the pyoutput() method expects an object of type FMEFeature, but in your case self.bgt_point is always undefined because setAttribute() doesn't have a return value (line 6 above).

Here's how you could rewrite it:

class FeatureProcessor(object):
    def __init__(self):
        self.bgt_point = '' # Initialize as empty string
    def input(self,feature):
        # Append _coordinates attribute to end of self.bgt_point
        self.bgt_point += feature.getAttribute('_coordinates')
    def close(self):
        # Create a new feature
        new_feature = FMEFeature()
        # Assign self.bgt_point minus last character to feature attribute _coordinates
        new_feature.setAttribute("_coordinates", self.bgt_point[:-1])
        self.pyoutput(new_feature)

Here's the output of the single feature exiting the PythonCaller at the end of the translation:

_coordinates (encoded: utf-8): (98858.662,437724.223),(98545.853,437751.119),(98736.242,437554.716),(98758.669,437714.289),(98826.957,437616.275),(98539.194,437634.46),(98860.333,437724.647),(98540.577,437682.626),(98638.13,437563.122),(98564.843,437818.05),(98750.72,437726.321),(98750.23,437727.173)

Hope that makes sense, if not let me know.

david_r
Evangelist
  • david_rEvangelist
  • October 30, 2017

Alternative and simpler solution without having to resort to Python:

  • Modify the StringConcatenator to remove the last comma. Or replace the CoordinateExtractor and the StringConcatenator with a simple CoordinateConcatenator.

     

  • Replace the PythonCaller with an Aggregator as follows:

The result will be the same.

S
Forum|alt.badge.img+1
  • sraapril
  • Author
  • October 30, 2017

Hello @david_r

Thanks a bunch for the help! Yes, I kind of understand now. The code works perfectly. I could use the aggregator however I was looking to understand how to use attributes in a python code. I have a lot more processing to do and not just remove the last character.

S
Forum|alt.badge.img+1
  • sraapril
  • Author
  • November 2, 2017

Hi @david_r,

I'm kind of stuck again. I finished the coding script I wanted to implement and this is how my new code looks like


import fme
from fmeobjects import FMEFeature
import numpy

class FeatureProcessor(object):
    def __init__(self):
        self.CID = 0
    def input(self,feature):
        bgt_pt = feature.getAttribute('bgt_points')
        las_pt = feature.getAttribute('las_points')
        
distance_list = []
        
        def coordinateMatch(bgt_pt,las_pt):
            for b1, b2 in enumerate(bgt_pt):
                for l1, l2 in enumerate(las_pt):
                    bp1 = bgt_pt[b1][0]
                    bp2 = bgt_pt[b1][1]
                    lp1 = las_pt[l1][0]
                    lp2 = las_pt[l1][1]
                    d = numpy.sqrt((bp1 - lp1)*(bp1 - lp1) + (bp2 - lp2)*(bp2 - lp2))
                    distance_list.append((las_pt[l1][2], d))

            component = min(distance_list, key = lambda t: t[1])
            return component

        bgt_pt = bgt_pt.split(',')
        las_pt = las_pt.split(',')
        
        las_pt = las_pt[:-1]

        bgt_pt = [float(i) for i in bgt_pt]
        las_pt = [float(i) for i in las_pt]

        las_pt = [i for i in zip(*[iter(las_pt)]*3)]
        bgt_pt = [i for i in zip(*[iter(bgt_pt)]*2)]

        self.CID = coordinateMatch(bgt_pt,las_pt)
    def close(self):
        new_feature = FMEFeature()
        new_feature.setAttribute("CID", self.CID)
        self.pyoutput(new_feature)<br>

I have created an attribute called CID and assigned it to NULL in my workflow. I want the output of my python script i.e. the value stored in component to be stored in CID. 

Can you help where I am going wrong? According to your last post, I have a feature that I want to send out and I am setting it to be the value of component. Still I get the following error :(

Python Exception <TypeError>: Could not convert attribute value to a supported attribute type.
Traceback (most recent call last):
  File "<string>", line 41, in close
TypeError: Could not convert attribute value to a supported attribute type.
f_27(PythonFactory): PythonFactory failed to close properly
f_27(PythonFactory): A fatal error has occurred. Check the logfile above for details
takashi
Contributor
Forum|alt.badge.img+21
  • takashiContributor
  • Contributor
  • November 3, 2017
sraapril wrote:

Hi @david_r,

I'm kind of stuck again. I finished the coding script I wanted to implement and this is how my new code looks like


import fme
from fmeobjects import FMEFeature
import numpy

class FeatureProcessor(object):
    def __init__(self):
        self.CID = 0
    def input(self,feature):
        bgt_pt = feature.getAttribute('bgt_points')
        las_pt = feature.getAttribute('las_points')
        
distance_list = []
        
        def coordinateMatch(bgt_pt,las_pt):
            for b1, b2 in enumerate(bgt_pt):
                for l1, l2 in enumerate(las_pt):
                    bp1 = bgt_pt[b1][0]
                    bp2 = bgt_pt[b1][1]
                    lp1 = las_pt[l1][0]
                    lp2 = las_pt[l1][1]
                    d = numpy.sqrt((bp1 - lp1)*(bp1 - lp1) + (bp2 - lp2)*(bp2 - lp2))
                    distance_list.append((las_pt[l1][2], d))

            component = min(distance_list, key = lambda t: t[1])
            return component

        bgt_pt = bgt_pt.split(',')
        las_pt = las_pt.split(',')
        
        las_pt = las_pt[:-1]

        bgt_pt = [float(i) for i in bgt_pt]
        las_pt = [float(i) for i in las_pt]

        las_pt = [i for i in zip(*[iter(las_pt)]*3)]
        bgt_pt = [i for i in zip(*[iter(bgt_pt)]*2)]

        self.CID = coordinateMatch(bgt_pt,las_pt)
    def close(self):
        new_feature = FMEFeature()
        new_feature.setAttribute("CID", self.CID)
        self.pyoutput(new_feature)<br>

I have created an attribute called CID and assigned it to NULL in my workflow. I want the output of my python script i.e. the value stored in component to be stored in CID. 

Can you help where I am going wrong? According to your last post, I have a feature that I want to send out and I am setting it to be the value of component. Still I get the following error :(

Python Exception <TypeError>: Could not convert attribute value to a supported attribute type.
Traceback (most recent call last):
  File "<string>", line 41, in close
TypeError: Could not convert attribute value to a supported attribute type.
f_27(PythonFactory): PythonFactory failed to close properly
f_27(PythonFactory): A fatal error has occurred. Check the logfile above for details
I think the error is caused by that the variable self.CID contains a tuple consisting of two values, when it is going to be set to a feature attribute with the FMEFeature.setAttribute method called in the close method. Check the value type returned by the coordinateMatch function.

 

 

david_r
Evangelist
  • david_rEvangelist
  • November 3, 2017
takashi wrote:
I think the error is caused by that the variable self.CID contains a tuple consisting of two values, when it is going to be set to a feature attribute with the FMEFeature.setAttribute method called in the close method. Check the value type returned by the coordinateMatch function.

 

 

I agree with Takashi, it's probably self.CID containing something that the fmeobjects.setAttribute() function doesn't know how to handle.

 

Try inserting some logging into your script at strategic places to see what is going on, e.g.

 

FMELogFile().logMessageString('self.CID is: ' + repr(self.CID))
You'll need to modify your import for this to work, e.g.

 

from fmeobjects import FMEFeature, FMELogFile
S
Forum|alt.badge.img+1
  • sraapril
  • Author
  • November 3, 2017
sraapril wrote:

Hi @david_r,

I'm kind of stuck again. I finished the coding script I wanted to implement and this is how my new code looks like


import fme
from fmeobjects import FMEFeature
import numpy

class FeatureProcessor(object):
    def __init__(self):
        self.CID = 0
    def input(self,feature):
        bgt_pt = feature.getAttribute('bgt_points')
        las_pt = feature.getAttribute('las_points')
        
distance_list = []
        
        def coordinateMatch(bgt_pt,las_pt):
            for b1, b2 in enumerate(bgt_pt):
                for l1, l2 in enumerate(las_pt):
                    bp1 = bgt_pt[b1][0]
                    bp2 = bgt_pt[b1][1]
                    lp1 = las_pt[l1][0]
                    lp2 = las_pt[l1][1]
                    d = numpy.sqrt((bp1 - lp1)*(bp1 - lp1) + (bp2 - lp2)*(bp2 - lp2))
                    distance_list.append((las_pt[l1][2], d))

            component = min(distance_list, key = lambda t: t[1])
            return component

        bgt_pt = bgt_pt.split(',')
        las_pt = las_pt.split(',')
        
        las_pt = las_pt[:-1]

        bgt_pt = [float(i) for i in bgt_pt]
        las_pt = [float(i) for i in las_pt]

        las_pt = [i for i in zip(*[iter(las_pt)]*3)]
        bgt_pt = [i for i in zip(*[iter(bgt_pt)]*2)]

        self.CID = coordinateMatch(bgt_pt,las_pt)
    def close(self):
        new_feature = FMEFeature()
        new_feature.setAttribute("CID", self.CID)
        self.pyoutput(new_feature)<br>

I have created an attribute called CID and assigned it to NULL in my workflow. I want the output of my python script i.e. the value stored in component to be stored in CID. 

Can you help where I am going wrong? According to your last post, I have a feature that I want to send out and I am setting it to be the value of component. Still I get the following error :(

Python Exception <TypeError>: Could not convert attribute value to a supported attribute type.
Traceback (most recent call last):
  File "<string>", line 41, in close
TypeError: Could not convert attribute value to a supported attribute type.
f_27(PythonFactory): PythonFactory failed to close properly
f_27(PythonFactory): A fatal error has occurred. Check the logfile above for details
@takashi, @david_r

 

 

Thanks a lot guys!! That was it. It was indeed returning a tuple and the probably FME did not know how to handle it. Since I only wanted the first element of the tuple, I modified my code to retrieve that and it works wonderfully well :-)

 

david_r
Evangelist
  • david_rEvangelist
  • November 3, 2017
sraapril wrote:
@takashi, @david_r

 

 

Thanks a lot guys!! That was it. It was indeed returning a tuple and the probably FME did not know how to handle it. Since I only wanted the first element of the tuple, I modified my code to retrieve that and it works wonderfully well :-)

 

Good to hear. I really recommend the fmeobjects API documentation if you haven't really looked at it before: http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html

 

By far the most important part to understand is the FMEFeature class.
S
Forum|alt.badge.img+1
  • sraapril
  • Author
  • November 8, 2017

Hello again @takashi, @david_r

I realized that I indeed want to return a tuple from my python program. I am able to do it successfully. 

components = coordinateMatch(bgt_pt,las_pt)
for item1, item2 in components:
	newFeature = fmeobjects.FMEFeature()
	newFeature.setAttribute('CID', item1)
        newFeature.setAttribute('Class', item2)
        self.pyoutput(newFeature)

So, my output looks like this

0684Q00000ArKwoQAF.png 

So far so good.

Now, the values I have obtained here must be used further in the workflow. So, it would be ideal if it will be stored as a list i.e. List.CID and List.Class where I could access the values as 

List.CID{0} = 21697

List.CID{1} = 22420

List.Class{0} = 6

List.Class{1} = 6

and so on.

However, no matter how I set my attribute i.e. to a pre-existing list etc, it does not seem to work. Requesting you to please help! 

S
Forum|alt.badge.img+1
  • sraapril
  • Author
  • November 8, 2017
sraapril wrote:

Hello again @takashi, @david_r

I realized that I indeed want to return a tuple from my python program. I am able to do it successfully. 

components = coordinateMatch(bgt_pt,las_pt)
for item1, item2 in components:
	newFeature = fmeobjects.FMEFeature()
	newFeature.setAttribute('CID', item1)
        newFeature.setAttribute('Class', item2)
        self.pyoutput(newFeature)

So, my output looks like this

0684Q00000ArKwoQAF.png 

So far so good.

Now, the values I have obtained here must be used further in the workflow. So, it would be ideal if it will be stored as a list i.e. List.CID and List.Class where I could access the values as 

List.CID{0} = 21697

List.CID{1} = 22420

List.Class{0} = 6

List.Class{1} = 6

and so on.

However, no matter how I set my attribute i.e. to a pre-existing list etc, it does not seem to work. Requesting you to please help! 

Ah! I figured it out. I thought i'll keep the question for some other user nonetheless.

 

 

Solution

 

  • I exposed the attributes CID and Class in Python caller
  • I created a list using an aggregator.
Now I can access them as list.CID and list.Class :-)

 

 

takashi
Contributor
Forum|alt.badge.img+21
  • takashiContributor
  • Contributor
  • November 8, 2017
sraapril wrote:

Hello again @takashi, @david_r

I realized that I indeed want to return a tuple from my python program. I am able to do it successfully. 

components = coordinateMatch(bgt_pt,las_pt)
for item1, item2 in components:
	newFeature = fmeobjects.FMEFeature()
	newFeature.setAttribute('CID', item1)
        newFeature.setAttribute('Class', item2)
        self.pyoutput(newFeature)

So, my output looks like this

0684Q00000ArKwoQAF.png 

So far so good.

Now, the values I have obtained here must be used further in the workflow. So, it would be ideal if it will be stored as a list i.e. List.CID and List.Class where I could access the values as 

List.CID{0} = 21697

List.CID{1} = 22420

List.Class{0} = 6

List.Class{1} = 6

and so on.

However, no matter how I set my attribute i.e. to a pre-existing list etc, it does not seem to work. Requesting you to please help! 

Yes, the Aggregator (or ListBuilder) can be used here. However, you can also create a list attribute with Python script. Consider individual list elements as feature attributes whose name contains index number surrounded by curly brackets. e.g.

 

        newFeature = fmeobjects.FMEFeature()
        components = coordinateMatch(bgt_pt,las_pt)
        for i, (item1, item2) in enumerate(components):
            newFeature.setAttribute('List{%d}.CID' % i, item1)
            newFeature.setAttribute('List{%d}.Class' % i, item2)
        self.pyoutput(newFeature) 

Reply


Helpful Members This Week

Community Stats

30,980
Posts
117,409
Replies
38,765
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.