Skip to main content
Question

PythonCaller all values for each attribute


Forum|alt.badge.img

I know that in PythonCaller processing is feature wise and each attribute can be access by feature.getAttribute().

But for my task I need all values of attribute (e.q. to check min, max, uniqueness etc).

I know that I can check those e.q. in AttributeValidator but for current task I need them in PythonCaller.

Is there some way to accomplish this using fmeobjects?

Something like getAllAttributeNames() but for let say column not line?

Thanks,

Submi

13 replies

danilo_fme
Evangelist
Forum|alt.badge.img+41
  • Evangelist
  • September 4, 2017

Hi @submi,

In 2015 Year i did ask about names in PythonCaller.

Link: Get Attributes

Thanks,

Danilo


david_r
Evangelist
  • September 4, 2017

How about using a StatisticsCalculator rather than the PythonCaller?


Forum|alt.badge.img
  • Author
  • September 4, 2017
david_r wrote:

How about using a StatisticsCalculator rather than the PythonCaller?

Yea I know how to do this stuff outside of PythonCaller - noo problem. But for whatever reasen I need then in PythonCaller...

david_r
Evangelist
  • September 4, 2017
submi wrote:
Yea I know how to do this stuff outside of PythonCaller - noo problem. But for whatever reasen I need then in PythonCaller...
Can you maybe post a concrete example of input features and show us what you expect as a result? That'll make it a lot easier to help.

Forum|alt.badge.img
  • Author
  • September 4, 2017
danilo_fme wrote:

Hi @submi,

In 2015 Year i did ask about names in PythonCaller.

Link: Get Attributes

Thanks,

Danilo

I need all values for each attribute not attribute names though.

Forum|alt.badge.img
  • Author
  • September 4, 2017
david_r wrote:
Can you maybe post a concrete example of input features and show us what you expect as a result? That'll make it a lot easier to help.
IDT11122134443417Let say that for ID a need to check uniqueness (so in line 4 and 5 will report duplicates).

 

And for T1 I need Min and Max (1 and 44) etc.

 

I know that it's easy outside of PythonCaller but since I doing other stuff there I would prefer to do those thing in PythonCaller as well. It's more question - can I work in PythonCaller let say column wise?

david_r
Evangelist
  • September 4, 2017

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.


Forum|alt.badge.img
  • Author
  • September 4, 2017
david_r wrote:

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.

 

Thanks, I will start with Python structures. I was just wondering if there is something in fmeobjects for such things.

 


david_r
Evangelist
  • September 4, 2017
submi wrote:

 

Thanks, I will start with Python structures. I was just wondering if there is something in fmeobjects for such things.

 

No, you'll have to implement that yourself.

 

Just in case, here's the complete fmeobjects documentation where you can find all the classes and methods available to you:

 

http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html

 


Forum|alt.badge.img
  • Author
  • September 4, 2017
david_r wrote:
No, you'll have to implement that yourself.

 

Just in case, here's the complete fmeobjects documentation where you can find all the classes and methods available to you:

 

http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html

 

Thanks @david_r

 


danilo_fme
Evangelist
Forum|alt.badge.img+41
  • Evangelist
  • September 4, 2017
david_r wrote:

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.

Great job @david_r

 


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Contributor
  • September 4, 2017

Hi submi,

If you just want to transfer all attributes from input feature to a newly created, it's as simple as:

for k in feature.getAllAttributeNames():
    newfeature.setAttribute(k,feature.getAttribute(k))

The trick is, as mentioned by others, to use the "attribute names" collection/list.

Cheers


dbaldacchino1
Supporter
Forum|alt.badge.img+11
david_r wrote:

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.

Thanks a lot @david_r, your example above really helped me get started with my first Python script and understand how you can store data as features pass through for use when needed etc.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings