Question

PythonCaller all values for each attribute

  • 4 September 2017
  • 13 replies
  • 27 views

Badge

I know that in PythonCaller processing is feature wise and each attribute can be access by feature.getAttribute().

But for my task I need all values of attribute (e.q. to check min, max, uniqueness etc).

I know that I can check those e.q. in AttributeValidator but for current task I need them in PythonCaller.

Is there some way to accomplish this using fmeobjects?

Something like getAllAttributeNames() but for let say column not line?

Thanks,

Submi


13 replies

Userlevel 4
Badge +30

Hi @submi,

In 2015 Year i did ask about names in PythonCaller.

Link: Get Attributes

Thanks,

Danilo

Userlevel 4

How about using a StatisticsCalculator rather than the PythonCaller?

Badge

How about using a StatisticsCalculator rather than the PythonCaller?

Yea I know how to do this stuff outside of PythonCaller - noo problem. But for whatever reasen I need then in PythonCaller...
Userlevel 4
Yea I know how to do this stuff outside of PythonCaller - noo problem. But for whatever reasen I need then in PythonCaller...
Can you maybe post a concrete example of input features and show us what you expect as a result? That'll make it a lot easier to help.
Badge

Hi @submi,

In 2015 Year i did ask about names in PythonCaller.

Link: Get Attributes

Thanks,

Danilo

I need all values for each attribute not attribute names though.
Badge
Can you maybe post a concrete example of input features and show us what you expect as a result? That'll make it a lot easier to help.
IDT11122134443417Let say that for ID a need to check uniqueness (so in line 4 and 5 will report duplicates).

 

And for T1 I need Min and Max (1 and 44) etc.

 

I know that it's easy outside of PythonCaller but since I doing other stuff there I would prefer to do those thing in PythonCaller as well. It's more question - can I work in PythonCaller let say column wise?
Userlevel 4

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.

Badge

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.

 

Thanks, I will start with Python structures. I was just wondering if there is something in fmeobjects for such things.

 

Userlevel 4

 

Thanks, I will start with Python structures. I was just wondering if there is something in fmeobjects for such things.

 

No, you'll have to implement that yourself.

 

Just in case, here's the complete fmeobjects documentation where you can find all the classes and methods available to you:

 

http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html

 

Badge
No, you'll have to implement that yourself.

 

Just in case, here's the complete fmeobjects documentation where you can find all the classes and methods available to you:

 

http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html

 

Thanks @david_r

 

Userlevel 4
Badge +30

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.

Great job @david_r

 

Userlevel 1
Badge +22

Hi submi,

If you just want to transfer all attributes from input feature to a newly created, it's as simple as:

for k in feature.getAllAttributeNames():
    newfeature.setAttribute(k,feature.getAttribute(k))

The trick is, as mentioned by others, to use the "attribute names" collection/list.

Cheers

Badge +3

Yes, you can use regular Python data structures such as dictionaries, lists and sets to accumulate values between features. Here's an example that returns duplicate values in attribute T1:

import fme
import fmeobjects
from collections import Counter

class FeatureProcessor(object):
    def __init__(self):
        self.T1_values = []
        
    def input(self,feature):
        t1 = feature.getAttribute('T1')
        self.T1_values.append(t1)
        
    def close(self):
        f = fmeobjects.FMEFeature()
        T1_duplicates = [k for k,v in Counter(self.T1_values).items() if v>1]
        f.setAttribute('T1_duplicates', T1_duplicates)
        self.pyoutput(f)

The PythonCaller will output one single feature containing a list attribute T1_duplicates{} that contain all the duplicates found in the T1 attribute of all the features.

Hopefully it can give you some ideas.

Thanks a lot @david_r, your example above really helped me get started with my first Python script and understand how you can store data as features pass through for use when needed etc.

Reply