Skip to main content
Solved

I would like to compare two sets in a PythonCaller and output the difference between these two sets.

  • October 13, 2022
  • 6 replies
  • 62 views

Forum|alt.badge.img

Hi all,

 

In the last few days we've set many small steps leading up to the problem we now have. We have polygons, where one attribute of each polygon is a list of bird species, which occur in that area. This list is of course different for all polygons, the polygon-specific list.

 

We also have a list of bird species which should occur in that area. This list is the same for all polygons (in this subset), the general list.

 

Now, we would like to compare the list of each polygon, with the list for all polygons. We have tried many transformers, and our last option (at the moment) is the PythonCaller, where we hardcode the second list as a set, and compare it with the polygon-specific list, converted to a set. We group by the id of the polygon, because we want to compare each polygon-specific list to the general list. 

This is our Python-code:

import fme
import fmeobjects
 
x = {'item 1','item 2','item 3'}
 
class FeatureProcessor(object):
    def __init__(self):
        self.feature_list = []
        self.diff = {}
 
    def input(self, feature):
        self.feature_list.append(feature)
        self.diff += x.difference(set(feature.getAttribute('_list{}')))
 
    def close(self):
        pass
 
    def process_group(self):
        for feature in self.feature_list:
            feature.setAttribute("diff", self.diff)
            self.pyoutput(feature)
        self.feature_list = []
        self.diff = {}

And this is the error we get in the Translation log:

Python Exception <TypeError>: 'NoneType' object is not iterable
Traceback (most recent call last):
  File "<string>", line 14, in input
TypeError: 'NoneType' object is not iterable

We have a feeling that the answer is very simple, but we don't understand why the error is NoneType. And if this can be done by a regular transformer, we would be very glad to hear it.

 

All help is welcome, thanks in advance!

Best answer by ebygomm

I think you are overcomplicating it if you want to compare a list from each polygon individually

import fme
import fmeobjects
 
def processFeature(feature):
    x = {'item 1','item 2','item 3'}
    specieslist = feature.getAttribute('list{}.species')
    diff = list(set(x) - set(specieslist))
    feature.setAttribute('list{}.difference',diff)

Something like this will return a list of any items in x that don't appear in the species list

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

6 replies

david_r
Celebrity
  • 8391 replies
  • October 13, 2022

My first guess would be that feature.getAttribute('_list{}') returns None, possibly because the list is empty.

Try something like this instead, so that you default to an empty list, rather than None:

self.diff += x.difference(set(feature.getAttribute('_list{}') or []))

 


ebygomm
Influencer
Forum|alt.badge.img+44
  • Influencer
  • 3422 replies
  • Best Answer
  • October 13, 2022

I think you are overcomplicating it if you want to compare a list from each polygon individually

import fme
import fmeobjects
 
def processFeature(feature):
    x = {'item 1','item 2','item 3'}
    specieslist = feature.getAttribute('list{}.species')
    diff = list(set(x) - set(specieslist))
    feature.setAttribute('list{}.difference',diff)

Something like this will return a list of any items in x that don't appear in the species list


debbiatsafe
Safer
Forum|alt.badge.img+21
  • Safer
  • 648 replies
  • October 15, 2022

I agree that a group-by and a for-loop should not be required for the workflow you have described.

 

Also sharing a non-Python workflow example that should work based on the ListConcatenator and StringReplacer transformers. It should output the species from the general list that were not in the polygon specific list as either a regular attribute or list attribute.


Forum|alt.badge.img
  • Author
  • 4 replies
  • October 17, 2022

I think you are overcomplicating it if you want to compare a list from each polygon individually

import fme
import fmeobjects
 
def processFeature(feature):
    x = {'item 1','item 2','item 3'}
    specieslist = feature.getAttribute('list{}.species')
    diff = list(set(x) - set(specieslist))
    feature.setAttribute('list{}.difference',diff)

Something like this will return a list of any items in x that don't appear in the species list

Thanks a lot, @ebygomm​ , this works perfectly.

This was the first time for me working with the PythonCaller, so the only examples I could find, were those in the documentation. But that overcomplicated things a bit indeed...


Forum|alt.badge.img
  • Author
  • 4 replies
  • October 17, 2022

I agree that a group-by and a for-loop should not be required for the workflow you have described.

 

Also sharing a non-Python workflow example that should work based on the ListConcatenator and StringReplacer transformers. It should output the species from the general list that were not in the polygon specific list as either a regular attribute or list attribute.

Hi @debbiatsafe​, many thanks for your FME solution. That also works great!


Forum|alt.badge.img
  • Author
  • 4 replies
  • October 17, 2022

My first guess would be that feature.getAttribute('_list{}') returns None, possibly because the list is empty.

Try something like this instead, so that you default to an empty list, rather than None:

self.diff += x.difference(set(feature.getAttribute('_list{}') or []))

 

Thanks for your suggestion, @david_r​, it did work, but it gave me another error, which I couldn't resolve... 😁