Skip to main content
  • EN

I have a dataset which needs joining. I need FME to identify which attribute values differ between touching features along the split, preferably producing a single attribute containing a message eg. "These attributes differ: #, #, #, etc". There are hundreds of attributes and I can't think of an easy way to do it.

Thanks!

Hi @jono

 

Could you specify what you mean with 'joining' and 'touching' features? Do I understand correctly that you want to get all attributes of all features touching the feature considered and then check what attributes are present in the neighboring features but not in the feature considered?

 

In this case, there might be an 1-N relation unless you are certain there isn't

 

What your looking for is essentially a DifferenceDetector. What I've done in similar situations is transfer the attributes from one feature (reference feature) to the other feature (comparison feature). I used a featureMerger on a key attribute, but you may need to use a spatialRelator instead. All of the attributes were prefixed with _ref_.

I then used a pythonCaller to loop through all the attributes and testes to see if _ref_X matched X, if not, then the attribute name was appended to a list attribute (_changeList{}).

You could then use a ListConcatenator to produce the single attribute you want.

Thanks jeroensiers, I'm sorry I should have been clearer. This only applies to lines which touch at the endpoints or areas which share a boundary. For example where 2 lines won't join (LineJoiner) or where 2 areas won't merge (Dissolver) only because of attribute differences each instance will need to be investigated to determine which attribute value is correct. It would be really helpful if it listed those attributes which differ between the two features.

 

Thanks.

 

 

What your looking for is essentially a DifferenceDetector. What I've done in similar situations is transfer the attributes from one feature (reference feature) to the other feature (comparison feature). I used a featureMerger on a key attribute, but you may need to use a spatialRelator instead. All of the attributes were prefixed with _ref_.

I then used a pythonCaller to loop through all the attributes and testes to see if _ref_X matched X, if not, then the attribute name was appended to a list attribute (_changeList{}).

You could then use a ListConcatenator to produce the single attribute you want.

Thanks jdh, LineJoiner and Dissolver both produce a nice List when Accumulation mode=Merge attributes is selected, however I haven't come up with an easy solution to comparing the list attributes. Maybe it's time to learn Python...

 

Wouldn't it be nice if there was a downloadable custom transformer for 'DifferenceDetector'?

 

 

Thanks jdh, LineJoiner and Dissolver both produce a nice List when Accumulation mode=Merge attributes is selected, however I haven't come up with an easy solution to comparing the list attributes. Maybe it's time to learn Python...

 

Wouldn't it be nice if there was a downloadable custom transformer for 'DifferenceDetector'?

 

 

Sad;y, I'm not allowed to post transformers developed at work to the hub.

 

 

What your looking for is essentially a DifferenceDetector. What I've done in similar situations is transfer the attributes from one feature (reference feature) to the other feature (comparison feature). I used a featureMerger on a key attribute, but you may need to use a spatialRelator instead. All of the attributes were prefixed with _ref_.

I then used a pythonCaller to loop through all the attributes and testes to see if _ref_X matched X, if not, then the attribute name was appended to a list attribute (_changeList{}).

You could then use a ListConcatenator to produce the single attribute you want.

Understood, thanks for pointing me in the right direction.

 

 

Understood, thanks for pointing me in the right direction.

 

 

The key fmeobjects methods here are

 

getAllAttributeNames() -> get all the attribute names

 

getAttribute(attrName) -> get the value of the attribute

 

setAttribute(attrName, attrValue) -> for the output changelist

 

 

There is some basic python fme explantions here:

 

https://knowledge.safe.com/articles/706/python-and-fme-basics.html

 

 

the fmeobjectsAPI is http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html

 

 

stackoverflow is an excellent resource for general python questions

 

 

Hi @jono 

The python code provided below will for every feature look for the list. For the first element it will get all attributes and the values of these attributes and store them in a dictionary. Next the code will loop through all attributes of every list element and if the value differs from the value of the first element, the attribute name is listed in the attribute 'attribute_with_different_values'.

0684Q00000ArKKuQAN.png

import fme
import fmeobjects
import re

class FeatureProcessor(object):
    def __init__(self):
        self.name_list = '_list'
        
    def input(self,feature):
        
        attributes = feature.getAllAttributeNames()
        
        # Keep only those attributenames linked to the first feature of the list and only keep the
        # name of the attribute itself
        attributes_to_compare = lre.search('(?<=\.).*$', x).group(0) for x in attributes if re.search('\{0\}\.', x)]
        
        # Store those attributenames in a dictionary having the value of the attribute as value
        attributes_first_element = {}

        for cAttribute in attributes_to_compare:
            attributes_first_elementÂcAttribute] = feature.getAttribute("{}{{0}}.{}".format(self.name_list, cAttribute))

        # Now loop through all attributes for all features and check which of them have a different value
        attributes_with_different_values = set()
        
        attributes = ox for x in attributes if re.search('\{\d+\}\.', x)]
        
        for cAttribute in attributes:
            value = feature.getAttribute(cAttribute)
            name_attribute = re.search('(?<=\.).*$', cAttribute).group(0)
            
            if value != attributes_first_elementbname_attribute]:
                
                attributes_with_different_values.add(name_attribute)
                
        # Store all the attributes that have different values in one attribute to export
        feature.setAttribute('attribute_with_different_values', ','.join(attributes_with_different_values))
        self.pyoutput(feature)
        
    def close(self):
        pass
        
      

Hi @jono 

The python code provided below will for every feature look for the list. For the first element it will get all attributes and the values of these attributes and store them in a dictionary. Next the code will loop through all attributes of every list element and if the value differs from the value of the first element, the attribute name is listed in the attribute 'attribute_with_different_values'.

0684Q00000ArKKuQAN.png

import fme
import fmeobjects
import re

class FeatureProcessor(object):
    def __init__(self):
        self.name_list = '_list'
        
    def input(self,feature):
        
        attributes = feature.getAllAttributeNames()
        
        # Keep only those attributenames linked to the first feature of the list and only keep the
        # name of the attribute itself
        attributes_to_compare = lre.search('(?<=\.).*$', x).group(0) for x in attributes if re.search('\{0\}\.', x)]
        
        # Store those attributenames in a dictionary having the value of the attribute as value
        attributes_first_element = {}

        for cAttribute in attributes_to_compare:
            attributes_first_elementÂcAttribute] = feature.getAttribute("{}{{0}}.{}".format(self.name_list, cAttribute))

        # Now loop through all attributes for all features and check which of them have a different value
        attributes_with_different_values = set()
        
        attributes = ox for x in attributes if re.search('\{\d+\}\.', x)]
        
        for cAttribute in attributes:
            value = feature.getAttribute(cAttribute)
            name_attribute = re.search('(?<=\.).*$', cAttribute).group(0)
            
            if value != attributes_first_elementbname_attribute]:
                
                attributes_with_different_values.add(name_attribute)
                
        # Store all the attributes that have different values in one attribute to export
        feature.setAttribute('attribute_with_different_values', ','.join(attributes_with_different_values))
        self.pyoutput(feature)
        
    def close(self):
        pass
        
      

Many, many thanks @Jeroenstiers, you have gone far beyond the help that I expected!

 

 

Many, many thanks @Jeroenstiers, you have gone far beyond the help that I expected!

 

 

You're welcome!

 

Hope this solves the issue. Could you close the question if it did?

 

 

 

Hi @jono 

The python code provided below will for every feature look for the list. For the first element it will get all attributes and the values of these attributes and store them in a dictionary. Next the code will loop through all attributes of every list element and if the value differs from the value of the first element, the attribute name is listed in the attribute 'attribute_with_different_values'.

0684Q00000ArKKuQAN.png

import fme
import fmeobjects
import re

class FeatureProcessor(object):
    def __init__(self):
        self.name_list = '_list'
        
    def input(self,feature):
        
        attributes = feature.getAllAttributeNames()
        
        # Keep only those attributenames linked to the first feature of the list and only keep the
        # name of the attribute itself
        attributes_to_compare = lre.search('(?<=\.).*$', x).group(0) for x in attributes if re.search('\{0\}\.', x)]
        
        # Store those attributenames in a dictionary having the value of the attribute as value
        attributes_first_element = {}

        for cAttribute in attributes_to_compare:
            attributes_first_elementÂcAttribute] = feature.getAttribute("{}{{0}}.{}".format(self.name_list, cAttribute))

        # Now loop through all attributes for all features and check which of them have a different value
        attributes_with_different_values = set()
        
        attributes = ox for x in attributes if re.search('\{\d+\}\.', x)]
        
        for cAttribute in attributes:
            value = feature.getAttribute(cAttribute)
            name_attribute = re.search('(?<=\.).*$', cAttribute).group(0)
            
            if value != attributes_first_elementbname_attribute]:
                
                attributes_with_different_values.add(name_attribute)
                
        # Store all the attributes that have different values in one attribute to export
        feature.setAttribute('attribute_with_different_values', ','.join(attributes_with_different_values))
        self.pyoutput(feature)
        
    def close(self):
        pass
        
      

This works brilliantly and will save me so much time, well done!  I'm sure that this code will be useful for so many people.  Thanks again.

 

 

Reply


Community Stats

30,126
Posts
114,374
Replies
37,765
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.