I have a dataset which needs joining. I need FME to identify which attribute values differ between touching features along the split, preferably producing a single attribute containing a message eg. "These attributes differ: #, #, #, etc". There are hundreds of attributes and I can't think of an easy way to do it.
Thanks!
Best answer by jeroenstiers
Hi @jono
The python code provided below will for every feature look for the list. For the first element it will get all attributes and the values of these attributes and store them in a dictionary. Next the code will loop through all attributes of every list element and if the value differs from the value of the first element, the attribute name is listed in the attribute 'attribute_with_different_values'.
import fme
import fmeobjects
import re
class FeatureProcessor(object):
def __init__(self):
self.name_list = '_list'
def input(self,feature):
attributes = feature.getAllAttributeNames()
# Keep only those attributenames linked to the first feature of the list and only keep the
# name of the attribute itself
attributes_to_compare = [re.search('(?<=\.).*$', x).group(0) for x in attributes if re.search('\{0\}\.', x)]
# Store those attributenames in a dictionary having the value of the attribute as value
attributes_first_element = {}
for cAttribute in attributes_to_compare:
attributes_first_element[cAttribute] = feature.getAttribute("{}{{0}}.{}".format(self.name_list, cAttribute))
# Now loop through all attributes for all features and check which of them have a different value
attributes_with_different_values = set()
attributes = [x for x in attributes if re.search('\{\d+\}\.', x)]
for cAttribute in attributes:
value = feature.getAttribute(cAttribute)
name_attribute = re.search('(?<=\.).*$', cAttribute).group(0)
if value != attributes_first_element[name_attribute]:
attributes_with_different_values.add(name_attribute)
# Store all the attributes that have different values in one attribute to export
feature.setAttribute('attribute_with_different_values', ','.join(attributes_with_different_values))
self.pyoutput(feature)
def close(self):
pass
+7Could you specify what you mean with 'joining' and 'touching' features? Do I understand correctly that you want to get all attributes of all features touching the feature considered and then check what attributes are present in the neighboring features but not in the feature considered?
In this case, there might be an 1-N relation unless you are certain there isn't
+40What your looking for is essentially a DifferenceDetector. What I've done in similar situations is transfer the attributes from one feature (reference feature) to the other feature (comparison feature). I used a featureMerger on a key attribute, but you may need to use a spatialRelator instead. All of the attributes were prefixed with _ref_.
I then used a pythonCaller to loop through all the attributes and testes to see if _ref_X matched X, if not, then the attribute name was appended to a list attribute (_changeList{}).
You could then use a ListConcatenator to produce the single attribute you want.
Thanks.
What your looking for is essentially a DifferenceDetector. What I've done in similar situations is transfer the attributes from one feature (reference feature) to the other feature (comparison feature). I used a featureMerger on a key attribute, but you may need to use a spatialRelator instead. All of the attributes were prefixed with _ref_.
I then used a pythonCaller to loop through all the attributes and testes to see if _ref_X matched X, if not, then the attribute name was appended to a list attribute (_changeList{}).
You could then use a ListConcatenator to produce the single attribute you want.
Wouldn't it be nice if there was a downloadable custom transformer for 'DifferenceDetector'?
+40Wouldn't it be nice if there was a downloadable custom transformer for 'DifferenceDetector'?
What your looking for is essentially a DifferenceDetector. What I've done in similar situations is transfer the attributes from one feature (reference feature) to the other feature (comparison feature). I used a featureMerger on a key attribute, but you may need to use a spatialRelator instead. All of the attributes were prefixed with _ref_.
I then used a pythonCaller to loop through all the attributes and testes to see if _ref_X matched X, if not, then the attribute name was appended to a list attribute (_changeList{}).
You could then use a ListConcatenator to produce the single attribute you want.
+40
getAllAttributeNames() -> get all the attribute names
getAttribute(attrName) -> get the value of the attribute
setAttribute(attrName, attrValue) -> for the output changelist
There is some basic python fme explantions here:
https://knowledge.safe.com/articles/706/python-and-fme-basics.html
the fmeobjectsAPI is http://docs.safe.com/fme/html/FME_Objects_Python_API/index.html
stackoverflow is an excellent resource for general python questions
+7Hi @jono
The python code provided below will for every feature look for the list. For the first element it will get all attributes and the values of these attributes and store them in a dictionary. Next the code will loop through all attributes of every list element and if the value differs from the value of the first element, the attribute name is listed in the attribute 'attribute_with_different_values'.
import fme
import fmeobjects
import re
class FeatureProcessor(object):
def __init__(self):
self.name_list = '_list'
def input(self,feature):
attributes = feature.getAllAttributeNames()
# Keep only those attributenames linked to the first feature of the list and only keep the
# name of the attribute itself
attributes_to_compare = [re.search('(?<=\.).*$', x).group(0) for x in attributes if re.search('\{0\}\.', x)]
# Store those attributenames in a dictionary having the value of the attribute as value
attributes_first_element = {}
for cAttribute in attributes_to_compare:
attributes_first_element[cAttribute] = feature.getAttribute("{}{{0}}.{}".format(self.name_list, cAttribute))
# Now loop through all attributes for all features and check which of them have a different value
attributes_with_different_values = set()
attributes = [x for x in attributes if re.search('\{\d+\}\.', x)]
for cAttribute in attributes:
value = feature.getAttribute(cAttribute)
name_attribute = re.search('(?<=\.).*$', cAttribute).group(0)
if value != attributes_first_element[name_attribute]:
attributes_with_different_values.add(name_attribute)
# Store all the attributes that have different values in one attribute to export
feature.setAttribute('attribute_with_different_values', ','.join(attributes_with_different_values))
self.pyoutput(feature)
def close(self):
pass
Hi @jono
The python code provided below will for every feature look for the list. For the first element it will get all attributes and the values of these attributes and store them in a dictionary. Next the code will loop through all attributes of every list element and if the value differs from the value of the first element, the attribute name is listed in the attribute 'attribute_with_different_values'.
import fme
import fmeobjects
import re
class FeatureProcessor(object):
def __init__(self):
self.name_list = '_list'
def input(self,feature):
attributes = feature.getAllAttributeNames()
# Keep only those attributenames linked to the first feature of the list and only keep the
# name of the attribute itself
attributes_to_compare = [re.search('(?<=\.).*$', x).group(0) for x in attributes if re.search('\{0\}\.', x)]
# Store those attributenames in a dictionary having the value of the attribute as value
attributes_first_element = {}
for cAttribute in attributes_to_compare:
attributes_first_element[cAttribute] = feature.getAttribute("{}{{0}}.{}".format(self.name_list, cAttribute))
# Now loop through all attributes for all features and check which of them have a different value
attributes_with_different_values = set()
attributes = [x for x in attributes if re.search('\{\d+\}\.', x)]
for cAttribute in attributes:
value = feature.getAttribute(cAttribute)
name_attribute = re.search('(?<=\.).*$', cAttribute).group(0)
if value != attributes_first_element[name_attribute]:
attributes_with_different_values.add(name_attribute)
# Store all the attributes that have different values in one attribute to export
feature.setAttribute('attribute_with_different_values', ','.join(attributes_with_different_values))
self.pyoutput(feature)
def close(self):
pass
+7
Hope this solves the issue. Could you close the question if it did?
Hi @jono
The python code provided below will for every feature look for the list. For the first element it will get all attributes and the values of these attributes and store them in a dictionary. Next the code will loop through all attributes of every list element and if the value differs from the value of the first element, the attribute name is listed in the attribute 'attribute_with_different_values'.
import fme
import fmeobjects
import re
class FeatureProcessor(object):
def __init__(self):
self.name_list = '_list'
def input(self,feature):
attributes = feature.getAllAttributeNames()
# Keep only those attributenames linked to the first feature of the list and only keep the
# name of the attribute itself
attributes_to_compare = [re.search('(?<=\.).*$', x).group(0) for x in attributes if re.search('\{0\}\.', x)]
# Store those attributenames in a dictionary having the value of the attribute as value
attributes_first_element = {}
for cAttribute in attributes_to_compare:
attributes_first_element[cAttribute] = feature.getAttribute("{}{{0}}.{}".format(self.name_list, cAttribute))
# Now loop through all attributes for all features and check which of them have a different value
attributes_with_different_values = set()
attributes = [x for x in attributes if re.search('\{\d+\}\.', x)]
for cAttribute in attributes:
value = feature.getAttribute(cAttribute)
name_attribute = re.search('(?<=\.).*$', cAttribute).group(0)
if value != attributes_first_element[name_attribute]:
attributes_with_different_values.add(name_attribute)
# Store all the attributes that have different values in one attribute to export
feature.setAttribute('attribute_with_different_values', ','.join(attributes_with_different_values))
self.pyoutput(feature)
def close(self):
pass