Any thoughts?
-Sean
Question
Easy way to remove all Empty attributes
I am looking for an easy/elegant way to remove all attributes that are empty. I have taken a look the NullAttributeReplacer and while that "could" work, I have to know/expose the name of every attribute. I was looking into writing a python script to get all the attributes, test if empty then remove but writing python in FME hasnt gotten easier over the releases.
Here is what I have so far (not working though)
import fmeobjects
# Template Function interface:
#def processFeature(feature):
# pass
# Template Class Interface:
class FeatureProcessor(object):
def __init__(self):
pass
def input(self,feature):
for tmpAttrName in feature.getAllAttributeNames():
tmpAttrVal= feature.getAttribute(tmpAttrName)
if tmpAttrVal is None:
feature.performFunction()
self.pyoutput(feature)
def close(self):
pass
Hi Sean,
i think this should help:
import fmeobjects
# Template Class Interface:
class FeatureProcessor(object):
def __init__(self):
pass
def input(self,feature):
for tmpAttrName in feature.getAllAttributeNames():
tmpAttrVal = feature.getAttribute(tmpAttrName)
if (tmpAttrVal is None) or (tmpAttrVal == ''):
feature.removeAttribute(tmpAttrName)
self.pyoutput(feature)
def close(self):
pass
+4Hi Sean,
To clarify, are you after a solution that checks whether ALL features have a null for a specific attribute?
The following article may help with the usage of Python and attribute checking and removal: http://fmepedia.safe.com/articles/How_To/Extracting-a-schema-subset-for-dynamic-schemas
+7- Contributor
Hi Sean,
I had a bit of a play with to see if there was a way to do this is without Python in case you are interested - here are the steps:
- Pass your data into an AttributeExploder - this creates an attribute name/vluae pair for each attribute in your source data (being _attr_name and _attr_value)
- Using a Tester select those features where _attr_value is NULL (operator = Attribute is Null)
- Remove the duplicates with a DuplicateRemover (key attribute = _attr_name)
- Pass the data to an Aggregator transformer with the "Attribute to Concatenate = _attr_name) - your attr_name attribute is a now a comma delimited string of attribute names with NULL values (one record only)
- Using an AttributeFileWriter write the stringin _attr_name out to a txt file.
- Pass output from the AttributeFileWriter to a FeatureHolder transformer AND also pass the original source data (unprocessed) to the FeatureHolder
- Now use a AttributeFileReader to read in your single comma delimited txt string of attribute names to be deleted - this will get saved against a <new_attribute>
- Finally use an FMEFunctionCaller transformer with the @RemoveAttributes function with syntax like @RemoveAttributes( [new_attribute] )
Mike
Thanks for the suggestions folks!
The python that Hendrik wrote works perfect. I like your method Mike but since we are running over 1600 transformers in the workbench as is, I would like to keep the number down as much as possible.
Thanks,
Sean
+3- Contributor
Hi all, I'm having trouble getting this to work. When I connect my data source to the PythonCaller it automatically maps all the existing attribute fields to the output? I tried hiding all the output attribute values but that didn't seem to make a difference?
+1Hi
I have followed Michael Oberdries steps for using FME transformers for this but have got stuck on how to do step 7 & 8
7. Now use a AttributeFileReader to read in your single comma delimited txt string of attribute names to be deleted - this will get saved against a <new_attribute>
8. Finally use an FMEFunctionCaller transformer with the @RemoveAttributes function with syntax like @RemoveAttributes( [new_attribute] )
Can you give me some more advice please @MOberdries?
+15- Contributor
Explode the attributes.
Listbuilder on attr_name. Make list out of attr_value.
ListDuplicate remover.
Test if list_element_count =1.
Of those, test if list{0} has a value.
Yields a table of attributes where all records have empty values.
(you are talking about empty attributes, which exclude NULL or MISSING as I interpret it )
+19- Contributor
In the current FME (2014 or later), you can use the NullAttributeMapper to remove all empty and null attributes easily.
- Map: All Attributes
- If Attribute Value Is: Null Empty
- Map To: Missing
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.