Hi everyone, I am now doing data quality inspection to an esri personal geo database with multiple layers, and I have to calculate how many 0 or null values there are in every attribute of each layer(or table) in the database. I have tried to figure out some automation method with FME but in vain. Can anybody give some instructon or advice?
Solved
data examination
+1- Contributor
Best answer by bwn
I guess one solution generally is to use NullAttributeMapper + ChangeDetector.
Although there are higher performance solutions in coding a check into PythonCaller to pass through Features and perform a per Feature Check with Eg. fmeobjects.FMEFeature.isAttributeNull() method, but won’t go into that here.
Starting with sample data.
Send these to NullAttributeMappers to overwrite the original Null and Zero values with flagging text strings “NullValue” and “ZeroValue”. Then configure ChangeDetector to build per Feature the Lists of Attribute Names that had either a Zero or Null.
Finally send this to StatisticsCalculator to get an overview of how many Nulls and Zeros per Feature Class/Table. Note the “Group By Mode” and “Group By” Parameters need to be set carefully.
Gives the summary Count of Zeros and Nulls per Feature Class/Table Name, per Field Name.
1 person upvoted this
+26- Evangelist
- Best Answer
I guess one solution generally is to use NullAttributeMapper + ChangeDetector.
Although there are higher performance solutions in coding a check into PythonCaller to pass through Features and perform a per Feature Check with Eg. fmeobjects.FMEFeature.isAttributeNull() method, but won’t go into that here.
Starting with sample data.
Send these to NullAttributeMappers to overwrite the original Null and Zero values with flagging text strings “NullValue” and “ZeroValue”. Then configure ChangeDetector to build per Feature the Lists of Attribute Names that had either a Zero or Null.
Finally send this to StatisticsCalculator to get an overview of how many Nulls and Zeros per Feature Class/Table. Note the “Group By Mode” and “Group By” Parameters need to be set carefully.
Gives the summary Count of Zeros and Nulls per Feature Class/Table Name, per Field Name.
+1- Contributor
bwn wrote:
I guess one solution generally is to use NullAttributeMapper + ChangeDetector.
Although there are higher performance solutions in coding a check into PythonCaller to pass through Features and perform a per Feature Check with Eg. fmeobjects.FMEFeature.isAttributeNull() method, but won’t go into that here.
Starting with sample data.
Send these to NullAttributeMappers to overwrite the original Null and Zero values with flagging text strings “NullValue” and “ZeroValue”. Then configure ChangeDetector to build per Feature the Lists of Attribute Names that had either a Zero or Null.
Finally send this to StatisticsCalculator to get an overview of how many Nulls and Zeros per Feature Class/Table. Note the “Group By Mode” and “Group By” Parameters need to be set carefully.
Gives the summary Count of Zeros and Nulls per Feature Class/Table Name, per Field Name.
Thank you so much for your detailed instructuion. That really helped me out.
Reply
Rich Text Editor, editor1
Editor toolbars
Press ALT 0 for help
Helpful Members This Week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.