Hi everyone, I am now doing data quality inspection to an esri personal geo database with multiple layers, and I have to calculate how many 0 or null values there are in every attribute of each layer(or table) in the database. I have tried to figure out some automation method with FME but in vain. Can anybody give some instructon or advice?
Solved
data examination
+1- Contributor
Best answer by bwn
I guess one solution generally is to use NullAttributeMapper + ChangeDetector.
Although there are higher performance solutions in coding a check into PythonCaller to pass through Features and perform a per Feature Check with Eg. fmeobjects.FMEFeature.isAttributeNull() method, but won’t go into that here.
Starting with sample data.
Send these to NullAttributeMappers to overwrite the original Null and Zero values with flagging text strings “NullValue” and “ZeroValue”. Then configure ChangeDetector to build per Feature the Lists of Attribute Names that had either a Zero or Null.
Finally send this to StatisticsCalculator to get an overview of how many Nulls and Zeros per Feature Class/Table. Note the “Group By Mode” and “Group By” Parameters need to be set carefully.
Gives the summary Count of Zeros and Nulls per Feature Class/Table Name, per Field Name.
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+26- Evangelist
- Best Answer
I guess one solution generally is to use NullAttributeMapper + ChangeDetector.
Although there are higher performance solutions in coding a check into PythonCaller to pass through Features and perform a per Feature Check with Eg. fmeobjects.FMEFeature.isAttributeNull() method, but won’t go into that here.
Starting with sample data.
Send these to NullAttributeMappers to overwrite the original Null and Zero values with flagging text strings “NullValue” and “ZeroValue”. Then configure ChangeDetector to build per Feature the Lists of Attribute Names that had either a Zero or Null.
Finally send this to StatisticsCalculator to get an overview of how many Nulls and Zeros per Feature Class/Table. Note the “Group By Mode” and “Group By” Parameters need to be set carefully.
Gives the summary Count of Zeros and Nulls per Feature Class/Table Name, per Field Name.
+1- Contributor
I guess one solution generally is to use NullAttributeMapper + ChangeDetector.
Although there are higher performance solutions in coding a check into PythonCaller to pass through Features and perform a per Feature Check with Eg. fmeobjects.FMEFeature.isAttributeNull() method, but won’t go into that here.
Starting with sample data.
Send these to NullAttributeMappers to overwrite the original Null and Zero values with flagging text strings “NullValue” and “ZeroValue”. Then configure ChangeDetector to build per Feature the Lists of Attribute Names that had either a Zero or Null.
Finally send this to StatisticsCalculator to get an overview of how many Nulls and Zeros per Feature Class/Table. Note the “Group By Mode” and “Group By” Parameters need to be set carefully.
Gives the summary Count of Zeros and Nulls per Feature Class/Table Name, per Field Name.
Thank you so much for your detailed instructuion. That really helped me out.