Question

Evaluating a data set for errors of omission and commission

  • 22 August 2016
  • 3 replies
  • 0 views

I am evaluating FME and am wondering how the software suite can assist me in evaluating an existing geospatial dataset for errors of omission and commission. Any feedback would be appreciated.


3 replies

Userlevel 2
Badge +12

Look for transformers like GeometryValidator and AttributeValidator to test for geometry and attribute value errors. If you are looking for gaps or overlaps in polygon data the AreaonAreaOverlayer is the transformer to use.

Badge +9

@pbraun Here are some links to SafeSoftware content that give an overview of some data validation steps to take. I think they provide a very good highlight of what you can achieve through FME.

But for testing the data values just use the AttributeValidator (2016+) as @erik_jane says as this will go a long way in attribute testing. These presentations are pre 2016 so don't mention this. But still they should help you in picking out some transformers for evaluating your data.

http://blog.safe.com/2014/11/data-quality-checklist/

http://www.slideshare.net/SafeSoftware/validation-and-quality-control-with-fme-and-fme-server

http://www.slideshare.net/SafeSoftware/the-ultimate-geospatial-data-validation-checklist

Userlevel 4
Badge +25
Although it's not going to help you right now, we're working on a tutorial for data validation and QA with FME (I was literally working on it before stopping to check questions on here). So keep an eye out for that in the near future.

 

 

In this case, if it is attribute information (rather than geometry) the Tester transformer (or TestFilter) is a good way to check for problems like that. Check out the section on filter transformers in the Practical Transformer Use chapter of our training manual.

 

 

Reply