After the FME ETL process is completed, is there a way to validate the results automatically?
Page 1 / 1
Hi @fantomfoto,
In my opinion you need to create another Workspace to validate your Writer:
- Schema
- Attributes
- Geometry
Thanks,
Danilo
Hi @fantomfoto
you definitely can do some automatic validation, e.g. you can check if a table has indeed been written into your database or the number of files output into a folder matches the expected number. The validation depends on your data and what might go wrong or what you would like to confirm did not go wrong. If you describe your scenario in more details, I am sure community will be happy to share ideas and suggestions.
For a general idea of the types of validation you can implement into your workflow, please see this Knowledge Center Article.
And, generally speaking, you could leverage FeatureWriter / FeatureReader in place of the native Reader / Writers in FME to have your ETL and validation occur in the same workspace – but it can just as easily be split into two workspaces. Note that this validation process itself will more or less have to read all the data back into FME, so when dealing with very large datasets it may impart a measurable overhead (especially those datasets where you cannot leverage filters or SQL statements to limit the number of features actually coming into the workspace).
For a general idea of the types of validation you can implement into your workflow, please see this Knowledge Center Article.
And, generally speaking, you could leverage FeatureWriter / FeatureReader in place of the native Reader / Writers in FME to have your ETL and validation occur in the same workspace – but it can just as easily be split into two workspaces. Note that this validation process itself will more or less have to read all the data back into FME, so when dealing with very large datasets it may impart a measurable overhead (especially those datasets where you cannot leverage filters or SQL statements to limit the number of features actually coming into the workspace).
Hi @RylanAtSafe this article that you posted above is very interesting.