Solved

How to Detect a Schema difference (Dataset1 has X attributes, Dataset2 has X+or-n attributes)

  • 22 January 2016
  • 1 reply
  • 7 views

Very much a newbie. I'm somewhere around week two of actually touching FME. Have Searched the Transformers and Knowledge base and found close but not quite workflows.

Two Datasets. One published, one to be published (updated).

Need to compare the two. If # of attributes changed(addcol/deletecol) between the two, stop on error. Not caring so much about # of Records, data gets edited so those can change, care about attribute additions/deletions. This is step0.1 of a potentially complex data validation process that will likely take quite some time to develop the full flow. But need to start from somewhere.

Tester - pass = Both files have identical # attributes, and attribute type.

tester - FAIL = Different # of attributes, AND/OR attribute type difference

existence of fail halts process with error - "Illegal Schema Change".

Thought this would be a easy fast one to come up with....not so.

Thanks so much in advance, I'm going to keep trying different things, If I find something, I'll post.

icon

Best answer by erik_jan 22 January 2016, 19:29

View original

1 reply

Userlevel 2
Badge +12

Instead of reading the data of both datasets you can read the schema using the Schema (Any format) reader.

This will give you a list of all attributes and datatypes of your datasets.

You will be able to detect differences using the Matcher, the ChangeDetector or similar transformers.

Reply