Skip to main content
Solved

How to Detect a Schema difference (Dataset1 has X attributes, Dataset2 has X+or-n attributes)

  • January 22, 2016
  • 1 reply
  • 81 views
B

Very much a newbie. I'm somewhere around week two of actually touching FME. Have Searched the Transformers and Knowledge base and found close but not quite workflows.

Two Datasets. One published, one to be published (updated).

Need to compare the two. If # of attributes changed(addcol/deletecol) between the two, stop on error. Not caring so much about # of Records, data gets edited so those can change, care about attribute additions/deletions. This is step0.1 of a potentially complex data validation process that will likely take quite some time to develop the full flow. But need to start from somewhere.

Tester - pass = Both files have identical # attributes, and attribute type.

tester - FAIL = Different # of attributes, AND/OR attribute type difference

existence of fail halts process with error - "Illegal Schema Change".

Thought this would be a easy fast one to come up with....not so.

Thanks so much in advance, I'm going to keep trying different things, If I find something, I'll post.

Best answer by erik_jan

Instead of reading the data of both datasets you can read the schema using the Schema (Any format) reader.

This will give you a list of all attributes and datatypes of your datasets.

You will be able to detect differences using the Matcher, the ChangeDetector or similar transformers.

View original
Did this help you find an answer to your question?
This post is closed to further activity.
It may be a question with a best answer, an implemented idea, or just a post needing no comment.
If you have a follow-up or related question, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

1 reply

erik_jan
Contributor
Forum|alt.badge.img+17
  • erik_janContributor
  • Contributor
  • Best Answer
  • January 22, 2016

Instead of reading the data of both datasets you can read the schema using the Schema (Any format) reader.

This will give you a list of all attributes and datatypes of your datasets.

You will be able to detect differences using the Matcher, the ChangeDetector or similar transformers.

Helpful Members This Week

Community Stats

30,937
Posts
117,247
Replies
38,739
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.