Skip to main content
Question

Handling Data Validation Errors in FME Script: Capturing Invalid Records for Review

  • May 14, 2024
  • 1 reply
  • 110 views

leroy
Contributor
Forum|alt.badge.img+1
  • Contributor

In my scenario, I have an FME script that performs data validation. If the data is valid, it updates the database. However, if any records contain errors, I want to capture those records in an Excel file for users to review before uploading the data. In this error case, no data will be updated to database. 

So, the logic I'm aiming for is as follows:

  1. Validate the data.
  2. After validating all data, capture all invalid records to an Excel file. Then, terminate the FME script without uploading the data to the database.
  3. If all data is valid, proceed to upload it to the database.

I've attempted to create a flow with a terminator, but the Excel writer doesn't have a connector. Connecting the terminator to the output port of the TestFilter after validation would terminate the writing of the error Excel file as well.

Can anyone provide suggestions to solve this issue?

1 reply

nielsgerrits
VIP
Forum|alt.badge.img+54

Classic writers have no output ports but FeatureWriters have. When I discovered FeatureReaders and FeatureWriters, I switched and never used classic readers / writers anymore.

If I need a validation workflow it looks like this:

Sample workspace attached.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings