Skip to main content
  • EN

I am basically doing AttributeValidation but am looking to find values that are blank or empty. Looks like I can't look for blanks or empty's within the AttributeValidation transformer. So, right now I have a testFilter that is checking each field if it is blank or empty. I am using a TestFilter because then I can specify each of the fields that I want to be checked. The problem is that if one record goes through the tester and on the first test we find out that field is blank, then I believe that same record will not go through the rest of the tests. I need to know each field is blank, not just the first one. I am looking for a suggestions on how to do this efficiently.

 

I am basically running a QC process on a large dataset. We have a bunch of business rules that need to be followed. So I am setting FME up to check all those rules. If a rule is broken, then an email will be sent to the person that introduced the error and then they can go back and fix the error.

 

So this one step is looking for fields that were left as blank or spaces so that the data editor can go back and change them to NULLS.

Does the validation rule "has value" not work for you?

Does the validation rule "has value" not work for you?

But I want to know if it doesn't have a value. Is there a way to negate the rule? Oh, I guess I could put only the one rule into the transformer and see what comes out the failed port...

 

I ran it and the failed includes Nulls. I am specifically looking for blanks or ' '

Hi @juliarozema,

The AttributeFilter transformer may also be of interest to you as it separates Null, missing and empty - but it can still only test on a single attribute at a time.

Hi @juliarozema I think it might be helpful to share with us what you want to do once you've found the records with empty string or only a space in all the various attributes.

 

For example do you just want to report which attribute has empty or space value, on that record? or do you want to replace them with some other default value? There might be different approaches that are suitable for different subsequent processes you want to perform.

Hi @juliarozema I think it might be helpful to share with us what you want to do once you've found the records with empty string or only a space in all the various attributes.

 

For example do you just want to report which attribute has empty or space value, on that record? or do you want to replace them with some other default value? There might be different approaches that are suitable for different subsequent processes you want to perform.

Hi Xiaomeng,

 

Good point. I am basically running a QC process on a large dataset. We have a bunch of business rules that need to be followed. So I am setting FME up to check all those rules. If a rule is broken, then an email will be sent to the person that introduced the error and then they can go back and fix the error.

 

So this one step is looking for fields that were left as blank or spaces so that the data editor can go back and change them to NULLS.

If you need to determine if all the target attributes are empty or null or missing, a possible way is to concatenate the values of them and test if the resulting string matches the empty string. For example, this Tester routes the feature to the Passed port if the A, B and C attributes all didn't have any value, routes the feature to Failed port otherwise.

If you're only interested in reporting, and you don't need to use this data later, then one possible way is to use a nullattributemapper to set all the nulls to a default value, e.g. null then use the attributevalidator to test for has value, this will then fail the empty attributes and report in a list which have failed

 

Coming late to the party, but would it work to use an attribute exploder, than filter attribute value on your test and then using attribute name to see which records did not pass the test. You can either remove attributes or just keep an ID field to also trace matches back to a certain record.

Reply


Community Stats

30,352
Posts
115,347
Replies
38,087
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.