I am writing a routine to check an incoming dataset before loading. The first step is to make sure the dataset has all the required attributes, the second step is to make sure the values in the attributes are valid. I am using the AttributeValidator for this second step. Unfortunately, when I am validating a value, I have to hardcode all the values in the Rule Configuration box. I would like to be able to read a table and get these values and populate it in the Rule Configuration box. How can I do this?

populate the values for the "in" validation rule from a list

Hi @tnarladni, at this time there is currently no way to import a list of rules into the AttributeValidator. I recommend up-voting on the Idea to have it added: Add import attributes button on AttributeValidator .

Hi @tnarladni, the Rule Configuration field in the AttributeValidator doesn't have any function that allows you to import the rule from an external table, but you can use an attribute value as a rule configuration here. For the In operator, if you add an attribute that stores comma-separated valid values to every input feature, you can then select it as the Rule Configuration.

For example, if the valid values are stored in a text file (a value per a row), read it with the Text File reader, concatenate the values with comma-separated format (Aggregator), merge it to all the features unconditionally (FeatureMerger), and set the concatenated values to the Rule Configuration in the AttributeValidator.

Hi @tnarladni, the Rule Configuration field in the AttributeValidator doesn't have any function that allows you to import the rule from an external table, but you can use an attribute value as a rule configuration here. For the In operator, if you add an attribute that stores comma-separated valid values to every input feature, you can then select it as the Rule Configuration.

For example, if the valid values are stored in a text file (a value per a row), read it with the Text File reader, concatenate the values with comma-separated format (Aggregator), merge it to all the features unconditionally (FeatureMerger), and set the concatenated values to the Rule Configuration in the AttributeValidator.

thanks @takashi. Always the FME boss! :)

Hi @tnarladni, at this time there is currently no way to import a list of rules into the AttributeValidator. I recommend up-voting on the Idea to have it added: Add import attributes button on AttributeValidator .

Thanks @TiaAtSafe! I up voted on that idea and a few others as well.

Why not use the list (from a CSV file or any other format) and validate being in the list by merging on the list values (FeatureMerger): Merged: value is in the list, Not Merged: invalid value.

Thanks for the suggestion @erik_jan. The reason why FeatureMerger would not work well in this case is because there are multiple fields that need to be validated, including whether or not there are nulls in the fields. It is cleaner to put all the validation rules into one transformer versus multiple featureMerger and testers.

OK, that makes sense. But was not clear from the description.

Hi @tnarladni, the Rule Configuration field in the AttributeValidator doesn't have any function that allows you to import the rule from an external table, but you can use an attribute value as a rule configuration here. For the In operator, if you add an attribute that stores comma-separated valid values to every input feature, you can then select it as the Rule Configuration.

For example, if the valid values are stored in a text file (a value per a row), read it with the Text File reader, concatenate the values with comma-separated format (Aggregator), merge it to all the features unconditionally (FeatureMerger), and set the concatenated values to the Rule Configuration in the AttributeValidator.

I am at another crossroad. @takashi, I would appreciate your thoughts/solution. First, I am using a featureReader and schema reader to compare the schemas. Then I read in an SDE table which has a list of fields, and all the codes associated with those fields. I use the aggregator to form a comma delimited list for each of the "field". Then using the FeatureMerger, I get just those lists that needs to be validated against. So here's the problem, the lists are all called Code. The only way to know which Code list to use is another field called "Field". Help!

accessroad.fmw

Hi @tnarladni, the Rule Configuration field in the AttributeValidator doesn't have any function that allows you to import the rule from an external table, but you can use an attribute value as a rule configuration here. For the In operator, if you add an attribute that stores comma-separated valid values to every input feature, you can then select it as the Rule Configuration.

For example, if the valid values are stored in a text file (a value per a row), read it with the Text File reader, concatenate the values with comma-separated format (Aggregator), merge it to all the features unconditionally (FeatureMerger), and set the concatenated values to the Rule Configuration in the AttributeValidator.

As far as I see your screenshot, not clear why you cannot create code lists with different names for each rule.

And, you have to merge the code list(s) to the features which you need to validate with the AttributeValidator. Looks like the attribute "Code" has been created with the Aggregator and it would be merged to the 11 features, but it won't be merged to the 180 features output from the FeatureReader_2. Is it your intention?

As far as I see your screenshot, not clear why you cannot create code lists with different names for each rule.

And, you have to merge the code list(s) to the features which you need to validate with the AttributeValidator. Looks like the attribute "Code" has been created with the Aggregator and it would be merged to the 11 features, but it won't be merged to the 180 features output from the FeatureReader_2. Is it your intention?

I attached the fmw because it was hard to explain. but essentially, I used the schema reader to check make sure the attributes are named correctly and are there. if that first test passed, I use the featureReader to read in the shapefile itself. so the tester (which has the field names as values) is used to featuremerged against the domain table (screenshot to show how it looks) to get the grouped "field" that is the same as the "name". the "code" is where all the rule configuration resides and in the aggregation process, I have field = NewExist and Code = Existing,New. I also have field = PermTemp and the Code = Permanent,Temporary. In the AttributeValidator, there's not a way to know which Code to use unless you know which Field you need.

So the 11 that is used in the FeatureMerger is the field names. and the 180 FeatureReader is the actual features

The domain table was something I created so if you think it should be done a different way, I am open to that as well.

Hi @tnarladni, the Rule Configuration field in the AttributeValidator doesn't have any function that allows you to import the rule from an external table, but you can use an attribute value as a rule configuration here. For the In operator, if you add an attribute that stores comma-separated valid values to every input feature, you can then select it as the Rule Configuration.

For example, if the valid values are stored in a text file (a value per a row), read it with the Text File reader, concatenate the values with comma-separated format (Aggregator), merge it to all the features unconditionally (FeatureMerger), and set the concatenated values to the Rule Configuration in the AttributeValidator.

Probably this workflow example could help you.

Input: a table containing possible code values for each field.

Field,Code
PermTemp,PT1
PermTemp,PT2
PermTemp,PT3
PublicPriv,PP1
PublicPriv,PP2
PublicPriv,PP3
NewExist,NE1
NewExist,NE2
NewExist,NE3
SurfaceMat,SM1
SurfaceMat,SM2
SurfaceMat,SM3

Output: a single feature containing validation rules for each field.

You can merge the feature containing validation rules to the data features from the FeatureReader, then validate the features with the AttributeValidator using the rules.

I am at another crossroad. @takashi, I would appreciate your thoughts/solution. First, I am using a featureReader and schema reader to compare the schemas. Then I read in an SDE table which has a list of fields, and all the codes associated with those fields. I use the aggregator to form a comma delimited list for each of the "field". Then using the FeatureMerger, I get just those lists that needs to be validated against. So here's the problem, the lists are all called Code. The only way to know which Code list to use is another field called "Field". Help!

accessroad.fmw

Probably this workflow example could help you.

Input: a table containing possible code values for each field.

Field,Code
PermTemp,PT1
PermTemp,PT2
PermTemp,PT3
PublicPriv,PP1
PublicPriv,PP2
PublicPriv,PP3
NewExist,NE1
NewExist,NE2
NewExist,NE3
SurfaceMat,SM1
SurfaceMat,SM2
SurfaceMat,SM3

Output: a single feature containing validation rules for each field.

You can merge the feature containing validation rules to the data features from the FeatureReader, then validate the features with the AttributeValidator using the rules.

Reply

Community Stats

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded