I'm using a schema mapper to update a number of attributes that have string values to a corresponding code. I'd like to catch features where one or more of the attributes to be mapped have a value that isn't in the schema mapping table.
There doesn't appear to be an easy way to do this, since some of the attributes on the feature are mapped, the feature exits the mapped port.
Has anyone come up with a good solution for this?
I want to say I've encountered this in the past, and I ended up doing separate SchemaMappers for each mapped attribute. I'm sure it creates excessive overhead processing, but that was the only thing I could figure out at the time.
We do something similar that could probably be expanded to do this.
We make sure we have a uniform source Value(s) and source Field(s) in our schemamapper file and extract those.
For the test data, we read that all in an ensure that we only keep the relevant fields in our schemamapper and attributeexplode those features.
Then we merge the field and value from the schemamapper to the field and value for each feature created in the attributeexploder.
Those are the main steps and while it is not perfect (e.g. where we have schemamapper with multiple filters, we are comparing each value of the filter rather than all together), it has made a big difference to finding those missing attribute values
We do something similar that could probably be expanded to do this.
We make sure we have a uniform source Value(s) and source Field(s) in our schemamapper file and extract those.
For the test data, we read that all in an ensure that we only keep the relevant fields in our schemamapper and attributeexplode those features.
Then we merge the field and value from the schemamapper to the field and value for each feature created in the attributeexploder.
Those are the main steps and while it is not perfect (e.g. where we have schemamapper with multiple filters, we are comparing each value of the filter rather than all together), it has made a big difference to finding those missing attribute values
We have seperated this into its own process, rather than including it into our scripts that runs often.
I think for one of my use cases as I'm always mapping strings to numbers then I can validate to see if fields are numeric after the schema mapper and identify unmapped fields that way.
For another process I'm mapping to new attribute names, I'd hoped to use the AttributeValidator after and could check if Attribute exists (since it won't exist if it hasn't been mapped) but I need to differentiate between null and missing, so I'll probably have to mess about with the NullAttributeMapper instead. Probably worth it though as i can probably replace 27 existing transformers with 3 :-)
In my case I'm mapping existing values to a new value with a new name. By using a NullAttributeMapper to map Null values to a string of Not Recorded, and including that as a valid lookup value. I can then validate that all the newly created attributes have a value. Attributes that don't have a valid lookup value end up as missing so I can then handle that. Final step is to set my Not Recorded back to Null.
So slightly convoluted but it seems to work ok