Allow Logic in AttributeValidator Dialogue

I fully agree, +1 for me!

I would even take it a step further. It is currently already really nice that the AttributeValidator has the ability to perform one validation rule on multiple attributes. For example;

attribute_1,attribute_2,attribute_3 Has a Value

However, it will always use the 'AND' condition for each statement, and also 'between' the attributes in such a grouped statement. The above would thus boil down to

'attribute_1 Has a Value' AND 'attribute_2 Has a Value' AND 'attribute_3 Has a Value'.

I would really like to see that besides having the option to configure a 'composite logic expression' between individual attribute validator lines (like e.g. 1 AND (2 OR 3)), as is possible with e.g. a Tester or a TestFilter, to also be able to change the 'generic logic statement' for an 'attribute group condition' in the AttributeValidator.

E.g. for my example to be able to change the default 'between logic' to 'OR', so that the generic group statement would boil down to boil down to 'attribute_1 Has a Value' OR 'attribute_2 Has a Value' OR 'attribute_3 Has a Value'.

The reason I stumbled into this is related to trying to remove 'empty excel rows'. See also idea 'https://community.safe.com/s/idea/0874Q000000j0VfQAI/detail' and 'https://community.safe.com/s/idea/0874Q000000Tl9AQAS/detail'. At first I used a TestFilter to create a lengthy statement with attribute_1 Has a Value OR attribute_2 Has a Value, OR attribute_3 Has a Value ... For a brief moment I thought to have found an easy way to not enter all those individual lines, by simply using an AttributeValidator and selecting all attributes to check for a value. However, as the generic logic statement' for an 'attribute group condition' is currently locked to AND, I couldn't make it work :(.

In the end I made things a bit easier by using the AttributeConcatenaror custom transformer from takashi (thanks Takashi!), and combining that with a testfilter to see if the concatenated string has a value. However, the performance of a direct (lengty) testfilter vs that option is down by quite a bit. That said, the performance by using this option of using takashi's transformer is already way up from my own attempts to achieve the same with existing transformers.

This idea seems to be very similar to the following Ideas on the FME community:

1. https://community.safe.com/s/idea/0874Q000000TlLAQA0/detail
2. https://community.safe.com/s/idea/0874Q000000TktxQAC/detail

I've spotted your comments on a few related AttributeValidator threads, so you may have spotted my recent post in another thread. Apologies if this is spamming you.

I've built a BulkAttributeValidatorWithMapping, takes external configuration file like SchemaMapper, allows logical testing like Tester (including OR, NOT), can group tests together (eg "Attribute A is a number, or Attribute A is blank"; or... "Attribute A is ABC or Attribute B is DEF"), and custom error messages can be entered (each failed test is concatenated onto an error string). It works with un-exposed attributes (designed to work with an upstream SchemaMapper). May take a while to run as it checks each feature against each validation rule.

This should be able to handle a group of conditions with OR logic to get you the validation you require. It doesnt have a pass or fail output port instead just one port but any errors are written to an attribute (as features may pass some validation groups but fail others), so you may need a Tester to check if that error attribute has a value and handle any errors accordingly.

Hi Itwigg, thx for the heads up. Nice work! :)

At the moment I'm just one FME version behind the build you are using (2022.1.0.0 - b22618 versus 2022.1.1.0 - b22623), so unfortunately I can't check it out yet.

Personally I hope some of the additional functionality may also become part of the AttributeValidator. At the moment it just feels like the transformer is a bit behind on similar transformers like e.g. the TestFilter and Tester.

As a note, for the usecase of checking if any single attribute of a feature has a value, I ended up making a custom transformer that uses TCL. I used the AttributeConcatenator from Takashi as a basis, and basically stripped out most of the functionality, to check each value in the list for a value, and break looping if it found one.

Kind regards,

Thijs