Skip to main content
Question

Conditional Check Across Multiple Attributes

M
Forum|alt.badge.img+2

Looking for inspiration for processing example below. Clearly can be achieved with a few conditional values, but what if there were more combinations (additional attribute pairs) - is it then time to resort to Python?

A feature has three attributes (WDA, WDB, WDC) with either Y/N value

Three matching attributes (MY_A, MY_B, MY_C ) containing numeric value 1 to 5

If any combination of two or more WD attributes = Y then the equivalent MY values must have the same numeric value

 

PASS (always because only one WD is Y so nothing to compare)

 

WDA = Y WDB = N WDC = N

 

PASS (MY_C is ignored because is WDC = N)WDA = Y WDB = Y WDC = NMY_A = 2 MY_B = 2 MY_C =5

 

 

FAIL (MY_C is ignored because WDC = N)

WDA = Y WDB = Y WDC = N

 

MY_A = 2 MY_B = 5 MY_C =2

 

FAIL (all three MY values should be the same)

 

WDA = Y WDB = Y WDC = Y

 

MY_A = 2 MY_B = 2 MY_C =5

9 replies

erik_jan
Contributor
Forum|alt.badge.img+17
  • erik_janContributor
  • Contributor
  • June 30, 2017

Just a thought:

I would list all valid combinations in CSV file and use a FeatureMerger to merge that to my data:

Merged leads to pass

NotMerged leads to Fail

That way you only have to update the CSV file if you get new combinations.

david_r
Evangelist
  • david_rEvangelist
  • June 30, 2017

Unless there are too many possible tests I'd probably use a TestFilter. Since you can name the output ports as needed it's pretty self-documenting when you re-opening the workspace at a later date.

takashi
Influencer
  • takashiInfluencer
  • July 1, 2017

# am playing with FME in a rainy Saturday :-)

This workflow populates the attribute values into a structured list (_list{}.flag, _list{}.value), filters the elements having flag = Y, removes duplicate elements having the same value, then filters features by testing if _list{1}.value is missing. The ListElementFilter is a custom transformer from Hub.

0684Q00000ArJECQA3.png


The point is how to create a list that only contains values corresponding Y flag, and I think XQuery could be used here effectively.

For instance, an XMLTemplater with this template expression creates an XML document having <value> elements each of which contains a value corresponding Y flag. You can then flatten the doc with an XMLFlattener (Elements to Match: root) to create a list called "value{}".

<root>{
let $attr1 := ("WDA", "WDB", "WDC")
let $attr2 := ("MY_A", "MY_B", "MY_C")
for $a at $p in $attr1
where fme:get-attribute($a) eq "Y"
return <value>{fme:get-attribute($attr2[$p])}</value>
}</root>

XQuery expression can also be used in the JSONTemplater to generate a JSON array. e.g.

[
    let $attr1 := ("WDA", "WDB", "WDC")
    let $attr2 := ("MY_A", "MY_B", "MY_C")
    for $a at $p in $attr1
    where fme:get-attribute($a) eq "Y"
    return fme:get-attribute($attr2[$p])
]

You can flatten the resulting JSON array with a JSONFlattener to create a list called "array{}".

Python does the trick of course.

# PythonCaller Script Example
def processFeature(feature):
    attr1 = ["WDA", "WDB", "WDC"]
    attr2 = ["MY_A", "MY_B", "MY_C"]
    values = set([])
    for a1, a2 in zip(attr1, attr2):
        if feature.getAttribute(a1) == 'Y':
            values.add(feature.getAttribute(a2))
    feature.setAttribute('_result', 'passed' if len(values) < 2 else 'failed')

[Addition] I cannot resist adding a Tcl version anyway. A TclCaller script example:

proc check {} {
    set attr1 [list "WDA" "WDB" "WDC"]
    set attr2 [list "MY_A" "MY_B" "MY_C"]
    set values {}
    foreach a1 $attr1 a2 $attr2 {
        if {[FME_GetAttribute $a1] == "Y"} {
            lappend values [FME_GetAttribute $a2]
        }
    }
    set values [lsort $values]
    if {[lindex $values 0] == [lindex $values end]} {
        return "passed"
    } else {
        return "failed"
    }
}
takashi
Influencer
  • takashiInfluencer
  • July 1, 2017
takashi wrote:

# am playing with FME in a rainy Saturday :-)

This workflow populates the attribute values into a structured list (_list{}.flag, _list{}.value), filters the elements having flag = Y, removes duplicate elements having the same value, then filters features by testing if _list{1}.value is missing. The ListElementFilter is a custom transformer from Hub.

0684Q00000ArJECQA3.png


The point is how to create a list that only contains values corresponding Y flag, and I think XQuery could be used here effectively.

For instance, an XMLTemplater with this template expression creates an XML document having <value> elements each of which contains a value corresponding Y flag. You can then flatten the doc with an XMLFlattener (Elements to Match: root) to create a list called "value{}".

<root>{
let $attr1 := ("WDA", "WDB", "WDC")
let $attr2 := ("MY_A", "MY_B", "MY_C")
for $a at $p in $attr1
where fme:get-attribute($a) eq "Y"
return <value>{fme:get-attribute($attr2[$p])}</value>
}</root>

XQuery expression can also be used in the JSONTemplater to generate a JSON array. e.g.

[
    let $attr1 := ("WDA", "WDB", "WDC")
    let $attr2 := ("MY_A", "MY_B", "MY_C")
    for $a at $p in $attr1
    where fme:get-attribute($a) eq "Y"
    return fme:get-attribute($attr2[$p])
]

You can flatten the resulting JSON array with a JSONFlattener to create a list called "array{}".

Python does the trick of course.

# PythonCaller Script Example
def processFeature(feature):
    attr1 = ["WDA", "WDB", "WDC"]
    attr2 = ["MY_A", "MY_B", "MY_C"]
    values = set([])
    for a1, a2 in zip(attr1, attr2):
        if feature.getAttribute(a1) == 'Y':
            values.add(feature.getAttribute(a2))
    feature.setAttribute('_result', 'passed' if len(values) < 2 else 'failed')

[Addition] I cannot resist adding a Tcl version anyway. A TclCaller script example:

proc check {} {
    set attr1 [list "WDA" "WDB" "WDC"]
    set attr2 [list "MY_A" "MY_B" "MY_C"]
    set values {}
    foreach a1 $attr1 a2 $attr2 {
        if {[FME_GetAttribute $a1] == "Y"} {
            lappend values [FME_GetAttribute $a2]
        }
    }
    set values [lsort $values]
    if {[lindex $values 0] == [lindex $values end]} {
        return "passed"
    } else {
        return "failed"
    }
}
I've got another way to create the list.

 

Concatenate the attribute values to form a single string with this expression,

 

@Value(WDA)@Value(MY_A),@Value(WDB)@Value(MY_B),@Value(WDC)@Value(MY_C)
replace every "N<number>" and "Y" with the empty string using the StringReplacer with this regular expression,

 

N\d+|Y
then split the string with the AttributeSplitter (Drop Empty Parts: Yes).[Addition] The @ReplaceRegEx function can also be used here instead of the StringReplacer.

 

0684Q00000ArM4mQAF.png

I
Forum|alt.badge.img
  • imapping
  • July 1, 2017

My approach (workspace attached) explodes the attributes into individual features and looks for the amount of duplicate values for the Y records. If more than one then it is a fail. It should scale for any amount of attributes.

takashi
Influencer
  • takashiInfluencer
  • July 2, 2017
takashi wrote:

# am playing with FME in a rainy Saturday :-)

This workflow populates the attribute values into a structured list (_list{}.flag, _list{}.value), filters the elements having flag = Y, removes duplicate elements having the same value, then filters features by testing if _list{1}.value is missing. The ListElementFilter is a custom transformer from Hub.

0684Q00000ArJECQA3.png


The point is how to create a list that only contains values corresponding Y flag, and I think XQuery could be used here effectively.

For instance, an XMLTemplater with this template expression creates an XML document having <value> elements each of which contains a value corresponding Y flag. You can then flatten the doc with an XMLFlattener (Elements to Match: root) to create a list called "value{}".

<root>{
let $attr1 := ("WDA", "WDB", "WDC")
let $attr2 := ("MY_A", "MY_B", "MY_C")
for $a at $p in $attr1
where fme:get-attribute($a) eq "Y"
return <value>{fme:get-attribute($attr2[$p])}</value>
}</root>

XQuery expression can also be used in the JSONTemplater to generate a JSON array. e.g.

[
    let $attr1 := ("WDA", "WDB", "WDC")
    let $attr2 := ("MY_A", "MY_B", "MY_C")
    for $a at $p in $attr1
    where fme:get-attribute($a) eq "Y"
    return fme:get-attribute($attr2[$p])
]

You can flatten the resulting JSON array with a JSONFlattener to create a list called "array{}".

Python does the trick of course.

# PythonCaller Script Example
def processFeature(feature):
    attr1 = ["WDA", "WDB", "WDC"]
    attr2 = ["MY_A", "MY_B", "MY_C"]
    values = set([])
    for a1, a2 in zip(attr1, attr2):
        if feature.getAttribute(a1) == 'Y':
            values.add(feature.getAttribute(a2))
    feature.setAttribute('_result', 'passed' if len(values) < 2 else 'failed')

[Addition] I cannot resist adding a Tcl version anyway. A TclCaller script example:

proc check {} {
    set attr1 [list "WDA" "WDB" "WDC"]
    set attr2 [list "MY_A" "MY_B" "MY_C"]
    set values {}
    foreach a1 $attr1 a2 $attr2 {
        if {[FME_GetAttribute $a1] == "Y"} {
            lappend values [FME_GetAttribute $a2]
        }
    }
    set values [lsort $values]
    if {[lindex $values 0] == [lindex $values end]} {
        return "passed"
    } else {
        return "failed"
    }
}
If every "MY_*" is surely limited to a single digit, this is also possible.

 

0684Q00000ArM4rQAF.png

Attributes To Create:New AttributeAttribute Value_concat@ReplaceRegEx(@Value(WDA)@Value(MY_A)@Value(WDB)@Value(MY_B)@Value(WDC)@Value(MY_C),N\d|Y,"")_concat@ReplaceString(@CurrentAttribute(),@Left(@CurrentAttribute(),1),"")Powerful FME String Functions!

 

 

M
Forum|alt.badge.img+2
  • mark_f
  • Author
  • July 3, 2017
david_r wrote:

Unless there are too many possible tests I'd probably use a TestFilter. Since you can name the output ports as needed it's pretty self-documenting when you re-opening the workspace at a later date.

Yep this was my initial approach

 

 

M
Forum|alt.badge.img+2
  • mark_f
  • Author
  • July 3, 2017
takashi wrote:
If every "MY_*" is surely limited to a single digit, this is also possible.

 

Attributes To Create:New AttributeAttribute Value_concat@ReplaceRegEx(@Value(WDA)@Value(MY_A)@Value(WDB)@Value(MY_B)@Value(WDC)@Value(MY_C),N\\d|Y,"")_concat@ReplaceString(@CurrentAttribute(),@Left(@CurrentAttribute(),1),"")Powerful FME String Functions!

 

 

Hi @takashi

 

 

I like this approach for it's simplicity and truly Powerful FME String Functions! This is what I'll use in this case but the other options are also interesting.

 

 

Many thanks for all the suggested options - must have been a lot of rain over the weekend!
M
Forum|alt.badge.img+2
  • mark_f
  • Author
  • July 3, 2017
imapping wrote:

My approach (workspace attached) explodes the attributes into individual features and looks for the amount of duplicate values for the Y records. If more than one then it is a fail. It should scale for any amount of attributes.

Interesting and agree it would be a generic scalable approach.

Reply


Helpful Members This Week

Community Stats

31,105
Posts
117,929
Replies
38,858
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.