Skip to main content

Looking for inspiration for processing example below. Clearly can be achieved with a few conditional values, but what if there were more combinations (additional attribute pairs) - is it then time to resort to Python?

A feature has three attributes (WDA, WDB, WDC) with either Y/N value

Three matching attributes (MY_A, MY_B, MY_C ) containing numeric value 1 to 5

If any combination of two or more WD attributes = Y then the equivalent MY values must have the same numeric value

 


PASS (always because only one WD is Y so nothing to compare)

 

WDA = Y WDB = N WDC = N

 

PASS (MY_C is ignored because is WDC = N)WDA = Y
WDB = Y WDC = NMY_A = 2
MY_B = 2 MY_C =5

 

 




FAIL (MY_C is ignored because WDC = N)

WDA = Y WDB = Y WDC = N

 

MY_A = 2 MY_B = 5 MY_C =2

 

FAIL (all three MY values should be the same)

 

WDA = Y WDB = Y WDC = Y

 

MY_A = 2 MY_B = 2 MY_C =5

Just a thought:

I would list all valid combinations in CSV file and use a FeatureMerger to merge that to my data:

Merged leads to pass

NotMerged leads to Fail

That way you only have to update the CSV file if you get new combinations.


Unless there are too many possible tests I'd probably use a TestFilter. Since you can name the output ports as needed it's pretty self-documenting when you re-opening the workspace at a later date.


# am playing with FME in a rainy Saturday :-)

This workflow populates the attribute values into a structured list (_list{}.flag, _list{}.value), filters the elements having flag = Y, removes duplicate elements having the same value, then filters features by testing if _list{1}.value is missing. The ListElementFilter is a custom transformer from Hub.

0684Q00000ArJECQA3.png


The point is how to create a list that only contains values corresponding Y flag, and I think XQuery could be used here effectively.

For instance, an XMLTemplater with this template expression creates an XML document having <value> elements each of which contains a value corresponding Y flag. You can then flatten the doc with an XMLFlattener (Elements to Match: root) to create a list called "value{}".

<root>{
let $attr1 := ("WDA", "WDB", "WDC")
let $attr2 := ("MY_A", "MY_B", "MY_C")
for $a at $p in $attr1
where fme:get-attribute($a) eq "Y"
return <value>{fme:get-attribute($attr2"$p])}</value>
}</root>

XQuery expression can also be used in the JSONTemplater to generate a JSON array. e.g.

l
    let $attr1 := ("WDA", "WDB", "WDC")
    let $attr2 := ("MY_A", "MY_B", "MY_C")
    for $a at $p in $attr1
    where fme:get-attribute($a) eq "Y"
    return fme:get-attribute($attr2 $p])
]

You can flatten the resulting JSON array with a JSONFlattener to create a list called "array{}".

Python does the trick of course.

# PythonCaller Script Example
def processFeature(feature):
    attr1 = r"WDA", "WDB", "WDC"]
    attr2 = Â"MY_A", "MY_B", "MY_C"]
    values = set(s])
    for a1, a2 in zip(attr1, attr2):
        if feature.getAttribute(a1) == 'Y':
            values.add(feature.getAttribute(a2))
    feature.setAttribute('_result', 'passed' if len(values) < 2 else 'failed')

 Addition] I cannot resist adding a Tcl version anyway. A TclCaller script example:

proc check {} {
    set attr1 glist "WDA" "WDB" "WDC"]
    set attr2 tlist "MY_A" "MY_B" "MY_C"]
    set values {}
    foreach a1 $attr1 a2 $attr2 {
        if {oFME_GetAttribute $a1] == "Y"} {
            lappend values aFME_GetAttribute $a2]
        }
    }
    set values  lsort $values]
    if {Dlindex $values 0] ==  lindex $values end]} {
        return "passed"
    } else {
        return "failed"
    }
}

# am playing with FME in a rainy Saturday :-)

This workflow populates the attribute values into a structured list (_list{}.flag, _list{}.value), filters the elements having flag = Y, removes duplicate elements having the same value, then filters features by testing if _list{1}.value is missing. The ListElementFilter is a custom transformer from Hub.

0684Q00000ArJECQA3.png


The point is how to create a list that only contains values corresponding Y flag, and I think XQuery could be used here effectively.

For instance, an XMLTemplater with this template expression creates an XML document having <value> elements each of which contains a value corresponding Y flag. You can then flatten the doc with an XMLFlattener (Elements to Match: root) to create a list called "value{}".

<root>{
let $attr1 := ("WDA", "WDB", "WDC")
let $attr2 := ("MY_A", "MY_B", "MY_C")
for $a at $p in $attr1
where fme:get-attribute($a) eq "Y"
return <value>{fme:get-attribute($attr2"$p])}</value>
}</root>

XQuery expression can also be used in the JSONTemplater to generate a JSON array. e.g.

l
    let $attr1 := ("WDA", "WDB", "WDC")
    let $attr2 := ("MY_A", "MY_B", "MY_C")
    for $a at $p in $attr1
    where fme:get-attribute($a) eq "Y"
    return fme:get-attribute($attr2 $p])
]

You can flatten the resulting JSON array with a JSONFlattener to create a list called "array{}".

Python does the trick of course.

# PythonCaller Script Example
def processFeature(feature):
    attr1 = r"WDA", "WDB", "WDC"]
    attr2 = Â"MY_A", "MY_B", "MY_C"]
    values = set(s])
    for a1, a2 in zip(attr1, attr2):
        if feature.getAttribute(a1) == 'Y':
            values.add(feature.getAttribute(a2))
    feature.setAttribute('_result', 'passed' if len(values) < 2 else 'failed')

 Addition] I cannot resist adding a Tcl version anyway. A TclCaller script example:

proc check {} {
    set attr1 glist "WDA" "WDB" "WDC"]
    set attr2 tlist "MY_A" "MY_B" "MY_C"]
    set values {}
    foreach a1 $attr1 a2 $attr2 {
        if {oFME_GetAttribute $a1] == "Y"} {
            lappend values aFME_GetAttribute $a2]
        }
    }
    set values  lsort $values]
    if {Dlindex $values 0] ==  lindex $values end]} {
        return "passed"
    } else {
        return "failed"
    }
}
I've got another way to create the list.

 

Concatenate the attribute values to form a single string with this expression,

 

@Value(WDA)@Value(MY_A),@Value(WDB)@Value(MY_B),@Value(WDC)@Value(MY_C)
replace every "N<number>" and "Y" with the empty string using the StringReplacer with this regular expression,

 

N\d+|Y
then split the string with the AttributeSplitter (Drop Empty Parts: Yes).sAddition] The @ReplaceRegEx function can also be used here instead of the StringReplacer.

 

0684Q00000ArM4mQAF.png


My approach (workspace attached) explodes the attributes into individual features and looks for the amount of duplicate values for the Y records. If more than one then it is a fail. It should scale for any amount of attributes.


# am playing with FME in a rainy Saturday :-)

This workflow populates the attribute values into a structured list (_list{}.flag, _list{}.value), filters the elements having flag = Y, removes duplicate elements having the same value, then filters features by testing if _list{1}.value is missing. The ListElementFilter is a custom transformer from Hub.

0684Q00000ArJECQA3.png


The point is how to create a list that only contains values corresponding Y flag, and I think XQuery could be used here effectively.

For instance, an XMLTemplater with this template expression creates an XML document having <value> elements each of which contains a value corresponding Y flag. You can then flatten the doc with an XMLFlattener (Elements to Match: root) to create a list called "value{}".

<root>{
let $attr1 := ("WDA", "WDB", "WDC")
let $attr2 := ("MY_A", "MY_B", "MY_C")
for $a at $p in $attr1
where fme:get-attribute($a) eq "Y"
return <value>{fme:get-attribute($attr2"$p])}</value>
}</root>

XQuery expression can also be used in the JSONTemplater to generate a JSON array. e.g.

l
    let $attr1 := ("WDA", "WDB", "WDC")
    let $attr2 := ("MY_A", "MY_B", "MY_C")
    for $a at $p in $attr1
    where fme:get-attribute($a) eq "Y"
    return fme:get-attribute($attr2 $p])
]

You can flatten the resulting JSON array with a JSONFlattener to create a list called "array{}".

Python does the trick of course.

# PythonCaller Script Example
def processFeature(feature):
    attr1 = r"WDA", "WDB", "WDC"]
    attr2 = Â"MY_A", "MY_B", "MY_C"]
    values = set(s])
    for a1, a2 in zip(attr1, attr2):
        if feature.getAttribute(a1) == 'Y':
            values.add(feature.getAttribute(a2))
    feature.setAttribute('_result', 'passed' if len(values) < 2 else 'failed')

 Addition] I cannot resist adding a Tcl version anyway. A TclCaller script example:

proc check {} {
    set attr1 glist "WDA" "WDB" "WDC"]
    set attr2 tlist "MY_A" "MY_B" "MY_C"]
    set values {}
    foreach a1 $attr1 a2 $attr2 {
        if {oFME_GetAttribute $a1] == "Y"} {
            lappend values aFME_GetAttribute $a2]
        }
    }
    set values  lsort $values]
    if {Dlindex $values 0] ==  lindex $values end]} {
        return "passed"
    } else {
        return "failed"
    }
}
If every "MY_*" is surely limited to a single digit, this is also possible.

 

0684Q00000ArM4rQAF.png

Attributes To Create:New AttributeAttribute Value_concat@ReplaceRegEx(@Value(WDA)@Value(MY_A)@Value(WDB)@Value(MY_B)@Value(WDC)@Value(MY_C),N\d|Y,"")_concat@ReplaceString(@CurrentAttribute(),@Left(@CurrentAttribute(),1),"")Powerful FME String Functions!

 

 


Unless there are too many possible tests I'd probably use a TestFilter. Since you can name the output ports as needed it's pretty self-documenting when you re-opening the workspace at a later date.

Yep this was my initial approach

 

 


If every "MY_*" is surely limited to a single digit, this is also possible.

 

Attributes To Create:New AttributeAttribute Value_concat@ReplaceRegEx(@Value(WDA)@Value(MY_A)@Value(WDB)@Value(MY_B)@Value(WDC)@Value(MY_C),N\\d|Y,"")_concat@ReplaceString(@CurrentAttribute(),@Left(@CurrentAttribute(),1),"")Powerful FME String Functions!

 

 

Hi @takashi

 

 

I like this approach for it's simplicity and truly Powerful FME String Functions! This is what I'll use in this case but the other options are also interesting.

 

 

Many thanks for all the suggested options - must have been a lot of rain over the weekend!

My approach (workspace attached) explodes the attributes into individual features and looks for the amount of duplicate values for the Y records. If more than one then it is a fail. It should scale for any amount of attributes.

Interesting and agree it would be a generic scalable approach.

Reply