Question

Matching Values Between Attributes

util_ryan

Hi All,

Not to be hyperspecific but i'm trying to match phase values between two attributes on the same feature.

 

I have an electric transformer with a phase value, and I've joined the phase value of it's connected electric cable to the same feature. The phase values can be any combination of A,B, or C. For example, the electric transformer might have a value of AC, and the cable it's joined to has a value of ABC.

 

I'm trying to find a way to test if the phase values of the transformer are contained within the phase values of the cable. I've tried setting up a tester and matching XFMRPhase - IN - PrimaryPhase, the inverse of that. I've tried removing the commas and testing the same. I've tried testing XFMRPhase - LIKE - PrimaryPhase and I keep finding results that don't match. For example, i'll find a transformer with phasing ABC, primary cable with phasing ABC as unmatched. I'm not sure what the best way to tackle this would be, but a nudge in the right direction would be appreciated.

 

Here is an example of the data i'm looking at:

 

13 replies

erik_jan
Rank
Userlevel 2
Badge +12

Since both attributes are on the same feature (from your screenshot), I would use a Tester transformer to test:

Hope this helps.

util_ryan

Thanks Erik. I was hoping there was a way to iterate over all of the values from one attribute and check each of them against the values in the other attribute. Ideally I would be able to use a tester, but with a much simpler set up. Something like:

(This returns from what I can tell just random/bad data.)

While there is a maximum amount of possibilities that I could test for (only 3x possible phases on the XFMR, another 3x on the primary), i'd like to be able to find a way to do this without having to test out each possibility in the tester individually.

erik_jan
Rank
Userlevel 2
Badge +12

Thanks Erik. I was hoping there was a way to iterate over all of the values from one attribute and check each of them against the values in the other attribute. Ideally I would be able to use a tester, but with a much simpler set up. Something like:

(This returns from what I can tell just random/bad data.)

While there is a maximum amount of possibilities that I could test for (only 3x possible phases on the XFMR, another 3x on the primary), i'd like to be able to find a way to do this without having to test out each possibility in the tester individually.

The issue with this approach is that A,C is not in A,B,C, all other matches could be caught by PrimaryPhase CONTAINS XFMRPhase.

That would definitely shorten the number of options to 2:

PrimaryPhase CONTAINS XFMRPhase OR

PrimaryPhase = A,B,C AND XFMRPhase = A,C.

ebygomm
Rank
Userlevel 1
Badge +10

I have a very horrible approach that I used when trying to do something similar for advent of code and avoiding python, I'd never do it in day to day work! It basically reformulates the values as a regular expression then searches for that regular expression

0684Q00000ArLZFQA3.png

Left Value

@FindRegEx(@Value(PrimaryPhase),@ReplaceString(@ReplaceRegEx(@Value(XFMRPhase),([A-Z]),(?=.*\1)),",",""))

So for example

If PrimaryPhase is A,B,C and XFMRPhase is B,C the XFMRPhase is converted to a regular expression

(?=.*B)(?=.*C)

It then looks for this regular expression in the PrimaryPhase attribute, if there is a match it returns 0 and passes the test, otherwise it returns -1 and fails the test

 

Edit: It's actually slightly simpler if you put use the same test in an AttributeValidator

0684Q00000ArLJwQAN.png

mark2atsafe
Rank
Userlevel 4
Badge +25

OK, one method is this:

  • Add an AttributeSplitter
    • Split the XFMRPhase attribute
    • Split on the , character
    • Put into a list (say called xList)
  • Add a Tester
    • Test for:
      • @FindRegEx(@Value(PrimaryPhase),(?=.*@Value(xList{0}))(?=.*@Value(xList{1}))(?=.*@Value(xList{2})))
      • !=
      • -1

That seems to work for me. It assumes you don't have more than 3 items in the XFMR list.

Demo workspace: phasechecker.fmw

mark2atsafe
Rank
Userlevel 4
Badge +25

OK, one method is this:

  • Add an AttributeSplitter
    • Split the XFMRPhase attribute
    • Split on the , character
    • Put into a list (say called xList)
  • Add a Tester
    • Test for:
      • @FindRegEx(@Value(PrimaryPhase),(?=.*@Value(xList{0}))(?=.*@Value(xList{1}))(?=.*@Value(xList{2})))
      • !=
      • -1

That seems to work for me. It assumes you don't have more than 3 items in the XFMR list.

Demo workspace: phasechecker.fmw

The Regex part says the PrimaryPhase has to contain the first XFMRPhase AND the second XFMRPhase AND the third XFMRPhase. If it does then it returns zero, else it returns -1. At least, it works in all the testing I've done. It doesn't seem to matter if there are not 3 values. ie if xList{2} doesn't exist, it doesn't seem to cause a mismatch.

mark2atsafe
Rank
Userlevel 4
Badge +25

OK, one method is this:

  • Add an AttributeSplitter
    • Split the XFMRPhase attribute
    • Split on the , character
    • Put into a list (say called xList)
  • Add a Tester
    • Test for:
      • @FindRegEx(@Value(PrimaryPhase),(?=.*@Value(xList{0}))(?=.*@Value(xList{1}))(?=.*@Value(xList{2})))
      • !=
      • -1

That seems to work for me. It assumes you don't have more than 3 items in the XFMR list.

Demo workspace: phasechecker.fmw

Since the AttributeSplitter is mostly regex and regsub, I think it likely you could encapsulate that part into the Tester string too; I haven't tried it, but I'm about 75% certain it could be done.

redgeographics
Rank
Userlevel 4
Badge +25

I have a very horrible approach that I used when trying to do something similar for advent of code and avoiding python, I'd never do it in day to day work! It basically reformulates the values as a regular expression then searches for that regular expression

0684Q00000ArLZFQA3.png

Left Value

@FindRegEx(@Value(PrimaryPhase),@ReplaceString(@ReplaceRegEx(@Value(XFMRPhase),([A-Z]),(?=.*\1)),",",""))

So for example

If PrimaryPhase is A,B,C and XFMRPhase is B,C the XFMRPhase is converted to a regular expression

(?=.*B)(?=.*C)

It then looks for this regular expression in the PrimaryPhase attribute, if there is a match it returns 0 and passes the test, otherwise it returns -1 and fails the test

 

Edit: It's actually slightly simpler if you put use the same test in an AttributeValidator

0684Q00000ArLJwQAN.png

Creating regexes on the fly must be the ultimate high level FME hack!

ebygomm
Rank
Userlevel 1
Badge +10

Creating regexes on the fly must be the ultimate high level FME hack!

In real life, I'd go to python for this, as it gives some more control on reporting what doesn't match. 

import fme
import fmeobjects

def processFeature(feature):
        primary = feature.getAttribute('PrimaryPhase').split(',')
        xfmr = feature.getAttribute('XFMRPhase').split(',')
        templist = [i for i in xfmr if i not in primary]
        if len(templist) == 0:
            feature.setAttribute("outcome",'pass')
        else:
            feature.setAttribute("outcome",'fail')
            feature.setAttribute("missing_phases", ','.join(templist))

But I'd given myself some no python rules for advent of code - sometimes fme is harder than python!

 

erik_jan
Rank
Userlevel 2
Badge +12

I think this is the easiest way, using the Tester:

The only clause not covered by the contains is when the XFMRPhase = A,C and the PrimaryPhase has all phases.

Hope this helps.

util_ryan

OK, one method is this:

  • Add an AttributeSplitter
    • Split the XFMRPhase attribute
    • Split on the , character
    • Put into a list (say called xList)
  • Add a Tester
    • Test for: 
      • @FindRegEx(@Value(PrimaryPhase),(?=.*@Value(xList{0}))(?=.*@Value(xList{1}))(?=.*@Value(xList{2})))
      • !=
      • -1

That seems to work for me. It assumes you don't have more than 3 items in the XFMR list.

Demo workspace: phasechecker.fmw

I feel like i'm missing something somewhere. (Opening up the demo bench doesn't work, we're running 2018.1.2.0 currently)

 

Here's how I set up the tester:

 

0684Q00000ArMmAQAV.png

Regex:

@FindRegEx(@Value(PRIMARYPHASE),(?=.*@Value(_PhaseList{0}))(?=.*@Value(_PhaseList{1}))(?=.*@Value(_PhaseList{2})))

 

The results i'm seeing in output to Passed seem to be correct, but then i'm just getting weird results that should be passing out of the Failed output:

 

0684Q00000ArMUtQAN.png

ebygomm
Rank
Userlevel 1
Badge +10

I feel like i'm missing something somewhere. (Opening up the demo bench doesn't work, we're running 2018.1.2.0 currently)

 

Here's how I set up the tester:

 

0684Q00000ArMmAQAV.png

Regex:

@FindRegEx(@Value(PRIMARYPHASE),(?=.*@Value(_PhaseList{0}))(?=.*@Value(_PhaseList{1}))(?=.*@Value(_PhaseList{2})))

 

The results i'm seeing in output to Passed seem to be correct, but then i'm just getting weird results that should be passing out of the Failed output:

 

0684Q00000ArMUtQAN.png

The tester should be set up like this

0684Q00000ArNBKQA3.png

util_ryan

The tester should be set up like this

That seemed to the do the trick. This looks like it's working, but now i'm noticing incorrect results because of the way i'm joining the phasing over from the XFMR to the Primary.

C'est la FME as they say.

 

I still am not extremely clear on using RegEx, so I appreciate the examples in here even though i'll admit that I don't quite understand them just yet.

Reply


Community Stats

28,650
Posts
109,262
Replies
36,523
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.