Question

Finding gaps in a list

T
Badge

Dear FME community,

I have a aggregated line with a list that looks like this:

 

_agg_{0}.begin = 0

 

_agg_{1}.begin = 100

 

_agg_{0}.end = 100

 

_agg_{1}.end = 200

 

 

The above one is the ideal, correct one. Unfortunately, not all data looks like this. Here two examples.

 

1) I want to find gaps with no data, for example:

_agg_{0}.begin = 0

 

_agg_{1}.begin = 102

 

_agg_{0}.end = 98

 

_agg_{1}.end = 200

 

 

2) If possible also overlapping data, for example:

_agg_{0}.begin = 0

 

_agg_{1}.begin = 100

 

_agg_{0}.end = 102

 

_agg_{1}.end = 200

 

 

The values from begin(+1) should be exactly the same value as end(0) and so on.

Any ideas how to solve this?

 

Many thanks and kind regards

Thomas

10 replies

erik_jan
Rank
Userlevel 2
Badge +12

For this example you could use a Tester transformer with clause:

@Value(_agg_{0}.end ) = @Value(_agg_{1}.begin)

All examples that do not match this will come out of the Failed port of the Tester.

T
Badge

For this example you could use a Tester transformer with clause:

@Value(_agg_{0}.end ) = @Value(_agg_{1}.begin)

All examples that do not match this will come out of the Failed port of the Tester.

Thank you, but this will only test for the first and second entry of the list. Do you know how I can test for the hole list, i.e. with n number of elements?

ebygomm
Rank
Userlevel 1
Badge +10

You could explode the list and then use adjacent attribute handling to flag where the end value of one feature doesn't match the beginning value of the adjacent attribute.

If you don't want to explode the list, you're probably looking at a python solution

erik_jan
Rank
Userlevel 2
Badge +12

For this example you could use a Tester transformer with clause:

@Value(_agg_{0}.end ) = @Value(_agg_{1}.begin)

All examples that do not match this will come out of the Failed port of the Tester.

I created a workspace (FME 2019) that loops through the list and executes the test.

It is attached here: Begin_End_Tester.fmw

jdh
Rank
Badge +22

You could explode the list and then use adjacent attribute handling to flag where the end value of one feature doesn't match the beginning value of the adjacent attribute.

If you don't want to explode the list, you're probably looking at a python solution

That would be my non pythonic solution too. With a caveat that I would probably sort the list by _agg{}.begin prior to exploding.

erik_jan
Rank
Userlevel 2
Badge +12

You could explode the list and then use adjacent attribute handling to flag where the end value of one feature doesn't match the beginning value of the adjacent attribute.

If you don't want to explode the list, you're probably looking at a python solution

Have a look at my attached workspace.

No need for Python or exploding the list.

I used a Loop to find the gaps.

jdh
Rank
Badge +22

Have a look at my attached workspace.

No need for Python or exploding the list.

I used a Loop to find the gaps.

I find that looping through a list tends to be slower than exploding it, as long as you don't need to recombine the features.

 

 

Perhaps more relevant, the loop exits at the first failure, whereas the above method should identify all failures. I can't say which is better behaviour for the needs of the OP, if a pass/failure is all that's required, the efficiency of exiting the test sooner will make up for the looping overhead.
ebygomm
Rank
Userlevel 1
Badge +10

You could explode the list and then use adjacent attribute handling to flag where the end value of one feature doesn't match the beginning value of the adjacent attribute.

If you don't want to explode the list, you're probably looking at a python solution

I've assumed the solution would need to flag which list elements have gaps/overlaps between them rather than just flagging if the list has gaps

 

I'd probably stick with python to avoid exploding the list, something like

import fme
import fmeobjects

def processFeature(feature):
    agg = list(zip(feature.getAttribute('agg_{}.begin'),feature.getAttribute('agg_{}.end')))
    gaplist = []
    for i, (a, b) in enumerate(agg[:-1]):
        if b != agg[i+1][0]:
            gaplist.append(str(i)+'-'+str(i+1))
    feature.setAttribute('issues',','.join(gaplist))
takashi
Rank
Userlevel 2
Badge +17

Hi @thomask, excellent answers has been provided already but I would add another approach.

In most cases, a Loop can be replaced with feature cloning. See this article: The Cloner-Loop Technique: FME and a Soccer Fan’s Travels

This workspace will store the gap between "_agge_{i}.begin" and "_agge__{i-1}.end" into "_agge_{i}.gap" (0 < i, "_agge_{0}.gap" will always be 0).

finding-gaps-in-a-list.fmwt (FME 2019.1)

T
Badge

Many thanks for all your help! I will look up the answers and let you know as soon as I have tested.

Reply


Community Stats

28,654
Posts
109,286
Replies
36,528
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.