Question

network accumulation

  • 21 September 2018
  • 10 replies
  • 32 views

Badge

I have a working pipe network where I know how many service connections are on each pipe saved as the attribute [_count] on each pipe.

I'd like to count the number of connections upstream of each pipe segment and save it as an attribute on that pipe. Our goal is to prioritize emergency repairs based on the # of properties a particular pipe segment serves.

Most of the examples and transformers are based on the shortest value (time, travel, length, etc). I kind of want the opposite. StreamOrderCalculator could work if I were allowed to 1) use an attribute instead of Stream Order Type, and 2) accumulate that attribute at nodes instead of checking which one is larger or if they are equal. Same with StreamPriorityCalculator. It's looking for shortest path, not longest.

Any suggestions are appreciated.


10 replies

Userlevel 2
Badge +17
Does "the number of connections upstream" of "pipe segment X" mean the number of pipe segments which touch the start node of the "pipe segment X" at their end nodes?

 

 

Userlevel 4
Badge +25

Really the task is to extract that part of the network upstream of the original segment, after which counting is fairly easy. I think you can do this by NetworkTopologyCounter and leaving out the pipe segment that is being worked on. Then you get a set of networks, one of which will be the part upstream of the original segment.

How to find which network? Well I'd take the start point (or end point depending on the direction) of the original pipe segment, do an overlay, and find out what network ID the line has. A simple test or merge removes all other networks and then a Counter/StatsCounter tells you how many features there are.

So here I have a network of streams. The one marked with a red line is my "pipe segment"...

And here's my workspace:

With this I can isolate the network upstream of the original segment and count that there are 8 line segments.

I'm not sure what counts as a "connection" for you. If it's the number of pipe segments, then this is your answer. If it's a connection at the end of each pipe, then the number of connections is the number of lines plus one. To be absolutely certain you could run it through the TopologyBuilder and count the number of nodes. Or count the number of vertices in a pipe using the VertexCounter.

Anyway, however you want to count, I think the important part is isolating that part of the network, and I think this method will do that fairly well.

Here's my workspace if it's of help. I picked the original segment using a Counter/Tester, but obviously you'd select it using an ID of some sort.

Badge

No, it's the number of houses connected to that pipe that I want to keep adding as an attribute.

If I have 2 pipes; one that supplies water to 10 houses and one that supplies water to 100 - all other things being equal - I need to fix the one that supplies 100 houses first.

Userlevel 4
Badge +25

No, it's the number of houses connected to that pipe that I want to keep adding as an attribute.

If I have 2 pipes; one that supplies water to 10 houses and one that supplies water to 100 - all other things being equal - I need to fix the one that supplies 100 houses first.

Right, so how do you know how many houses connect to a pipe? Is it an attribute on the pipe, or a line that connects pipe to house? So are all line endpoints houses? Or every vertex on the pipe? Or is there a connection point feature? If it's just a pipe with no attributes, no connections to houses, or no junction point features, then I guess all you can do is put a buffer around it and overlay against address points.

 

 

Userlevel 1
Badge +21

Not sure how well, or performant it would be on a real network but.

Assuming your network is something like this, with an attribute with number of houses of that supply pipe.

Extract every start point of network and create line with final end point. Use ShortestPathFinder to get every path from start to finish. LineOnLine Overlayer to find overlapping segments, accumulate your house count into a list, sum the list to find number of houses upstream

count-upstream.fmwt

Userlevel 2
Badge +17
I think the workflow @egomm suggested (just remove the AttributeCreator_2) would generate your desired result, if the lines form a dendritic network (i.e. not contain a closed circuit).

 

Alternatively, Python scripting in conjunction with some transformers could be effective. Assuming that all the lines orient to downstream and have a unique ID attribute called "pipe_id":count-upstream-python.fmwt  (FME 2018.1.0.2)

 

0684Q00000ArL6KQAV.png

 

PythonCaller Script:

 

import fmeimport fmeobjectsclass UpstreamHousesAccumulator(object):    def __init__(self):        self.idToFeature = {}            def input(self, feature):        id = str(feature.getAttribute('pipe_id'))        feature.removeAttribute('total')        self.idToFeature[id] = feature            def close(self):        for feature in self.idToFeature.values():            self.setTotal(feature)            self.pyoutput(feature)                def setTotal(self, feature):        _, isMissing, _ = feature.getAttributeNullMissingAndType('total')        if isMissing:            n = int(feature.getAttribute('houses'))            upstreamIds = feature.getAttribute('_relationships{}.pipe_id')            if upstreamIds:                for id in upstreamIds:                    upstream = self.idToFeature[id]                    self.setTotal(upstream) # recursive call                    n += upstream.getAttribute('total')            feature.setAttribute('total', n)
Userlevel 2
Badge +17

Not sure how well, or performant it would be on a real network but.

Assuming your network is something like this, with an attribute with number of houses of that supply pipe.

0684Q00000ArKx0QAF.jpg

Extract every start point of network and create line with final end point. Use ShortestPathFinder to get every path from start to finish. LineOnLine Overlayer to find overlapping segments, accumulate your house count into a list, sum the list to find number of houses upstream

count-upstream.fmwt

I think the workflow @egomm suggested (just remove the AttributeCreator_2) would generate your desired result, if the lines form a dendritic network (i.e. not contain a closed circuit).

 

Alternatively, Python scripting in conjunction with some transformers could be effective. Assuming that all the lines orient to downstream and have a unique ID attribute called "pipe_id":count-upstream-python.fmwt  (FME 2018.1.0.2)

 

0684Q00000ArMfKQAV.png

 

PythonCaller Script:

 

import fmeimport fmeobjectsclass UpstreamHousesAccumulator(object):    def __init__(self):        self.idToFeature = {}            def input(self, feature):        id = str(feature.getAttribute('pipe_id'))        feature.removeAttribute('total')        self.idToFeature[id] = feature            def close(self):        for feature in self.idToFeature.values():            self.setTotal(feature)            self.pyoutput(feature)                def setTotal(self, feature):        _, isMissing, _ = feature.getAttributeNullMissingAndType('total')        if isMissing:            n = int(feature.getAttribute('houses'))            upstreamIds = feature.getAttribute('_relationships{}.pipe_id')            if upstreamIds:                for id in upstreamIds:                    upstream = self.idToFeature[id]                    self.setTotal(upstream) # recursive call                    n += upstream.getAttribute('total')            feature.setAttribute('total', n)
Userlevel 2
Badge +17
I think the workflow @egomm suggested (just remove the AttributeCreator_2) would generate your desired result, if the lines form a dendritic network (i.e. not contain a closed circuit).

 

Alternatively, Python scripting in conjunction with some transformers could be effective. Assuming that all the lines orient to downstream and have a unique ID attribute called "pipe_id":count-upstream-python.fmwt  (FME 2018.1.0.2)

 

0684Q00000ArL6KQAV.png

 

PythonCaller Script:

 

import fmeimport fmeobjectsclass UpstreamHousesAccumulator(object):    def __init__(self):        self.idToFeature = {}            def input(self, feature):        id = str(feature.getAttribute('pipe_id'))        feature.removeAttribute('total')        self.idToFeature[id] = feature            def close(self):        for feature in self.idToFeature.values():            self.setTotal(feature)            self.pyoutput(feature)                def setTotal(self, feature):        _, isMissing, _ = feature.getAttributeNullMissingAndType('total')        if isMissing:            n = int(feature.getAttribute('houses'))            upstreamIds = feature.getAttribute('_relationships{}.pipe_id')            if upstreamIds:                for id in upstreamIds:                    upstream = self.idToFeature[id]                    self.setTotal(upstream) # recursive call                    n += upstream.getAttribute('total')            feature.setAttribute('total', n)
Another approach, using global variables: count-upstream-variables.fmwt (FME 2018.1.0.2)
Badge
I think the workflow @egomm suggested (just remove the AttributeCreator_2) would generate your desired result, if the lines form a dendritic network (i.e. not contain a closed circuit).

 

Alternatively, Python scripting in conjunction with some transformers could be effective. Assuming that all the lines orient to downstream and have a unique ID attribute called "pipe_id":count-upstream-python.fmwt  (FME 2018.1.0.2)

 

0684Q00000ArL6KQAV.png

 

PythonCaller Script:

 

import fmeimport fmeobjectsclass UpstreamHousesAccumulator(object):    def __init__(self):        self.idToFeature = {}            def input(self, feature):        id = str(feature.getAttribute('pipe_id'))        feature.removeAttribute('total')        self.idToFeature[id] = feature            def close(self):        for feature in self.idToFeature.values():            self.setTotal(feature)            self.pyoutput(feature)                def setTotal(self, feature):        _, isMissing, _ = feature.getAttributeNullMissingAndType('total')        if isMissing:            n = int(feature.getAttribute('houses'))            upstreamIds = feature.getAttribute('_relationships{}.pipe_id')            if upstreamIds:                for id in upstreamIds:                    upstream = self.idToFeature[id]                    self.setTotal(upstream) # recursive call                    n += upstream.getAttribute('total')            feature.setAttribute('total', n)

Takashi, how would look this python script if the house would have an address, and multiple addresses are associated as a list to each segment. is it possible that the output to be an accumulated list of addresses?

Userlevel 1
Badge +11

Takashi, how would look this python script if the house would have an address, and multiple addresses are associated as a list to each segment. is it possible that the output to be an accumulated list of addresses?

Hi @cipriancu,

This seems to be an older thread that you've posted to. Would you be open to posting a new question with your scenario for better visibility? You can always link back to this one in your new post. There's been a lot of development in FME since the 2018 version so there might be some new ways to tackle this potentially without using python even. Thanks!

Reply