Skip to main content
  • Home
  • Forums
  • FME Form
  • Integration
  • I am new to fme python scripting and looking to create a python caller that will produce the cartesian product of the list of routes at an intersection (points feature data). The code below produces what I am looking foR. Thank you!
Solved

I am new to fme python scripting and looking to create a python caller that will produce the cartesian product of the list of routes at an intersection (points feature data). The code below produces what I am looking foR. Thank you!

fme
Contributor
Forum|alt.badge.img+7
  • fmeContributor
  • Contributor

I was able to implement this in the python caller which produces the result but when I substitute the hard coded data for "myList" it does not work for me.

import itertools

 

myList = [

  ['a1', 'b1', 'c1'],

  ['a1', 'b1', 'c1']

]

for x in itertools.product(*myList):

  results = list(itertools.product(myList))

  print (x)

 

How do I pass the output in my list out? How do I substitute myList with the data from my DB.

CODE in Python Caller which lists results in the translation log

import fme

import fmeobjects

import itertools

# Template Function interface:

# When using this function, make sure its name is set as the value of

# the 'Class or Function to Process Features' transformer parameter

def processFeature(feature):

  pass

 

# Template Class Interface:

# When using this class, make sure its name is set as the value of

# the 'Class or Function to Process Features' transformer parameter

class FeatureProcessor(object):

  def __init__(self):

    pass

  def input(self,feature):

    somelists = [['a1', 'b1', 'c1'],['a1', 'b1', 'c1']]

    for elements in itertools.product(*somelists):

      print (elements)

     

    self.pyoutput(feature)

     

  def close(self):

    pass

Best answer by ebygomm

So you have thousands of features, but presumably each intersection has a much smaller number of possible routes associated with it?

 

The first step I think would be to look at creating a feature for each intersection with a list of routes, e.g. with a list builder group by intersection and add the selected attribute identifying the route

 

You could then do something like this in a python caller

import fme
import fmeobjects
import itertools
 
class FeatureProcessor(object):
    def __init__(self):
        pass
    def input(self,feature):
        mylist=feature.getAttribute('_list{}.NLF_ID')
        for elements in set(itertools.permutations(mylist,2)):
           feature.setAttribute("NLF_ID", elements[0])
           feature.setAttribute("NLF_ID_1",elements[1])
           self.pyoutput(feature)
        
    def close(self):
        pass

 

View original
Did this help you find an answer to your question?

11 replies

ebygomm
Influencer
Forum|alt.badge.img+31
  • ebygommInfluencer
  • Influencer
  • November 24, 2021

Can you share how the list of routes is stored in your data?

fme
Contributor
Forum|alt.badge.img+7
  • fmeContributor
  • Author
  • Contributor
  • November 24, 2021

@ebygomm please find attached a screenshot of the sample data i am using.

 

1 Attachments
ebygomm
Influencer
Forum|alt.badge.img+31
  • ebygommInfluencer
  • Influencer
  • November 24, 2021
fme wrote:

@ebygomm please find attached a screenshot of the sample data i am using.

 

And are you hoping to get 9 features resulting from your python caller in this example where there are 3 routes at an intersection?

 

So including A1-A1, B1-B1, C1-C1 even though presumably these wouldn't be valid movements across an intersection?

 

If you can explain exactly what you are looking to achieve there are probably ways to get this without resorting to python

fme
Contributor
Forum|alt.badge.img+7
  • fmeContributor
  • Author
  • Contributor
  • November 24, 2021
ebygomm wrote:

And are you hoping to get 9 features resulting from your python caller in this example where there are 3 routes at an intersection?

 

So including A1-A1, B1-B1, C1-C1 even though presumably these wouldn't be valid movements across an intersection?

 

If you can explain exactly what you are looking to achieve there are probably ways to get this without resorting to python

Yes I am hoping to get 9 features including the duplicates a1-a1, b1-b1, c1-c1 which i plan to eliminate using the transformer in fme. You can factor taking the duplicates out though so having 6 features excluding a1-a1, b1-b1, c1-c1 also works for me. FYI The 6 features or 9 feature must have its intersect features within the attributes too differentiating it with a prefix.

ebygomm
Influencer
Forum|alt.badge.img+31
  • ebygommInfluencer
  • Influencer
  • November 24, 2021

So you can do this quite easily in an inline querier

 

image

fme
Contributor
Forum|alt.badge.img+7
  • fmeContributor
  • Author
  • Contributor
  • November 24, 2021
ebygomm wrote:

So you can do this quite easily in an inline querier

 

image

I was trying to stay away from doing this using SQL since i have thousands of features to look through and felt python will be faster to run.

fme
Contributor
Forum|alt.badge.img+7
  • fmeContributor
  • Author
  • Contributor
  • November 24, 2021
ebygomm wrote:

So you can do this quite easily in an inline querier

 

image

The excel sheet was just a sample to show what my data looks like. My datasource will be MS SQL Server

ebygomm
Influencer
Forum|alt.badge.img+31
  • ebygommInfluencer
  • Influencer
  • Best Answer
  • November 24, 2021

So you have thousands of features, but presumably each intersection has a much smaller number of possible routes associated with it?

 

The first step I think would be to look at creating a feature for each intersection with a list of routes, e.g. with a list builder group by intersection and add the selected attribute identifying the route

 

You could then do something like this in a python caller

import fme
import fmeobjects
import itertools
 
class FeatureProcessor(object):
    def __init__(self):
        pass
    def input(self,feature):
        mylist=feature.getAttribute('_list{}.NLF_ID')
        for elements in set(itertools.permutations(mylist,2)):
           feature.setAttribute("NLF_ID", elements[0])
           feature.setAttribute("NLF_ID_1",elements[1])
           self.pyoutput(feature)
        
    def close(self):
        pass

 

fme
Contributor
Forum|alt.badge.img+7
  • fmeContributor
  • Author
  • Contributor
  • November 24, 2021
ebygomm wrote:

So you have thousands of features, but presumably each intersection has a much smaller number of possible routes associated with it?

 

The first step I think would be to look at creating a feature for each intersection with a list of routes, e.g. with a list builder group by intersection and add the selected attribute identifying the route

 

You could then do something like this in a python caller

import fme
import fmeobjects
import itertools
 
class FeatureProcessor(object):
    def __init__(self):
        pass
    def input(self,feature):
        mylist=feature.getAttribute('_list{}.NLF_ID')
        for elements in set(itertools.permutations(mylist,2)):
           feature.setAttribute("NLF_ID", elements[0])
           feature.setAttribute("NLF_ID_1",elements[1])
           self.pyoutput(feature)
        
    def close(self):
        pass

 

Thank you for your time @ebygomm​ . Doing a group by using this field which is same id for all intersecting routes, I get a single feature instead of 3 features as seen in the sample data.  The only unique and PK field here is the Cross_Route_ID. 

NLF_ID	| Measure|	|RT_Name| 	|Cross_RouteID|	 | Inter_ID
a1	         8.666	              A-1	                 111111	            987654321
b1	         4.217	              B-1	               222222	            987654321
c1	           9.624                C-1	               333333	             987654321

 

ebygomm
Influencer
Forum|alt.badge.img+31
  • ebygommInfluencer
  • Influencer
  • November 24, 2021
fme wrote:

Thank you for your time @ebygomm​ . Doing a group by using this field which is same id for all intersecting routes, I get a single feature instead of 3 features as seen in the sample data.  The only unique and PK field here is the Cross_Route_ID. 

NLF_ID	| Measure|	|RT_Name| 	|Cross_RouteID|	 | Inter_ID
a1	         8.666	              A-1	                 111111	            987654321
b1	         4.217	              B-1	               222222	            987654321
c1	           9.624                C-1	               333333	             987654321

 

That's expected, it is a single feature which contains a list of the routes. The python script then uses this list to get the permutations, and outputs 6 features, with the two routes in separate attributes

fme
Contributor
Forum|alt.badge.img+7
  • fmeContributor
  • Author
  • Contributor
  • November 24, 2021
fme wrote:

Thank you for your time @ebygomm​ . Doing a group by using this field which is same id for all intersecting routes, I get a single feature instead of 3 features as seen in the sample data.  The only unique and PK field here is the Cross_Route_ID. 

NLF_ID	| Measure|	|RT_Name| 	|Cross_RouteID|	 | Inter_ID
a1	         8.666	              A-1	                 111111	            987654321
b1	         4.217	              B-1	               222222	            987654321
c1	           9.624                C-1	               333333	             987654321

 

Thank you! That worked for me! However, I would want to include all the other attributes in my data instead of just exposing NLF_ID, NLF_ID_1. 

Reply


Most helpful members this week

Community Stats

30,851
Posts
116,965
Replies
38,682
Members
Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.