Skip to main content
Solved

permute a list, output to csv

  • November 27, 2014
  • 9 replies
  • 60 views

I have a list of features, and I want to output a permutation of all combinations of two and two list members to a csv file.

 

Example:

 

Feature 1

 

List{0} = 11

 

List{1} = 22

 

List{3} = 33

 

Feature 2

 

List{0} = 44

 

List{1} = 55

 

....more features

 

 

Should produce this csv-file:

 

11, 22

 

11, 33

 

22, 11

 

22, 33

 

33, 11

 

33, 22

 

44, 55    Remark: This is feature 2

 

55, 44

 

....

 

 

The lists seem to be fine, I can use AttributeExposer and output single items. So my challenge is to output the combinations while not knowing before rruntime how many list members there are (will most often be 2, and generally less than 4).

 

 

I have been experimenting with listExploder to give me one line per list member, but it leaves me with the combination problem. I'm new to FME, so a suggestion of appropriate transformers would be appreciated. Thank,s.

 

Regards Erik

 

 

Best answer by takashi

Hi Eric,

 

 

I think you are on the right track - the ListExploder.

 

The point would be how to do "CROSS JOIN" after the exploding.

 

 

The FeatureMerger (Process Dupliacte Suppliers: Yes, Duplicate Supplier List Name: <list name>) + ListExploder performs the processing like "CROSS JOIN".

 

 

 

If you are familiar with SQL, the InlineQuerier can also be used effectively.

 

 

 

Takashi
View original
Did this help you find an answer to your question?
This post is closed to further activity.
It may be a question with a best answer, an implemented idea, or just a post needing no comment.
If you have a follow-up or related question, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

9 replies

takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • Best Answer
  • November 28, 2014
Hi Eric,

 

 

I think you are on the right track - the ListExploder.

 

The point would be how to do "CROSS JOIN" after the exploding.

 

 

The FeatureMerger (Process Dupliacte Suppliers: Yes, Duplicate Supplier List Name: <list name>) + ListExploder performs the processing like "CROSS JOIN".

 

 

 

If you are familiar with SQL, the InlineQuerier can also be used effectively.

 

 

 

Takashi

takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • November 28, 2014
In the first approach, the ListExploder_2 is not essential.

 

You can remove it, and connect the AttributeRenamer_2 to the first ListExploder.

takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • November 28, 2014
Update.

 


takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • November 28, 2014
Scripting could be simpler and faster. All the transformers can be replaced with a PythonCaller (Attributes to Expose: Field1, Field2).

 

-----

 

# Script Example for the PythonCaller

 

import fmeobjects

 

class FeatureProcessor(object):

 

    def input(self, feature):

 

        L = feature.getAttribute('List{}')

 

        if L:

 

            for v1, v2 in [(v1, v2) for v1 in L for v2 in L if v1 != v2]:

 

                newFeature = fmeobjects.FMEFeature()

 

                # If you need to retain the List{} and other attributes,

 

                # replace the line above with this.

 

                # newFeature = feature.cloneAttributes()

 

                newFeature.setAttribute('Field1', v1)

 

                newFeature.setAttribute('Field2', v2)

 

                self.pyoutput(newFeature)

 

-----

 

Several ways. You can have your choice ;)

  • Author
  • November 28, 2014
Hi Takashi,

 

Many thanks. The InlineQuerier did the trick perfectly for now. Maybe I'll also check out the PythonCaller at a later stage, so thanks for that example as well.

 

Erik

takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • November 29, 2014
I've overlooked a case where duplicate elements occur in a List.

 

e.g. List{} = [11, 22, 22]

 

 

In such a case, the workspaces discard combinations that consist of same values.

 

11, 22

 

11, 22

 

22, 11

 

22, 11

 

 

Could there be duplicate elements in a List?

 

If you need to create {22, 22} in such a case, you will have to modify them.

 

11, 22

 

11, 22

 

22, 11

 

22. 22

 

22, 11

 

22, 22

 

 

SQL:

 

1) Set the "Element Index Attribute" of the ListIndexcer. e.g. "_element_index"

 

2) In the InlineQuerier, add column to the Source Table.

 

Name: _element_index, Type: integer

 

3) Modify the SQL statement:

 

-----

 

select a.List as Field1, b.List as Field2

 

from Source as a cross join Source as b

 

where a._count = b._count and a._element_index <> b._element_index

 

-----

 

 

Python:

 

Modify the script.

 

-----

 

# Script Example for the PythonCaller

 

import fmeobjects

 

class FeatureProcessor(object):

 

    def input(self, feature):

 

        L = feature.getAttribute('List{}')

 

        if L:

 

            for v1, v2 in [(v1, v2) for i, v1 in enumerate(L) for j, v2 in enumerate(L) if i != j]:

 

                newFeature = fmeobjects.FMEFeature()

 

                # If you need to retain the List{} and other attributes,

 

                # replace the line above with this.

 

                # newFeature = feature.cloneAttributes()

 

                newFeature.setAttribute('Field1', v1)

 

                newFeature.setAttribute('Field2', v2)

 

                self.pyoutput(newFeature)

 

-----

  • Author
  • December 1, 2014
For this specific list, there should be no duplicates in the list (and if it were, I'd be happy to get rid of them). For more general use, you point out an important thing to consider, so thanks for the update.

  • August 12, 2015
Hi! I am having a simular problem, but cannt get the feature merger method to work and I am not an SQL person...

 

 

I have a table with cell IDs, that all relate to a path. some cells relate to start and some relate to end. There can be duplicate cell IDs relating to different paths. I need to find all possible combinations of start to end cells by path which I will later use to route with. Here is an example of my table structure: 

 

Cell_ID, Type, Path

 

1 End 3

 

2 End 3

 

3 End 3

 

4 End 3

 

5 Start 3

 

6 Start 3

 

7 Start 3

 

8 Start 3

 

9 Start 3

 

10 End 5

 

11 End 5

 

12 End 5

 

13 End 5

 

14 Start 5

 

15 Start 5

 

16 Start 5

 

1 Start 5

 

2 Start 5

 

3 Start 5

 

 

I have tried seperating start and end then feature merging on path with duplicate suppliers. This gives me the correct number of records once the list has been exploaded (e.g. is there are 10 start and 10 end in a path I should have 100 records). But they are all duplicates.

 

 

I have also tried the counter idea from above and I get the same amount of records out as I put in. Instead of finding all combinations it is just a 1-1 join.

 

 

Any ideas?

takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • August 12, 2015
Hi,

 

 

I think you are on the right track, but maybe the setting of the "Accumulation Mode" parameter in the ListExploder should be modified.

 

e.g.

 

 

Hope this helps.

 

Takashi

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings