Skip to main content
Solved

Can I export source and destination values from the AttributeValueMapper?


I have a very long list of attributes that I have typed into an AttributeValueMapper transformer. I now want to share this mapping with someone (not an FME user), so I would like to export the source and destination values to excel. I see that there is a button to import to this transformer, but not one to export. Is there a way around this, or doing I have a lot of typing to do?

Best answer by david_r

Attached is a workspace (FME 2016.1.3+ required) that uses the workspace reader to export all the AttributeValueMapper settings of a specified workspace to a CSV file. The resulting CSV file has a column for the transformer name and the source and destination values. It should be handling international characters just fine (but do let me know if it doesn't).

Sample input:

Output generated by exportattributevaluemapper.fmw, as imported into Excel:

It should be fairly simple to extend so as to include the default values as well as the source and destination attribute names.

Main workspace: exportattributevaluemapper.fmw

Test workspace: avm.fmw

Output from test workspace: attributevaluemappersettings.csv

View original
Did this help you find an answer to your question?

27 replies

ebygomm
Influencer
Forum|alt.badge.img+31
  • Influencer
  • March 5, 2015

Can you not just write out the source and desitination attributes to a csv after passing through a duplicate remover?


gio
Contributor
Forum|alt.badge.img+15
  • Contributor
  • March 5, 2015
If all the source values are present prior to going into the Valuemapper, you can, as Grommm suggested, write them out to txt\\csv using the FileWriter.

 

Sameway you can use filewriter to write destination vlaues after the Valuemapper.

 

 

Of course it is somewhat more handy to create the ValueMap in a txt-file int he first place..

  • Author
  • March 10, 2015
I dont think I was using all of the values so I ended up opening up the FMW in a text editor and did some find a replaces.

Forum|alt.badge.img+4
  • February 16, 2017

I have this same problem, except I don't have a source dataset with all the values in it.

Just inherited an AttributeValueMapper with a long list of Source and Destination values (about 300), and now have no way of getting them out of there. Any suggestions? Maybe @takashi knows?


erik_jan
Contributor
Forum|alt.badge.img+17
  • Contributor
  • February 16, 2017

And this is exactly why you should use a Joiner, joining for example to CSV or Excel, instead of an AttributeValueMapper if you have a large number of values you want to map.


takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 16, 2017
dmatranga wrote:

I have this same problem, except I don't have a source dataset with all the values in it.

Just inherited an AttributeValueMapper with a long list of Source and Destination values (about 300), and now have no way of getting them out of there. Any suggestions? Maybe @takashi knows?

Hi @dmatranga, I don't think there is a way to export the mapping table from the AttributeValueMapper parameters setting to an external table, unfortunately. If you will have to maintain the mapping rule in the future, consider creating an external table manually and modifying the workflow so that the attribute mapping refers to the external table, as @erik_jan suggested.

Forum|alt.badge.img+4
  • February 16, 2017
erik_jan wrote:

And this is exactly why you should use a Joiner, joining for example to CSV or Excel, instead of an AttributeValueMapper if you have a large number of values you want to map.

Joiner no longer exists. There is now a DatabaseJoiner, but that cannot be used for joining to Excel or a CSV. What would you suggest using instead? FeatureMerger?

 

 


takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 16, 2017
dmatranga wrote:
Joiner no longer exists. There is now a DatabaseJoiner, but that cannot be used for joining to Excel or a CSV. What would you suggest using instead? FeatureMerger?

 

 

Really? It seems to be planned that the Joiner will be upgraded and renamed to DatabaseJoiner in FME 2017, but I believe that the transformer still supports any tabular format such as CSV and Excel. Of course you can also use the FeatureMerger.

 


erik_jan
Contributor
Forum|alt.badge.img+17
  • Contributor
  • February 16, 2017
dmatranga wrote:
Joiner no longer exists. There is now a DatabaseJoiner, but that cannot be used for joining to Excel or a CSV. What would you suggest using instead? FeatureMerger?

 

 

I checked in the FME 2017 beta and it seems the DatabaseJoiner can join to Excel:

 

 


erik_jan
Contributor
Forum|alt.badge.img+17
  • Contributor
  • February 16, 2017

I do have a quick (and dirty) workaround for this:

Copy the AttributeValueMapper in the workspace to the clipboard.

The open a text editor (notepad or similar) and paste from the clipboard.

This will give you the definition of the transformer (including a list of all values) in text format.

This can be read (after a little editing) using the CSV reader.


takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 17, 2017
erik_jan wrote:

I do have a quick (and dirty) workaround for this:

Copy the AttributeValueMapper in the workspace to the clipboard.

The open a text editor (notepad or similar) and paste from the clipboard.

This will give you the definition of the transformer (including a list of all values) in text format.

This can be read (after a little editing) using the CSV reader.

Good idea. However, unicode characters and some special characters within the parameter values have been encoded in the mapping file. The method cannot be adopted by the users in a non-ascii locale like me, unfortunately...

 

The attachment is an experimental workspace, which demonstrates a possible way to parse a workspace (*.fmw) with the FMW reader, extract source/destination values from AttributeValueMapper parameters setting, and decode them.

 

extract-mapping-table-from-attributevaluemapper.fmw (FME 2016.1.3.1)

 


itay
Contributor
Forum|alt.badge.img+16
  • Contributor
  • February 17, 2017

Next to the joiner another possibility to use the external mapping file is the SchemaMapper.

I usually try to avoid the AttributeValueMapper as much as possible and just use it in 'throw away' workspaces.


taojunabc
Participant
Forum|alt.badge.img+6
  • Participant
  • February 20, 2017
erik_jan wrote:

I do have a quick (and dirty) workaround for this:

Copy the AttributeValueMapper in the workspace to the clipboard.

The open a text editor (notepad or similar) and paste from the clipboard.

This will give you the definition of the transformer (including a list of all values) in text format.

This can be read (after a little editing) using the CSV reader.

@takashi

The workspace you provided is excellent, thanks.

 

But I also found another simple way to avoid the problem of coding,

 

That is, first create the Summary Annotation for AttributeValueMapper, and then copy the Annotion paste into the text editor.


david_r
Evangelist
  • February 20, 2017
taojunabc wrote:

@takashi

The workspace you provided is excellent, thanks.

 

But I also found another simple way to avoid the problem of coding,

 

That is, first create the Summary Annotation for AttributeValueMapper, and then copy the Annotion paste into the text editor.

Very creative, excellent.

takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 20, 2017
taojunabc wrote:

@takashi

The workspace you provided is excellent, thanks.

 

But I also found another simple way to avoid the problem of coding,

 

That is, first create the Summary Annotation for AttributeValueMapper, and then copy the Annotion paste into the text editor.

Cool! Could also be used for some other scenarios.

 


taojunabc
Participant
Forum|alt.badge.img+6
  • Participant
  • February 20, 2017
taojunabc wrote:

@takashi

The workspace you provided is excellent, thanks.

 

But I also found another simple way to avoid the problem of coding,

 

That is, first create the Summary Annotation for AttributeValueMapper, and then copy the Annotion paste into the text editor.

I found that there is another way, directly open the workspace(*.fmv) with a text editor, Find the desired row by searching for a list value, and then copy the required content.

 

The advantage of this method is not only can be applied to Transformer, can also be applied to user parameters, For example, get the values of choice type parameters etc.

 


takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 20, 2017
taojunabc wrote:
I
 found that there is another way, directly open the workspace(*.fmv) with a 
text editor, Find the desired row by searching for a list value, and then copy the required content. 

 

The advantage of this method is not only can be applied to Transformer, 
can also be applied to user parameters, For example, get the values of choice type parameters etc.

 

The disadvantage of the method is that unicode characters and some special characters within the AttributeValueMapper parameters have been encoded. e.g.

 

#!     <XFORM_PARM PARM_NAME="LUT" PARM_VALUE="<u65e5><u672c><u8a9e>,<u6bcd><u56fd><u8a9e>,<u82f1><u8a9e>,<u5916><u56fd><u8a9e>"/>
I don't know an easy way to decode them, other than using FME predefined Tcl FME_DecodeText function.

 


david_r
Evangelist
  • February 20, 2017
takashi wrote:
The disadvantage of the method is that unicode characters and some special characters within the AttributeValueMapper parameters have been encoded. e.g.

 

#!     <XFORM_PARM PARM_NAME="LUT" PARM_VALUE="<u65e5><u672c><u8a9e>,<u6bcd><u56fd><u8a9e>,<u82f1><u8a9e>,<u5916><u56fd><u8a9e>"/>
I don't know an easy way to decode them, other than using FME predefined Tcl FME_DecodeText function.

 

Since I like Python :-)

 

from fmeobjects import FMESession
def DecodeText(feature):
    text = feature.getAttribute('text')
    decoded_text = FMESession().decodeFromFMEParsableText(text)
    feature.setAttribute('text', decoded_text)
Returns:

 

`text' has value `="???,???,??,???'
Hopefully it's correct!

taojunabc
Participant
Forum|alt.badge.img+6
  • Participant
  • February 20, 2017
takashi wrote:
The disadvantage of the method is that unicode characters and some special characters within the AttributeValueMapper parameters have been encoded. e.g.

 

#!     <XFORM_PARM PARM_NAME="LUT" PARM_VALUE="<u65e5><u672c><u8a9e>,<u6bcd><u56fd><u8a9e>,<u82f1><u8a9e>,<u5916><u56fd><u8a9e>"/>
I don't know an easy way to decode them, other than using FME predefined Tcl FME_DecodeText function.

 

Oh, I did make a mistake.

 

However,
 you can use the first method to get the list value of transformer, and 
use second methods to get values of user parameters.

 

In
 addition, I query some information, as well as after testing, found 
that Python can be relatively simple to restore these characters.

 

def processFeature(feature):   s= u'\u65e5\u672c\u8a9e'   feature.setAttribute("attr",s)
It will get result '???'.

 

 

 


takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 20, 2017
takashi wrote:
The disadvantage of the method is that unicode characters and some special characters within the AttributeValueMapper parameters have been encoded. e.g.

 

#!     <XFORM_PARM PARM_NAME="LUT" PARM_VALUE="<u65e5><u672c><u8a9e>,<u6bcd><u56fd><u8a9e>,<u82f1><u8a9e>,<u5916><u56fd><u8a9e>"/>
I don't know an easy way to decode them, other than using FME predefined Tcl FME_DecodeText function.

 

@david_r, yes, absolutely correct. I wasn't able to find out the FMESettion.decodeFromFMEParsableText method so far. Learned something new. Thanks!

 


david_r
Evangelist
  • Best Answer
  • February 20, 2017

Attached is a workspace (FME 2016.1.3+ required) that uses the workspace reader to export all the AttributeValueMapper settings of a specified workspace to a CSV file. The resulting CSV file has a column for the transformer name and the source and destination values. It should be handling international characters just fine (but do let me know if it doesn't).

Sample input:

Output generated by exportattributevaluemapper.fmw, as imported into Excel:

It should be fairly simple to extend so as to include the default values as well as the source and destination attribute names.

Main workspace: exportattributevaluemapper.fmw

Test workspace: avm.fmw

Output from test workspace: attributevaluemappersettings.csv


takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 20, 2017
david_r wrote:

Attached is a workspace (FME 2016.1.3+ required) that uses the workspace reader to export all the AttributeValueMapper settings of a specified workspace to a CSV file. The resulting CSV file has a column for the transformer name and the source and destination values. It should be handling international characters just fine (but do let me know if it doesn't).

Sample input:

Output generated by exportattributevaluemapper.fmw, as imported into Excel:

It should be fairly simple to extend so as to include the default values as well as the source and destination attribute names.

Main workspace: exportattributevaluemapper.fmw

Test workspace: avm.fmw

Output from test workspace: attributevaluemappersettings.csv

@david_r, thanks for sharing the elegant Python script. To make it robuster, I would suggest modifying the script so that decoding will be performed after splitting the parameter by commas. Because, if a source or destination string value had a comma, it has been encoded as "<comma>" here and it can not be decoded before splitting.

 


david_r
Evangelist
  • February 20, 2017
takashi wrote:
@david_r, thanks for sharing the elegant Python script. To make it robuster, I would suggest modifying the script so that decoding will be performed after splitting the parameter by commas. Because, if a source or destination string value had a comma, it has been encoded as "<comma>" here and it can not be decoded before splitting.

 

Good catch, I agree completely. Fixed.

takashi
Contributor
Forum|alt.badge.img+19
  • Contributor
  • February 20, 2017
takashi wrote:
The disadvantage of the method is that unicode characters and some special characters within the AttributeValueMapper parameters have been encoded. e.g.

 

#!     <XFORM_PARM PARM_NAME="LUT" PARM_VALUE="<u65e5><u672c><u8a9e>,<u6bcd><u56fd><u8a9e>,<u82f1><u8a9e>,<u5916><u56fd><u8a9e>"/>
I don't know an easy way to decode them, other than using FME predefined Tcl FME_DecodeText function.

 

@taojunabc, yes, Python unicode operations could be applied to decode encoded unicode characters generally. However, some special characters such as comma, bracket within a parameter value will also be encoded with FME specific manner. I therefore think you will have to use FME functions (Python or Tcl) to decode the encoded parameter values.

 


bilal
Contributor
Forum|alt.badge.img+4
  • Contributor
  • March 13, 2020
david_r wrote:

Attached is a workspace (FME 2016.1.3+ required) that uses the workspace reader to export all the AttributeValueMapper settings of a specified workspace to a CSV file. The resulting CSV file has a column for the transformer name and the source and destination values. It should be handling international characters just fine (but do let me know if it doesn't).

Sample input:

Output generated by exportattributevaluemapper.fmw, as imported into Excel:

It should be fairly simple to extend so as to include the default values as well as the source and destination attribute names.

Main workspace: exportattributevaluemapper.fmw

Test workspace: avm.fmw

Output from test workspace: attributevaluemappersettings.csv

David, This is awesome. Thank you so much for sharing.


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings