Dynamic string concatanation in FME DESKTOP 2013

Hi Subrat,

 

Have you tried using the stringconcatenator ? 

Hi Itay,

 

In StringConcatenator, i have to hard code attribute names and their sequence. but here, X attribute may contain

 

Concat(Concat(a,b),c)

 

or

 

Concat(Concat(b,a),c)

 

or

 

Concat(b,c)

 

or

 

b

 

 

I have to evaluate this expression and write it.

 

I tried with inline querier but unable to use feature attributes.

 

Please let me know how to solve this.

 

 

Thanks.

 

Hi Subrat,

 

 

You can use the TestFilter to branch the feature flow into four flows according to X value, and then use four StringConcatenators to do different concatenating operations.

 

 

Alternatively, if you are using FME 2013 SP1 or later, you can also use "Conditional Mapping" functionality of the AttributeCreator, like this.

 

Select "Set To Conditional Value..." for setting Value parameter.

 

 

 

  I like this functionality very much.

 

See also this article: FME 2013-SP1: Conditional Processing in FME http://evangelism.safe.com/fmeevangelist113/

 

 

Takashi

Yes conditional mapping has my preference too

This is an experimental solution. A PythonCaller with this script might work. -----

 

import fmeobjects, re   def processFeature(feature):     x = str(feature.getAttribute('X'))     s = ''     for name in re.sub('.*\\(|\\)|\\s', '', x).split(','):         v = feature.getAttribute(name)         if v: s += str(v)     feature.setAttribute('concatenated', s)

 

-----

 

Hi Takashi,

 

  I am using FME(R) 2013 (20130207 - Build 13264 - WIN32) version of FME desktop,but i am not able to open screens shown by you for AttributeCreator.

 

 

Please let me know whether i am using a wrong version else how can i open "Conditional Mapping" functionality of the AttributeCreator.

 

 

Actually i have around 8-10 columns and their combination can be one or more,also  in any order. So python may suit my requirement. 

 

My feature has 2 million records. Will usage of python hamper my execution timings??

 

Thanks.

 

 

 

Hi,

 

 

The minimum requirement for "Conditional Mapping" is FME 2013 SP1 (released in late of March), I don't think your FME version supports it unfortunately. I recommend you to upgrade your FME to the latest version if possible.

 

FME Downloads

 

http://www.safe.com/support/support-resources/fme-downloads/   > My feature has 2 million records. Will usage of python hamper my execution timings?

 

Depending on the environment. If the performance of execution may be too inefficient, I would also consider to use two SQLExecutors in a series. Naturally, this approach can be applicable only if the source dataset is a database.   1st SQLExecutor: select distinct X from table_name   2nd SQLExecutor: select id, foo, bar, @Value(X) as concatenated from table_name where X = '@Value(X)'

 

Takashi

Hi, I am currently using Sqlexecuter approach. But it is taking much time. Hence please let me know any work around. Thanks.

Hi,

 

 

I currently have no other ideas. In my experiences, querying with SQL statement seems to achieve the best performance in many cases. But I have never used SQL statement containing "concat" function in a workspace, the Python approach might be better in this case. The exact answer can not be known until you measure time actually.   About general issues on FME performance, see Dave's posts and related links in this Q&A: Dynamic Schema and Memory issues https://safe.secure.force.com/AnswersQuestionDetail?id=906a0000000coggAAA

 

 

Takashi

Hi,

 

 

the SQLExecutor is bound to be slow for thousands / millions of features as it makes a separate call to the database for each feature.

 

 

You could use the dynamical nature of Python to your advantage here, though. It should be much faster. Example:

 

 

import fmeobjects   def Concat(part1, part2):     return str(part1) + str(part2)   def FeatureProcessor(feature):     text = feature.getAttribute("text")     a = feature.getAttribute("a")     b = feature.getAttribute("b")     c = feature.getAttribute("c")     concatenated = eval(text)     feature.setAttribute("result", concatenated)

 

 

Given a feature with the following attributes:

 

 

text: "Concat(Concat(a,b),c)"

 

a: "aaa"

 

b: "bbb"

 

c: "ccc"

 

 

The above script will create the attribute

 

 

result: "aaabbbccc"

 

 

David