Question

Custome transformer - Overriding attribute doesn't work every time

  • 26 September 2019
  • 7 replies
  • 1 view

Badge +1

Hello,

 

I have a problem with my FME script.

My goal is to create a custom transformer that does the same thing as AttributeValueMapper but by using a dynamic list off matching value.

Why ? Because I have a file witch contain data that I need to clean :

File "data"

field_test_1

 

field_2

 

field_test_2

 

xxx

 

Good morning

 

ab

 

xxx

 

Hello

 

ac

 

xxx

 

Salut

 

ab

 

zzz

 

Bonjour

 

ab

 

yyy

 

Konishiwa

 

ac

 

 

 

To do this, I have a file which contain all my matching values like this :

File "code_list"

code

 

value

 

CODE_A_01

 

value xxx

 

CODE_A_02

 

value yyy

 

CODE_A_03

 

value zzz

 

CODE_B_01

 

value ab

 

CODE_B_02

 

value ac

 

CODE_C_01

 

value xxx

 

 

My goal is to replace value in "field_test_1" and "field_test_2" from the file "data" by codes listed in the file "code list".

Little things to keed in mind : codes need to be filtered as values from differents fields can be the same but not corresponding to the same code (has you can see lines 1 and 6 from file "code_list").

 

First try :

I've used TestFilter + FeatureJoiner + AttributeManager to replace values in my fields : It work as expected but it fill my script with lot's of transformers.

 

Second try :

I've created a custom transformer to achieve my goal but I have some trouble with it :

Here is published parameters of the transformer :

Here is what you can find inside my transformer :

I get the value of the fields I need, filter codes, perform a join, set the value for the input data field and then removed every field that I didn't need including fields from the "code list" file (with regex).

 

When I run my script whit one occurence, it works perfectly.

 

But when I put a second occurence of this custom transformer like this :

Data in "field_test_2" are <missing> and I don't know exactly why.

 

Bonus : If the tested field contain a space, inside fields aren't removed :

 

I put the script and the files as attachments if you want to give a try.

 

Thank you for your help.

Arthur Bazin


7 replies

Badge +10

I suspect there is some conflict between the parameters

Badge +3

@savoierider

After the first instancxe of the transformer, the inputs are not available

You can prevent that by making your customtransform generic. It looks generic, but it is not.

I found that doing following helped alleviate the problem in my generic custom transformers:

 

This is from my generic stringsearcher and remover (it removes a strings from targetstring)

 

The attribute creator @Value(@Value() does not work if you do it directly on the parameter! (i tried..of course)

You must fetch the parameter into attributes. It now contains the name of the (input) attribute.

To get the value, the second @Value is required. You have to type it after chosing the attribute and entering the texteditor!

Now you should not have the parameter confusion.

 

I tried it on your attributemanager2 in fme 2019:

It works, the input data is now passed.

 

I admit, this to be like a hidden mystery, and wonder if @safe should, like, fix it?

 

Greets!

Badge +3

..also you might reconsider using the parameter (its value i assume to get the target-attributename) as output.

 

I refer to attributes for that, leaving the parameter.

Badge +1

Hi @gio,

Thank's for your answer.

One more time, I missed the ParameterFetcher... Thank you for your solution.

 

I'v tried and it solves the problem with spaced field.

Unfortunatly, I keep the <missing> fields when I use the second occurance of the transformer.

 

Here is the new configuration :

I fetch my parameters into new attributes : "_param_input_XXX" then I set the value of the field mentionned in the param to the attribute "_input_XXX".

But it didn't resolve the first problem.

 

Any other idea ?

Here is the new version of the script : DynamicValueMapper_v2.zip

 

Thank you

Best regards

Badge +3

@savoierider

 

Hi,

I made a working custom.

I failed to do the parameter2attribute thing using a attributemanager. But it works when using a attributecreator. Also coulld not get past the featurejoiner so i used a featuremerger.

By exporting the custom (which i later embedded) the attributes pass.

The Liste now passes through (tidier)

(only thing is, i have not manage to split the attributes at the result though)

 

This one runs 3 instances:

DynamicValueMapper_G.fmw

tt.fmx

 

Badge +1

@savoierider

 

Hi,

I made a working custom.

I failed to do the parameter2attribute thing using a attributemanager. But it works when using a attributecreator. Also coulld not get past the featurejoiner so i used a featuremerger.

By exporting the custom (which i later embedded) the attributes pass.

The Liste now passes through (tidier)

(only thing is, i have not manage to split the attributes at the result though)

 

This one runs 3 instances:

DynamicValueMapper_G.fmw

tt.fmx

 

Hi @gio,

 

Ok, you definitively put me on the way of the solution. :-)

 

In fact, the feature joiner doesn't work for generic transformer. You need to use FeatureMerger.

In my second script, you just have to replace FeatureJoiner by FeatureMerger and it works...

 

So, if I sum up the solution :

If you want to create a generic transformer, you need :

  • To use ParameterFetcher to get the value of parameters associated with features and then using it for naming attributes.
  • To use FeatureMerger if you want to perform joins.

 

Thank's for the time you spend on my problem, I'll make more try with other transformers next time.

It seems that FME have some trouble with some transformers (or maybe I don't understand the hole logic behind some transformers :-)

 

Have a nice day.

Arthur Bazin

 

Badge +3

@savoierider

 

Hi,

In fme 2018 it is possible to use the featurejoiner as well as the atributemanager.

I just added them in the dynamicvaluemapper-g.fmw

using FME(R) 2018.0.0.2 (20180414 - Build 18301 - WIN64)

and it still does its job.

 

So maybe in fme 2019 version something went awry?

(I dislike the fme2019 interface anyways...;))

 

Reply