Solved

Conditional Value in AttributeManager evaluates to nan when the value of an Attribute is "NAN".


Hello, newbie here.

 

I'm using AttributeManager to rename and remove some Attributes - and to add a new Attribute using a Conditional Value based upon (among others) an Attribute named IATA, which contains airport call letters. For San Francisco International Airport, IATA contains "SFO" but for Nadi International Airport, the IATA Attribute contains "NAN". No matter how I try to force FME to treat those call letters as a string, it eventually becomes "if NAN != null... and I get "nan" as the result. Pretty sure that's a bug (2019.2).

 

Note: Set Value with @Value(IATA) works fine with "NAN". It seems to be only the Conditional Value code that has the problem.

 

Does anyone have an idea how to conditionally process an Attribute that may contain the value "NAN" (or other values that might also be mathematical constants)?

icon

Best answer by markatsafe 19 November 2021, 22:13

View original

10 replies

Userlevel 4
Badge +25

That's a very interesting issue. Just a few questions to start off with:

  • What format is your source data in?
  • If you check the source data in the FME Data Inspector and look at Nadi International Airport, what does it show for the IATA attribute?

 

The source data is in JSON format, from a RESTful API. After the HTTPCaller, I break the JSON into Features with a JSONFragmenter. Then I feed the Features into the AttributeManager.

 

I haven't had time to get familiar with the Data inspector; I just used the Visual Preview on the JSONFragmenter and the AttributeManager. Both show the IATA Attribute as "NAN" (all caps, without the quotes) for the Nadi Feature, but AttributeManager shows "nan" (without the quotes) in the new Attribute with the Conditional Value, and that's what FME writes into the Postgres database.

 

I modified the Conditional Value formula to wrap the IATA value in quotation marks and got "NAN" (with quotes) but further modification to then strip off the added quotation marks caused the "If NAN != null" problem to return. So I used AttributeManager to simply add another Attribute with Set Value @Value(IATA). Visual Preview successfully shows "NAN" (without quotes) for the new Attribute.

 

I tried using a StringConcatenator to append the constant "N" (without quotes) to the constant "NA" (without quotes) in a new Attribute named IATA, and feeding that into an AttributeManager with the Conditional Value "If @Value(IATA) NOT_ATTRIBUTE_VALUE_NULL". AttributeManager shows "nan" (without quotes) in its Visual Preview and so does StringConcatenator ! I also tried just an AttributeManager with the Conditional Value "If @Substring("NANN",0,3) NOT_ATTRIBUTE_VALUE_NULL" and it too produces "nan", even with Comparison Mode "Case Sensitive".

 

The string "NAN" seems to cause FME grief in certain Transformers.

Badge +2

The source data is in JSON format, from a RESTful API. After the HTTPCaller, I break the JSON into Features with a JSONFragmenter. Then I feed the Features into the AttributeManager.

 

I haven't had time to get familiar with the Data inspector; I just used the Visual Preview on the JSONFragmenter and the AttributeManager. Both show the IATA Attribute as "NAN" (all caps, without the quotes) for the Nadi Feature, but AttributeManager shows "nan" (without the quotes) in the new Attribute with the Conditional Value, and that's what FME writes into the Postgres database.

 

I modified the Conditional Value formula to wrap the IATA value in quotation marks and got "NAN" (with quotes) but further modification to then strip off the added quotation marks caused the "If NAN != null" problem to return. So I used AttributeManager to simply add another Attribute with Set Value @Value(IATA). Visual Preview successfully shows "NAN" (without quotes) for the new Attribute.

 

I tried using a StringConcatenator to append the constant "N" (without quotes) to the constant "NA" (without quotes) in a new Attribute named IATA, and feeding that into an AttributeManager with the Conditional Value "If @Value(IATA) NOT_ATTRIBUTE_VALUE_NULL". AttributeManager shows "nan" (without quotes) in its Visual Preview and so does StringConcatenator ! I also tried just an AttributeManager with the Conditional Value "If @Substring("NANN",0,3) NOT_ATTRIBUTE_VALUE_NULL" and it too produces "nan", even with Comparison Mode "Case Sensitive".

 

The string "NAN" seems to cause FME grief in certain Transformers.

@pbyhistorian​  Definitely looks like an issue we should fix. Do you have a small example workspace that illustrates the issue and the conditional. I have reproduce something similar, I just want to ensure I'm reproducing your precise issue. Thanks

@pbyhistorian​  Definitely looks like an issue we should fix. Do you have a small example workspace that illustrates the issue and the conditional. I have reproduce something similar, I just want to ensure I'm reproducing your precise issue. Thanks

Hello, Mark. Sorry for the delay; got a little free time today. Here’s the example and two records (one good, one “nan”) of our actual data. The JSON reader expects the JSON file to be in C:\\FME. Thanks, Eric
Badge +2

@pbyhistorian​  Definitely looks like an issue we should fix. Do you have a small example workspace that illustrates the issue and the conditional. I have reproduce something similar, I just want to ensure I'm reproducing your precise issue. Thanks

@pbyhistorian​ I don't see the attachment. Can you try again or send it to mark @ safe.com. Thx

@pbyhistorian​  Definitely looks like an issue we should fix. Do you have a small example workspace that illustrates the issue and the conditional. I have reproduce something similar, I just want to ensure I'm reproducing your precise issue. Thanks

Oops, didn't notice this post for a (long) while. Trying again.

 

This version is completely standalone; I got rid of the JSON file import. It shows the problem appearing in multiple Transformers and configurations, as well a couple of Transformers that don't have the problem - in the hope it will help find the bug.

Badge +2

This is an issue in FME and we will have it fixed in FME 2022. The workaround is to initialize the conditional result attribute first, probably as a string value - i.e result = default. You can do this in the same AttributeCreator:

result = default

result = <conditional>

This will prevent FME from attempting a type conversion (NAN is also a numeric - not-a-number). Y

Nicely spotted, Mark!

It worked in all three AttributeManagers. (StringConcatenator shouldn't be needed now.)

imageimageimageThanks!

Badge +9

Can anyone confirm that this issue has been fixed with FME 2022 ?

We struggle with it as well (FME 2020) and like to avoid the workarounds in future.

thanks

KLaus

Badge +2

This is fixed in FME 2022.1 which soon be available in beta.

Reply