Solved

DataTimeConverter corrupts data !?!

lifalin2016
Rank
Userlevel 1
Badge +22

Hi list.

I have a dataset with mixed date/time formats in a single field.

I want to create a cascade to transform these, i.e. "Try this format", and if it fails, "Try that format", etc.

HOWEVER, when data are output on DateTimeConverter's rejected port (signalling a data error), the field value is cleared, rendering a cascaded approach impossible. Why is that ??

Is there a way to avoid this unwarranted nulling of a possible valid field value ?

Cheers.

icon

Best answer by lifalin2016 9 June 2022, 10:59

View original
-- Cheers, Lars I.

9 replies

david_r
Rank
Userlevel 4

Unfortunately this is a known issue, you'll find several threads on these forums with users jumping through hoops to work around it, e.g. https://community.safe.com/s/question/0D54Q000080hbnrSAA/datetimeconverter-rejected-port-null

It's regrettable that the issue seems to still be there in 2022.0.

lifalin2016
Rank
Userlevel 1
Badge +22

And it would such an incredibly easy fix, so why is it that hard to fix it ?

-- Cheers, Lars I.
lifalin2016
Rank
Userlevel 1
Badge +22

And it would such an incredibly easy fix, so why is it that hard to fix it ?

Actually, it just dawned on me, that it's defined in this file: C:\\Program Files\\FME 2022\\transformers\\fmesuite.fmx

The code seems to be Tcl/Tk, which I never really learned.

Are there any Tcl/Tk programmers out there, that may be able to decipher it and suggest a fix ?

-- Cheers, Lars I.
lifalin2016
Rank
Userlevel 1
Badge +22

Actually, it just dawned on me, that it's defined in this file: C:\Program Files\FME 2022\transformers\fmesuite.fmx

The code seems to be Tcl/Tk, which I never really learned.

Are there any Tcl/Tk programmers out there, that may be able to decipher it and suggest a fix ?

YES!!!!

I did manage to decipher the TclTk code myself, and offer a correction.

This is the original code piece:

if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
            FME_SetAttribute $attr {};
         };
      }

This is my fix:

if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute $attr $oldAttrVal;
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
;#            FME_SetAttribute $attr {};
            FME_SetAttribute $attr $oldAttrVal;
         };
      }

And it works 🙂

Now I can use the transformers in a cascading manner.

Yippee!

--------------------- edit:

Actually, the fix need only be added to the "INVALID_INPUT" block:

      if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute $attr $oldAttrVal;
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
            FME_SetAttribute $attr {};
         };
      }

And the updated transformer definition can be kept in a separate FMX file in the same folder as fmesuite.fmx. Just remember to increase its version number.

-- Cheers, Lars I.
david_r
Rank
Userlevel 4

Actually, it just dawned on me, that it's defined in this file: C:\\Program Files\\FME 2022\\transformers\\fmesuite.fmx

The code seems to be Tcl/Tk, which I never really learned.

Are there any Tcl/Tk programmers out there, that may be able to decipher it and suggest a fix ?

Thanks for sharing!

DanAtSafe
Rank
Userlevel 1
Badge +11

Unfortunately this is a known issue, you'll find several threads on these forums with users jumping through hoops to work around it, e.g. https://community.safe.com/s/question/0D54Q000080hbnrSAA/datetimeconverter-rejected-port-null

It's regrettable that the issue seems to still be there in 2022.0.

FYI, the unresolved known issue is FMEENGINE-56103

lifalin2016
Rank
Userlevel 1
Badge +22

FYI, the unresolved known issue is FMEENGINE-56103

Hi Dan.

Did you see my fix ?

It's a very small thing to fix in the fmesuite.fmx file, so couldn't it, and similar fixes, be escalated to be fixed asap?

In the meantime:

Will any FMX file in the same folder be read upon start, so one can add custom updated versions as a seperate file?

-- Cheers, Lars I.
lifalin2016
Rank
Userlevel 1
Badge +22

FYI, the unresolved known issue is FMEENGINE-56103

Hi Dan.

I went ahead and tried it, and yes, a seperate FMX file will indeed be read upon startup, so such customizations can be kept in its own file. Easier to distribute too.

-- Cheers, Lars I.
DanAtSafe
Rank
Userlevel 1
Badge +11

FYI, the unresolved known issue is FMEENGINE-56103

Yes, I saw your fix and I've escalated the issue. Thanks!

Reply


Community Stats

28,694
Posts
109,434
Replies
36,544
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.