Skip to main content
Solved

DataTimeConverter corrupts data !?!


lifalin2016
Contributor
Forum|alt.badge.img+29

Hi list.

I have a dataset with mixed date/time formats in a single field.

I want to create a cascade to transform these, i.e. "Try this format", and if it fails, "Try that format", etc.

HOWEVER, when data are output on DateTimeConverter's rejected port (signalling a data error), the field value is cleared, rendering a cascaded approach impossible. Why is that ??

Is there a way to avoid this unwarranted nulling of a possible valid field value ?

Cheers.

Best answer by lifalin2016

lifalin2016 wrote:

Actually, it just dawned on me, that it's defined in this file: C:\Program Files\FME 2022\transformers\fmesuite.fmx

The code seems to be Tcl/Tk, which I never really learned.

Are there any Tcl/Tk programmers out there, that may be able to decipher it and suggest a fix ?

YES!!!!

I did manage to decipher the TclTk code myself, and offer a correction.

This is the original code piece:

if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
            FME_SetAttribute $attr {};
         };
      }

This is my fix:

if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute $attr $oldAttrVal;
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
;#            FME_SetAttribute $attr {};
            FME_SetAttribute $attr $oldAttrVal;
         };
      }

And it works 🙂

Now I can use the transformers in a cascading manner.

Yippee!

--------------------- edit:

Actually, the fix need only be added to the "INVALID_INPUT" block:

      if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute $attr $oldAttrVal;
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
            FME_SetAttribute $attr {};
         };
      }

And the updated transformer definition can be kept in a separate FMX file in the same folder as fmesuite.fmx. Just remember to increase its version number.

View original
Did this help you find an answer to your question?

9 replies

david_r
Evangelist
  • June 9, 2022

Unfortunately this is a known issue, you'll find several threads on these forums with users jumping through hoops to work around it, e.g. https://community.safe.com/s/question/0D54Q000080hbnrSAA/datetimeconverter-rejected-port-null

It's regrettable that the issue seems to still be there in 2022.0.


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • June 9, 2022

And it would such an incredibly easy fix, so why is it that hard to fix it ?


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • June 9, 2022
lifalin2016 wrote:

And it would such an incredibly easy fix, so why is it that hard to fix it ?

Actually, it just dawned on me, that it's defined in this file: C:\\Program Files\\FME 2022\\transformers\\fmesuite.fmx

The code seems to be Tcl/Tk, which I never really learned.

Are there any Tcl/Tk programmers out there, that may be able to decipher it and suggest a fix ?


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • Best Answer
  • June 9, 2022
lifalin2016 wrote:

Actually, it just dawned on me, that it's defined in this file: C:\Program Files\FME 2022\transformers\fmesuite.fmx

The code seems to be Tcl/Tk, which I never really learned.

Are there any Tcl/Tk programmers out there, that may be able to decipher it and suggest a fix ?

YES!!!!

I did manage to decipher the TclTk code myself, and offer a correction.

This is the original code piece:

if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
            FME_SetAttribute $attr {};
         };
      }

This is my fix:

if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute $attr $oldAttrVal;
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
;#            FME_SetAttribute $attr {};
            FME_SetAttribute $attr $oldAttrVal;
         };
      }

And it works 🙂

Now I can use the transformers in a cascading manner.

Yippee!

--------------------- edit:

Actually, the fix need only be added to the "INVALID_INPUT" block:

      if {[string length $newAttrVal] == 0} {
         if {{$(PASSTHROUGH)} eq {NO} || [string length $oldAttrVal] > 0} {
            FME_SetAttribute $attr $oldAttrVal;
            FME_SetAttribute fme_rejection_code "INVALID_INPUT";
            break;
         } elseif {!$oldAttrExists} {
            FME_UnsetAttributes $attr;
         } elseif {!$oldAttrNull} {
            FME_SetAttribute $attr {};
         };
      }

And the updated transformer definition can be kept in a separate FMX file in the same folder as fmesuite.fmx. Just remember to increase its version number.


david_r
Evangelist
  • June 9, 2022
lifalin2016 wrote:

Actually, it just dawned on me, that it's defined in this file: C:\\Program Files\\FME 2022\\transformers\\fmesuite.fmx

The code seems to be Tcl/Tk, which I never really learned.

Are there any Tcl/Tk programmers out there, that may be able to decipher it and suggest a fix ?

Thanks for sharing!


DanAtSafe
Safer
Forum|alt.badge.img+17
  • Safer
  • June 9, 2022
david_r wrote:

Unfortunately this is a known issue, you'll find several threads on these forums with users jumping through hoops to work around it, e.g. https://community.safe.com/s/question/0D54Q000080hbnrSAA/datetimeconverter-rejected-port-null

It's regrettable that the issue seems to still be there in 2022.0.

FYI, the unresolved known issue is FMEENGINE-56103


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • June 10, 2022
DanAtSafe wrote:

FYI, the unresolved known issue is FMEENGINE-56103

Hi Dan.

Did you see my fix ?

It's a very small thing to fix in the fmesuite.fmx file, so couldn't it, and similar fixes, be escalated to be fixed asap?

In the meantime:

Will any FMX file in the same folder be read upon start, so one can add custom updated versions as a seperate file?


lifalin2016
Contributor
Forum|alt.badge.img+29
  • Author
  • Contributor
  • June 10, 2022
DanAtSafe wrote:

FYI, the unresolved known issue is FMEENGINE-56103

Hi Dan.

I went ahead and tried it, and yes, a seperate FMX file will indeed be read upon startup, so such customizations can be kept in its own file. Easier to distribute too.


DanAtSafe
Safer
Forum|alt.badge.img+17
  • Safer
  • June 11, 2022
DanAtSafe wrote:

FYI, the unresolved known issue is FMEENGINE-56103

Yes, I saw your fix and I've escalated the issue. Thanks!


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings