Hi! I am using FME 2016 DateFormatter to process a field that some time will contain invalid text. I configured the DateFormatter to only process text in a specific format, but give the same text output to all other input formats.However, the DateFormatter will crash the FME for input of some specific input. The screenshot below shows the configuration I uses in DateFormatter and the error message. The input field value is "8 a.m." without quote. The other inputs such as "between 8 a.m. and 3 p.m." will also crash the FME execution. More details:if I input some other "nicer" random text such as "sldfjsldfj", the DateFormatter will successfully return "some text" as configured.if I change the Source Date Format configuration to "Auto-Detection", "some text" will also be returned.Attached is my test FME script.My guess is, DateFormatter is unfortunately tricked by some specific text when it attempt to parse the input. such that it started being parsed, and then the FME execution is crashed. My questions are:What is the simple and reliable way I can work around this issue? Unfortunately I cannot use "Auto-Detection" in the Source Date Format configuration, because the FME script was just changed from that to work-around another issue (https://knowledge.safe.com/questions/53076/dateformatter-shifts-time-if-source-date-format-se.html)Is this a known defect in FME 2016?Does the new DateTimeConvertor in FME 2017 behaves the same? or it has fixed the issue?Thank you!

Solved

FME 2016 DateFormatter failed to handle some invalid date attribute?

6 years ago
3 October 2017
9 replies
20 views

sui_huang
36 replies

Hi! I am using FME 2016 DateFormatter to process a field that some time will contain invalid text. I configured the DateFormatter to only process text in a specific format, but give the same text output to all other input formats.

However, the DateFormatter will crash the FME for input of some specific input. The screenshot below shows the configuration I uses in DateFormatter and the error message. The input field value is "8 a.m." without quote. The other inputs such as "between 8 a.m. and 3 p.m." will also crash the FME execution.

More details:

if I input some other "nicer" random text such as "sldfjsldfj", the DateFormatter will successfully return "some text" as configured.
if I change the Source Date Format configuration to "Auto-Detection", "some text" will also be returned.

Attached is my test FME script.

My guess is, DateFormatter is unfortunately tricked by some specific text when it attempt to parse the input. such that it started being parsed, and then the FME execution is crashed.

My questions are:

What is the simple and reliable way I can work around this issue? Unfortunately I cannot use "Auto-Detection" in the Source Date Format configuration, because the FME script was just changed from that to work-around another issue (https://knowledge.safe.com/questions/53076/dateformatter-shifts-time-if-source-date-format-se.html)
Is this a known defect in FME 2016?
Does the new DateTimeConvertor in FME 2017 behaves the same? or it has fixed the issue?

Thank you!

icon

Best answer by lenaatsafe 3 October 2017, 20:03

View original

9 replies

lenaatsafe
275 replies
6 years ago
3 October 2017
Best Answer

Hi @sui_huang

I would really want to suggest upgrading to FME 2017.1 for better datetime support. FME 2016 DateFormatter has a list of known issues - this was one of the reasons why we launched DateTime project for FME 2017.

You can have more than one FME install on your machine. Would you mind downloading FME 2017.1 from https://www.safe.com/support/support-resources/fme-downloads/ and giving it a try?

Hi @sui_huang

You can have more than one FME install on your machine. Would you mind downloading FME 2017.1 from https://www.safe.com/support/support-resources/fme-downloads/ and giving it a try?

cuserssui-huangdesktopfme2017test.fmw

Hi @LenaAtSafe

I tried FME 2017.1 with the new DateTimeConverter transformer. The same input will still crashes the FME execution session. In the attached fme2017 test workspace, if the AttributeCreator transformer that populates invalid value is disable, the other feature with valid data will go through.

Userlevel 3

+17

takashi
Contributor
7538 replies
6 years ago
3 October 2017

Hi @sui_huang

You can have more than one FME install on your machine. Would you mind downloading FME 2017.1 from https://www.safe.com/support/support-resources/fme-downloads/ and giving it a try?

Hi @sui_huang, unlike the previous DateFormatter, the new DataTimeConverter in FME 2017.1+ will reject the feature having an invalid date/time string against the specified Input Format.

You can connect a transformer (or a Junction) to the <Rejected> port in order to handle (or just discard) the rejected features appropriately.

Alternatively, you can also set "Continue Translation" to the "Rejected Feature Handling" parameter in the Navigator/Workspace Parameters/Translation, if all features rejected by transformers in the workspace can be discarded.

Hi @sui_huang, unlike the previous DateFormatter, the new DataTimeConverter in FME 2017.1+ will reject the feature having an invalid date/time string against the specified Input Format.

You can connect a transformer (or a Junction) to the <Rejected> port in order to handle (or just discard) the rejected features appropriately.

Hi @takashi in the case I tested, it did direct the feature to <reject> port, but still crashed, such that other features cannot be processed.

Userlevel 3

+17

takashi
Contributor
7538 replies
6 years ago
3 October 2017

Hi @takashi in the case I tested, it did direct the feature to <reject> port, but still crashed, such that other features cannot be processed.

Have you connected a transformer such as Logger or a Junction to the <Rejected> port?

cuserssui-huangdesktopfme2017test.fmw

Hi @LenaAtSafe

I agree with @takashi - it is the Workbench and workspace parameter now. Please check either

Tools > FME Options > Translation > Rejected feature Handling Default

Navigator > Workspace Parameters > Translation > Rejected Feature Handling

Hi @takashi in the case I tested, it did direct the feature to <reject> port, but still crashed, such that other features cannot be processed.

@takashi no I didn't, but I am sure it crashed because another feature already go through when the error happened.

Hi @takashi in the case I tested, it did direct the feature to <reject> port, but still crashed, such that other features cannot be processed.

Please alter your Rejected Feature Handling parameter.

Hi @takashi in the case I tested, it did direct the feature to <reject> port, but still crashed, such that other features cannot be processed.

@LenaAtSafe changing the parameter works. Thank you!

@Takashi I tried and found adding a logger as you suggested will also make it work. Thank you!

FME 2016 DateFormatter failed to handle some invalid date attribute?

9 replies

Reply

Community Stats

Reply

Community Stats

Sign up

An FME Account is required to contribute

Login to the community

An FME Account is required to contribute

Scanning file for viruses.

This file cannot be downloaded