Hi, I'm a pretty experienced FME user but this has got me stumped. I'm trying to parse a date field, but it's not parsing the Nulls/Empty/Missing entries despite being selected to

Can you share how you've set up the DateTimeConverter and what your input dates look like?

Hi,

The Transformer is setup as so. Slightly sensitive information so trying to be aware in the screenshots. As you can see, Pass Through is set to "Yes" but there's still plenty coming through the "Rejected" port

If things are passing through the rejected port, it is because they contain values that don't fit the input format. When this happens, the attributes that come out the rejected port have their values replaced by null, even if there was a mix of invalid and valid date formats (not very helpful, i think there is an idea about changing this somewhere).

Yeah that was the idea in trying to make them standardised (to contain a date or nothing/null) beforehand. FME seems to not pick up on a lot of them though, despite me selecting all the options in the Null Attribute Manager beforehand. Everything looks good, then it gets to the DateTimeConverter and it seems to deviate from what I expect by not finding the Nulls/Missing/Blank attributes

I don't think this is what you are seeing, what you are seeing is a Date that doesn't match your input format of %d-%b-%Y which is getting rejected, in the process of rejecting FME sets the date to null, it wasn't null going into the transformer. Do you have some warnings in the log file? Something like 

DateTimeConverter: '%d-%b' was successfully parsed, then failed to parse '-%Y'
DateTimeConverter: Failed to evaluate expression '@DateTimeParse(01-May 1980,%d-%b-%Y,repair)'.  Result is set to null

Thanks for your input 😃

Looking at the Translation Log, there are indeed similar warnings which, it may be the heatwave melting my brain here, but would indicate that even though they're being parsed, still aren't being dealt with optimally

Warning is as follows (and the dates are formatted as, for example, "01-Dec-2015")-

2020-08-12 09:13:26|  25.2|  0.2|INFORM|DateTimeConverter_1: Splitting bulk mode features into features
2020-08-12 09:13:26|  25.2|  0.0|STATS |Storing feature(s) to FME feature store file `E:\temp\wb-cache-Block_Transform_V4_No_D-bNKSak\Main_DateTimeConverter -1 61 fo 1 _lt_REJECTED_gt_  0  5b5d58d1be6ae7a7ac74bdab8bc98290309ba505.ffsupdating'
2020-08-12 09:13:27|  25.7|  0.5|WARN  |DateTimeConverter: '' was successfully parsed, then failed to parse '%d-%b-%Y'
2020-08-12 09:13:27|  25.7|  0.0|WARN  |DateTimeConverter: Failed to evaluate expression '@DateTimeParse(NULL,%d-%b-%Y)'.  Result is set to null

That looks to me like you have values that are the string NULL - not actual null values

Try adding to your Null attribute mapper to handle "Or if Attribute value is:" NULL

Ok - would removing the NullAttributeMapper help? The reasoning behind that was to make sure any non-date entries were treated as Null and not a mix of missing or otherwise not-a-date format.

Going into the DateTimeConverter, the dates look as follows - some will be null or missing as not all have a value (showing both, the dates and null in this screenshot)

Removing the NullAttributeMapper, this is the output, which is expected (as I took one that threw an error - there are 2 entries for it, one with a date, one without) and so it still processed it, but still went through the rejected port despite the option to process Nulls/Missing entries

Honing in on the issue, but just need to make it more reliable as it feels hit and miss

Ooh, no rejected features!

That appears to have fixed it :D, lesson learnt - "when is a Null not a <null>?"

Thank you, ebygomm! Upvoted and marked as the best response

It's even more fun if you get data like this, one of these is a null value the other is a string '<null>' :-)

I'm starting to appreciate up until now, just how nice my source data was from certain teams! :P