Skip to main content
Released

Date Formatter to handle milliseconds

Related products:FME Form
  • November 30, 2015
  • 9 replies
  • 102 views
D
Forum|alt.badge.img

I work on Internet of Things projects. We have lots of sensors that provide data that needs to be timestamped with milisecond accuracy so that we can see which combination of sensors recognize events and objects. I want to do correlation between data at a milisecond accurracy.

The issue is that I cannot directly convert the time information (milliseconds from epoc) to date with the DateFormatter, since the %s directive only takes integer values;

http://docs.safe.com/fme/html/FME_Transformers/FME_Transformers.htm#Transformers/dateformatter.htm

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

9 replies

david_r
Celebrity
  • david_rCelebrity
  • December 2, 2015

I agree, I've had the same need.

For what it's worth, I ended up solving it using Python.

M

I agree too and would 'enhance' the Topic.

a Date/Time Format with fractional second is needed and should applicable to all transformers which are using Date-Formats (TimeStamper, DateFormatter ...).

Oracle's TIME and TIMESTAMP Datatype is defined by

TIME [(fractional_second_precision)]

I've got to Import CSV-Data with Informations like "17Jul1990:11:22:33.4567", driven trough the DateFormatter with 'Source Date Format' = "%d%b%Y:%T" and

'Destination Date Format' = "%Y%m%d%H%M%S" Ends up with, suprise suprise,

"19900717112233". The fractional seconds are lost!

You've got extract them before going through DateFormatter, and You have to do this for each Time-attribut of the Feature. The DateFormatter can convert all of them in one transformer, the fractional-handling has to be done for each Attribut on it's own ..

t_hohenstrater
Contributor
Forum|alt.badge.img+2

I support this idea. I am current facing the same issue downloading the data from arcgisonline which stores its time/date in Epoch milliseconds.

fmelizard
Safer
Forum|alt.badge.img+21
  • fmelizardSafer
  • Safer
  • March 8, 2017

Here is a workspace to deal with epoch milliseconds until this idea is implemented.

Actually, this is not quite the answer as the sub second precision will be lost if I understand the comments here correctly.

 

epoch-to-date.fmw
fmelizard
Safer
Forum|alt.badge.img+21
  • fmelizardSafer
  • Safer
  • March 8, 2017
Could you share your code?

 

 

david_r
Celebrity
  • david_rCelebrity
  • March 9, 2017
I would, if I could remember where I used it, way back in 2015 :-)
L
Forum|alt.badge.img

Hi @dr_ysg

I am very happy to announce that FME 2017 supports fractional seconds (up to 9 decimal places). The new DateTimeFormatter transformer will be implemented in FME 2017.1. In 2017.0, please use datetime functions (see Text Editor > Date/Time Functions). For example, to convert seconds from epoch into human friendly temporal value:

_datetime = @DateTimeParse(_secondsFromEpoch, %s) _datetime = @DateTimeFormat(_datetime, %Y%m%d %T)

or

_datetime = @DateTimeFormat(@DateTimeParse(_secondsFromEpoch, %s), %Y%m%d %T)

Fractional seconds will be preserved in all datetime operations in FME 2017. We are now testing Readers and Writers to make sure they have no issues with reading/writing.

Please don't use the old DateFormatter though, as it will still drop the fractional part. DateFormatter will be deprecated as soon as we have DateTimeFormatter ready.

K
  • kiwigis
  • March 21, 2018
I couldn't get the text workarounds above to work in FME 2017.0

 

 

I had the data in the correct format for SQL Server datetime2 "2016-02-16 22:11:15.8990000", but FME refused to write this to the database as it didn't acknowledge it as a FME date type.

 

 

Work around was as follows:

 

1)

 

Atrtibute manager: new attribute: SaveMilliseconds TextFunction: @Substring(@Value(ReportDate),20,7)

 

 

2)

 

Then use DateFormatter to tell FME ReportDate is a Date field (losing milliseconds)

 

'

 

 

3)

 

Attribute Manager: Set value of ReportDate

 

TextFunction: @Value(ReportDate).@Value(SaveMilliseconds)

 

 

Remove column SaveMilliseconds

 

----------------------------------------------------------------------------------------------------

 

This allowed me to successfully write a datetime2 field in SQL server with milliseconds using a SQL Spatial Writer.

 

 

L
Forum|alt.badge.img
Hi @kiwigis

 

I guess, you get values read as strings. If you visualize them using Data Inspector, they are displayed in the same 2016-02-16 22:11:15.8990000 format, am I correct?

 

To write these values back into a database, you need to convert them into FME datetime format. This can be done either using @DateTimeParse() function or with DateTimeConverter:

 

 

'

 

 

Community Stats

32,207
Posts
122,305
Replies
39,891
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.