Skip to main content
  • EN

I'm using FME 2018.0.

I originally thought this was a problem with the dynamic attribute definition of the writer, but for this test I have used a Manual schema. 

I don't understand why, but the writer is trying to parse a date that I am not seeing in the inspector. The value for Modified_date in the inspector is: 

2011-05-27

The response I get from the error is:

{
    "took": 3,
    "errors": true,
    "items": <
        {
            "index": {
                "_index": "testing_2",
                "_type": "drawings",
                "_id": "3mN9KWMBLz2D_kWNISIW",
                "status": 400,
                "error": {
                    "type": "mapper_parsing_exception",
                    "reason": "failed to parse  Modified_date]",
                    "caused_by": {
                        "type": "illegal_field_value_exception",
                        "reason": "Cannot parse \"0000/00/00 00:00:00.000\": Value 0 for monthOfYear must be in the range Â1,12]"
                    }
                }
            }
        },
        {
            "index": {
                "_index": "testing_2",
                "_type": "drawings",
                "_id": "32N9KWMBLz2D_kWNISIW",
                "status": 400,
                "error": {
                    "type": "mapper_parsing_exception",
                    "reason": "failed to parse  Modified_date]",
                    "caused_by": {
                        "type": "illegal_field_value_exception",
                        "reason": "Cannot parse \"0000/00/00 00:00:00.000\": Value 0 for monthOfYear must be in the range S1,12]"
                    }
                }
            }
        }
    ]
}

What is the simplest workspace that can create a date, and write that to an ES index using dynamic field attributes? I've attached a MWE by way of example - maybe there's something fundamentally wrong with how I'm doing it. Can anyone help? Some ( @takashi, @jakemolnar) have previously said that dates must be FME format, which I've done, but I cannot get them parsed as dates by ES.

 

 

mwe-dates-to-es.fmw

As I have answered already in a relevant thread, FME handles date/time values as character strings while translating and the SchemaSetter won't consider any attribute value as a date/time value even if it was formatted with Standard FME Date/Time format.

If you actually need to dynamically configure destination schema, you will have to implement a process to define the destination schema (i.e. create "attribute{}" list). See these articles to learn more.

Or, if basically destination schema can be derived from the source dataset dynamically but you just need to add an identical date field always, you could also statically add it to every destination schema through the User Attributes tab in the dynamic writer feature type.

As I have answered already in a relevant thread, FME handles date/time values as character strings while translating and the SchemaSetter won't consider any attribute value as a date/time value even if it was formatted with Standard FME Date/Time format.

If you actually need to dynamically configure destination schema, you will have to implement a process to define the destination schema (i.e. create "attribute{}" list). See these articles to learn more.

Or, if basically destination schema can be derived from the source dataset dynamically but you just need to add an identical date field always, you could also statically add it to every destination schema through the User Attributes tab in the dynamic writer feature type.

Or, if basically destination schema can be derived from the source
dataset dynamically but you just need to add an identical date field
always, you could also statically add it to every destination schema through the User Attributes tab in the dynamic writer feature type.

 

Thanks Takashi, this was what I needed to do.

 

The ES Writer doesn't seem able to give a data type when in dynamic mode. All attributes get written as type text.

 

As I have answered already in a relevant thread, FME handles date/time values as character strings while translating and the SchemaSetter won't consider any attribute value as a date/time value even if it was formatted with Standard FME Date/Time format.

If you actually need to dynamically configure destination schema, you will have to implement a process to define the destination schema (i.e. create "attribute{}" list). See these articles to learn more.

Or, if basically destination schema can be derived from the source dataset dynamically but you just need to add an identical date field always, you could also statically add it to every destination schema through the User Attributes tab in the dynamic writer feature type.

Good to hear.

 

I believe that the ES writer can configure destination schema dynamically, but you need to define a schema definition correctly. Again, the SchemaSetter is useless in your case. See the articles I have linked above to learn how you can create a schema definition.

 

 

Reply


Community Stats

31,819
Posts
120,860
Replies
39,451
Members

Latest FME

Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.