Skip to main content

I am new to FME and Elastic search. I am trying to setup elasticsearch reader in FME. I have clicked on reader and set elastic search server, index , type_list and gave username and password. But on cliking ok, I am not able to see the reader on the workspace canvas. I am not able to move forward from this point. I have to get the data from Elasticsearch cloud into Fme desktop. Please help me in progessing further. Any example of loading data from Elastic search cloud to FME desktop would help a lot.

Hi @vkambham, thanks for your question. This sounds to me like you've encountered an error. Do you see any error message in the log window, after you click OK to add the reader? Can you share the logged messages with us?


Below is the log generated while adding a reader(to read from Elasticsearch).

And for the TypeList under constraint is not loading on clicking the browse button . I can see only properties instead of "main" Type(Type in created in Elasticsearch Index) as shown in the attached picture.

 

Starting translation...

FME 2019.0.0.1 Beta (20190415 - Build 19246 - WIN64)

FME Database Edition (node locked-crc)

Temporary License: 26 days left.

Machine host name is: vidya

Operating System: Microsoft Windows 10 64-bit (Build 17134)

Copyright (c) 1994 - 2019, Safe Software Inc.

Safe Software Inc.

FME Configuration: Connection Storage: 'C:\\Users\\meena\\AppData\\Roaming\\Safe Software\\FME\\'

FME Configuration: Using connection values from 'vidya123' for SOURCE_ELASTICSEARCH

Creating reader for format: Elastic Elasticsearch

Trying to find a DYNAMIC plugin for reader named `ELASTICSEARCH'

Loaded module 'ELASTICSEARCH' from file 'C:\\Program Files\\FME\\plugins/gdal/GDAL.dll'

FME API version of module 'ELASTICSEARCH' matches current internal version (3.8 20181030)

ELASTICSEARCH reader: Directive '_HOST' has value 'https://2faf0ad8197746299ab328990f9abae6.us-west-1.aws.found.io:9243/'

ELASTICSEARCH reader: Directive '_USE_AUTHENTICATION' has value 'YES'

ELASTICSEARCH reader: Directive '_AUTHENTICATION_METHOD' has value 'BASIC_AUTH'

ELASTICSEARCH reader: Directive '_BASIC_AUTH_USERNAME' has value 'elastic'

ELASTICSEARCH reader: Directive '_BASIC_AUTH_PASSWORD' has value '********'

ELASTICSEARCH reader: Directive '_RAW_JSON' has not been set. Using default value 'NO'

ELASTICSEARCH reader: Directive '_NUM_FEATURES_PER_REQUEST' has not been set. Using default value '100'

ELASTICSEARCH reader: Directive '_FME_JSON_ATTRIBUTE_FLATTENING_MODE' has value 'ALL_LEVELS'

ELASTICSEARCH reader: Opening index 'https://2faf0ad8197746299ab328990f9abae6.us-west-1.aws.found.io:9243/vidya'

ELASTICSEARCH reader: HTTP Error 400-599: HTTP error code : 400

A fatal error has occurred. Check the logfile above for details

Failed to obtain any schemas from reader 'ELASTICSEARCH' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information

FME Session Duration: 1.9 seconds. (CPU: 0.3s user, 0.2s system)

END - ProcessID: 30300, peak process memory usage: 62768 kB, current process memory usage: 47900 kB

Failed to obtain any schemas from reader 'ELASTICSEARCH' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading. See logfile for more information

Program Terminating

Translation FAILED.

Workspace generation failed. Parameter file was C:\\Users\\meena\\AppData\\Local\\Temp\\FME_1556090033239_31088.


@vkambham, So I am assuming this is a new workspace and new connection you are attempting and not the Readers>Import Feature Type command in an existing workspaces?

And to be certain I know for sure where the error is occurring, it seems that you can connect to the server, you are providing a username and password and then you can view the Type List dialog OK. It is just when you click OK that the error occurs and then FME is unable to add the feature type to the FME Workspace (canvas)?

 

I see you are using FME 2019, but what version of Elastic are you using and where is it being hosted?

 

I have tested that I can add a feature type to the canvas but that is unauthenticated and Elastic 6.0.


Reply