+6I am having issues connecting to one of our elastic databases. The error I am getting is
Failed to obtain any schemas from reader 'ELASTICSEARCH' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading.
Any ideas what how to fix.
Best answer by jakemolnar
What is the version of your Elasticsearch server? FME currently only supports versions 6.8 and lower. You can check this by visiting your server root page in a browser (example: http://my.server.name:9200/).
Here's a link to the idea for supporting v7+: https://knowledge.safe.com/content/idea/101746/add-support-for-elasticsearch-v7.html
Otherwise, what version of FME are you using? Is it the beta stream perhaps?
@miay did you add a new Elasticsearch Reader?
In the beta there should be a old Elasticsearch Reader (deprecated, supports versions 6.8 and lower) and a new Elasticsearch Reader (supports version 7+).
If it doesn't work, then we would definitely want to know. Do you have any logs or workspaces you can share?
Hi @jakemolnar & @bubblebeb ,
I've downloaded the new FME Desktop beta (build 21222) on Win64, and tried the Elastic reader (reading from an internal Elastic 7.9.1, both without and with a json query) and it seems to work!
Thanks @jakemolnar !
@miay did you add a new Elasticsearch Reader?
In the beta there should be a old Elasticsearch Reader (deprecated, supports versions 6.8 and lower) and a new Elasticsearch Reader (supports version 7+).
If it doesn't work, then we would definitely want to know. Do you have any logs or workspaces you can share?
@miay I'm glad I could help! 🙂
+6@miay did you add a new Elasticsearch Reader?
In the beta there should be a old Elasticsearch Reader (deprecated, supports versions 6.8 and lower) and a new Elasticsearch Reader (supports version 7+).
If it doesn't work, then we would definitely want to know. Do you have any logs or workspaces you can share?
Hi @jakemolnar
I will check the indices with the my dev team. Hopefully it is just the set up here that can be updated.
Thanks
Seb
+6Hi @jake
I have sorted the connections now. Thanks for your help with that.
Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
The index I am reading has over 2m features and is growing. Using a test I end up with 22k records I am interested in.
I would like to use a feature reader and a where clause but it is greyed out.
thanks
Seb
Hi @jake
I have sorted the connections now. Thanks for your help with that.
Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
The index I am reading has over 2m features and is growing. Using a test I end up with 22k records I am interested in.
I would like to use a feature reader and a where clause but it is greyed out.
thanks
Seb
Hi @bubblebeb,
Glad to hear you've sorted out the connection issues!
> Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
You're in luck: that question has been asked before! https://community.safe.com/s/question/0D54Q000080heSCSAY/how-to-use-sql-like-commands-in-json-query-and-pass-it-through-fme-if-possible-using-httpcaller-elasticreader-and-featurereader
That answer pretty much just links you to our documentation here: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/elasticsearch/elasticsearch-ft-param-r.htm
Hi @bubblebeb,
Glad to hear you've sorted out the connection issues!
> Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
You're in luck: that question has been asked before! https://community.safe.com/s/question/0D54Q000080heSCSAY/how-to-use-sql-like-commands-in-json-query-and-pass-it-through-fme-if-possible-using-httpcaller-elasticreader-and-featurereader
That answer pretty much just links you to our documentation here: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/elasticsearch/elasticsearch-ft-param-r.htm
Hi @bubblebeb & @jakemolnar ,
We've tried to use a query, and that could look like this:
{
"query": {
"bool": {
"must": [
{
"match": {
"user": "the_username"
}
},
{
"match": {
"code": "200"
}
}
]
}
}
}
