I am having issues connecting to one of our elastic databases. The error I am getting is
Failed to obtain any schemas from reader 'ELASTICSEARCH' from 1 datasets. This may be due to invalid datasets or format accessibility issues due to licensing, dependencies, or module loading.
Any ideas what how to fix.
@miay did you add a new Elasticsearch Reader?
In the beta there should be a old Elasticsearch Reader (deprecated, supports versions 6.8 and lower) and a new Elasticsearch Reader (supports version 7+).
If it doesn't work, then we would definitely want to know. Do you have any logs or workspaces you can share?
Hi @jakemolnar & @bubblebeb ,
I've downloaded the new FME Desktop beta (build 21222) on Win64, and tried the Elastic reader (reading from an internal Elastic 7.9.1, both without and with a json query) and it seems to work!
Thanks @jakemolnar !
@miay did you add a new Elasticsearch Reader?
In the beta there should be a old Elasticsearch Reader (deprecated, supports versions 6.8 and lower) and a new Elasticsearch Reader (supports version 7+).
If it doesn't work, then we would definitely want to know. Do you have any logs or workspaces you can share?
@miay I'm glad I could help! 🙂
@miay did you add a new Elasticsearch Reader?
In the beta there should be a old Elasticsearch Reader (deprecated, supports versions 6.8 and lower) and a new Elasticsearch Reader (supports version 7+).
If it doesn't work, then we would definitely want to know. Do you have any logs or workspaces you can share?
Hi @jakemolnar
I will check the indices with the my dev team. Hopefully it is just the set up here that can be updated.
Thanks
Seb
Hi @jake
I have sorted the connections now. Thanks for your help with that.
Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
The index I am reading has over 2m features and is growing. Using a test I end up with 22k records I am interested in.
I would like to use a feature reader and a where clause but it is greyed out.
thanks
Seb
Hi @jake
I have sorted the connections now. Thanks for your help with that.
Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
The index I am reading has over 2m features and is growing. Using a test I end up with 22k records I am interested in.
I would like to use a feature reader and a where clause but it is greyed out.
thanks
Seb
Hi @bubblebeb,
Glad to hear you've sorted out the connection issues!
> Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
You're in luck: that question has been asked before! https://community.safe.com/s/question/0D54Q000080heSCSAY/how-to-use-sql-like-commands-in-json-query-and-pass-it-through-fme-if-possible-using-httpcaller-elasticreader-and-featurereader
That answer pretty much just links you to our documentation here: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/elasticsearch/elasticsearch-ft-param-r.htm
Hi @bubblebeb,
Glad to hear you've sorted out the connection issues!
> Is there a way of creating a query to pass to Elastic so I don't read in the whole index?
You're in luck: that question has been asked before! https://community.safe.com/s/question/0D54Q000080heSCSAY/how-to-use-sql-like-commands-in-json-query-and-pass-it-through-fme-if-possible-using-httpcaller-elasticreader-and-featurereader
That answer pretty much just links you to our documentation here: https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/elasticsearch/elasticsearch-ft-param-r.htm
Hi @bubblebeb & @jakemolnar ,
We've tried to use a query, and that could look like this:
{
"query": {
"bool": {
"must": m
{
"match": {
"user": "the_username"
}
},
{
"match": {
"code": "200"
}
}
]
}
}
}