Add support for Java High Level REST Client [DATAES-407]

[spring-projects-issues]

Nikita opened DATAES-407 and commented

Starting from version 5.6.0 of Elasticsearch, a new Java client has been released: the Java High Level REST Client. This new client is designed to replace the TransportClient in Java applications which will be deprecated in future versions of Elasticsearch. So we must migrate our codebase from old TransportClient to new REST client.

Quotation from official documentation

We released a low-level REST client in 2016, which is based on the well known Apache HTTP client and it allows to communicate with an Elasticsearch cluster in any version using HTTP. On top of that we released the high-level REST client which is based on the low-level client but takes care of request marshalling and response un-marshalling.

Quotation from a blog post about the state of the official Elasticsearch Java clients

The Java REST client is the future for Java users of Elasticsearch. Please get involved and try out the high-level client as soon as it becomes available, as your feedback will help us to make it better faster. As soon as the REST client is feature complete and is mature enough to replace the Java API entirely, we will deprecate and finally remove the transport client and the Java API.

---

Reference URL: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/5.6/_motivations_around_a_new_java_client.html

Issue Links:

• DATAES-495 Deprecate TransportClient support
  ("is depended on by")
• DATAES-383 Elasticsearch transport client is unstable, switch to Elasticsearch Java ReST client
  ("is duplicated by")
• DATAES-272 Support for other clients besides of the elastic search native client
  ("supersedes")

19 votes, 34 watchers

[spring-projects-issues]

Nikita commented

I started work on this task. In the worst case, the prototype will be ready by October 29

[spring-projects-issues]

Nikita commented

After a deep immersion in the high and low-level rest clients, I found out that not everything is as simple as it seemed at first. A huge amount of functionality is simply not implemented in high level client, and for implementation I need to use a low-level client, which is essentially just a http client. In addition, it will be necessary to adapt almost all tests. On the sidelines, it will take another couple of weeks. Next week, I think, I will lay out a prototype in my repository

[spring-projects-issues]

Don Wellington commented

Is there an update? Can I help with tests if you can point me to a repository with the prototype?

[spring-projects-issues]

ronan quillevere commented

Hello,

Any update on this ?

[spring-projects-issues]

Oliver Henlich commented

Hi @Nikita,
Has this be abandoned in favor of https://github.com/VanRoy/spring-data-jest?

[spring-projects-issues]

Oliver Drotbohm commented

ronan quillevere – I've removed you last comment as it's offensive and rude. Please be sure to follow our Code of Conduct, thanks

[spring-projects-issues]

ronan quillevere commented

Sorry my last comment was not meant to be rude at all, I do not see how it was offensive. I never blamed or attacked anyone, I just noted that, taking into account that no one is responding on any channel and that last update is from more than 6 month ago, it was (in my opinion) fair to assume that this development has been stopped. Am I right ? Please correct me If I am wrong

[spring-projects-issues]

Oliver Drotbohm commented

Assuming someone's death from the lack of movement on a ticket is rude. There hasn't been any progress on this ticket so far. The SD ES team is working on the project on a purely voluntarily basis

[spring-projects-issues]

ronan quillevere commented

By "dev" I meant the development of the feature not the developer of course ! I would never have supposed something like that ?!

[spring-projects-issues]

ronan quillevere commented

So could you confirm that to your knowledge no one is working on this anymore ?

BTW of course thanks for all the voluntarily contributions to all of you !

[spring-projects-issues]

Don Wellington commented

It looks like we should really wait for the 6.3 release because 6.2 doesn't quite support enough APIs to fully implement ElasticsearchOperations.  Particularly, Alias, Mapping, and Settings operations

[spring-projects-issues]

Don Wellington commented

@Oliver Gierke  I have made some progress here, but need a bit of a consult on some possible breaking changes.

First, I am going with creating an ElasticsearchRestTemplate that uses the RestHighLevelClient, to maintain TransportClient support through ES 7.0+.  Does that sound right?

Second, RestHighLevelClient does not implement getClient, so I am going with remove the method from ElasticsearchOperations on the grounds that if you need to get the underlying client you probably need to get the underlying implementation, since you will have to know which client.  Alternatively, should I deprecate the getClient() method, and have ElasticsearchRestTemplate throw an unsupported operation exception, and provide a different method like getRestClient()?

I need to get the tests passing against a hand deployed elasticsearch, then figure out how to update the testing setup to work with the RestHighLevelClient.  Hopefully, I can put together a pull request soon

[spring-projects-issues]

Artur Konczak commented

Hi Don Wellington - good to hear that someone was able to bite this issue - regarding getClient() just make that method to return RestHighLevelClient this will give us access to RestClient via getLowLevelClient.

For missing functionalities - we can try to use LowLevelClient or try to create PR for Elasticseach-rest-client with implementation missing feature  

 

[spring-projects-issues]

Don Wellington commented

I had to take a break. 

Currently, the ElasticsearchRestTemplate is passing a copy of the ElasticsearchTemplate tests against a hand started Elasticsearch.  getSettings, getMappings, and queryAliases were the only things I had to use Low Level Rest Client on.  The Alias code was a bit more effort than it should have been because the REST response for aliases didn't quite match up.  Probably, why the High Level REST client doesn't support it, yet. 

I did some backwards compatibility hacks, but I am thinking the getSettings and getMappings interface should be changed to return Settings and Mappings instead of Map some day.

Next steps:

• Set up a TestNode to run embedded with HTTP enabled, and get the test running on that.
  • Does it need port detection?  I am a little worried about port conflicts on the build server. Not sure how to do this if it is needed.
• RestClientFactoryBean
  • Support XML configuration.
• Dive into spring-boot-starter-data-elasticsearch and see about setting up an AutoConfiguration for the rest client.

 

[spring-projects-issues]

Don Wellington commented

Ok.  Turns out setting up the test and the RestclientFactoryBean were not nearly as much effort as I thought.

Pull Request submitted for review.  Let's see what need fixing