Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Delete Index support to high-level REST client #27019

Merged
merged 14 commits into from
Oct 26, 2017

Conversation

catalin-ursachi
Copy link
Contributor

@catalin-ursachi catalin-ursachi commented Oct 15, 2017

Added high-level REST client support for one of the endpoints covered by #25847. Will try and add some more, if this one's fine. 🙂

(Haven't been able to get gradle check to run through yet - due to error:

:docs:integTestCluster#installAnalysisIcuPlugin FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':docs:integTestCluster#installAnalysisIcuPlugin'.
> A problem occurred starting process 'command 'cmd''

but ES builds fine, and the newly added tests pass.)

@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@s1monw
Copy link
Contributor

s1monw commented Oct 16, 2017

@elasticmachine ok to test

@s1monw
Copy link
Contributor

s1monw commented Oct 16, 2017

@catalin-ursachi this is awesome. It's exactly how we envisioned this client to grow. We will go through the code with you soon but bear with us we have a large team meeting this week so it might be delayed a bit. Thanks for opening this PR

@s1monw
Copy link
Contributor

s1monw commented Oct 16, 2017

@catalin-ursachi FYI you can just cd client/rest-high-level/ and rungradle check from there no need to run all tests here on you end. Also gradle precommit is useful as it allows you to only run stuff like checkstyle etc.

Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @catalin-ursachi thanks a lot for your PR! it looks good, I left some comments and requested some changes. Also, it would be nice to add some docs for this in https://github.com/elastic/elasticsearch/tree/master/docs/java-rest/high-level . Ping me if you need any help!

@@ -378,6 +390,11 @@ static String endpoint(String[] indices, String[] types, String endpoint) {
return endpoint(String.join(",", indices), String.join(",", types), endpoint);
}


static String endpoint(String[] indices, String endpoint) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you remove the endpoint argument here, as it is not needed. We will add it later if/when we need it for other APIs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that it would then clash with the method below:

    /**
     * Utility method to build request's endpoint.
     */
    static String endpoint(String... parts) {
        StringJoiner joiner = new StringJoiner("/", "/", "");
        for (String part : parts) {
            if (Strings.hasLength(part)) {
                joiner.add(part);
            }
        }
        return joiner.toString();
    }

To be fair, having multiple endpoint methods that all take String collections as parameters, but have different behaviour, does feel like asking for trouble. How about renaming one or the other to buildEndpoint, buildPath, or similar?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, I could just not add a new method and call:

  String endpoint = endpoint(deleteIndexRequest.indices(), new String[0], "");

Would that be preferable?


Params parameters = Params.builder();
parameters.withTimeout(deleteIndexRequest.timeout());
parameters.withIndicesOptions(deleteIndexRequest.indicesOptions());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that master_timeout is ignored, can you read and set that one too?

@@ -347,6 +349,26 @@ public void deleteAsync(DeleteRequest deleteRequest, ActionListener<DeleteRespon
}

/**
* Deletes an index using the Delete Index api
* <p>
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-delete-index.html">Delete API on elastic.co</a>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you replace here in the javadocs Delete API with Delete Index API?

/**
* Asynchronously deletes an index using the Delete Index api
* <p>
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-delete-index.html">Delete API on elastic.co</a>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above: Delete API -> Delete Index API?

*/
public void deleteIndexAsync(DeleteIndexRequest deleteIndexRequest, ActionListener<DeleteIndexResponse> listener, Header... headers) {
performRequestAsyncAndParseEntity(deleteIndexRequest, Request::deleteIndex, DeleteIndexResponse::fromXContent, listener,
Collections.singleton(404), headers);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ignore set is needed for response parsing, in case an error code is returned but the corresponding response body needs to be parsed into a response object rather than an exception. That is needed for the GET API for instance, as 404 corresponds to a valid GetResponse being returned with found set to false. IN this case, 404 corresponds to an error which needs to be parsed into an exception, so I don't think we need to add 404 to the ignore set, such set should be empty. Makes sense?


public class IndexAdminIT extends ESRestHighLevelClientTestCase {

public void testDeleteIndex_ifIndexExists() throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably merge the two methods into a single testDeleteIndex method

@@ -61,4 +68,41 @@ protected void readAcknowledged(StreamInput in) throws IOException {
protected void writeAcknowledged(StreamOutput out) throws IOException {
out.writeBoolean(acknowledged);
}

@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is a potentially good change, but should not be part of this PR as it's not really needed. Also, if we go down this route, we should move code from AcknowledgedRestListener#buildResponse to AcknowledgedResponse and change the way RestDeleteIndexAction prints the response body out, otherwise we introduce code duplication. Would you mind doing this in another PR and reverting it here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, sorry, that was just some over-enthusiastic copy & pasting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no worries!

static String endpoint(String[] indices, String endpoint) {
return endpoint(String.join(",", indices), endpoint);
}

/**
* Utility method to build request's endpoint.
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a unit test method to RequestTests for the new deleteIndex method?

@@ -48,4 +51,34 @@ public void writeTo(StreamOutput out) throws IOException {
super.writeTo(out);
writeAcknowledged(out);
}

public static DeleteIndexResponse fromXContent(XContentParser parser) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to add unit tests for this parsing method, similar to what we did in IndexResponseTests, GetResponseTests etc.

}

protected static void parseInnerToXContent(XContentParser parser, AcknowledgedResponse.Builder context) throws IOException {
XContentParser.Token token = parser.currentToken();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given this hierarchy, maybe we should check already if we support classes that add their custom fields to the response, like CreateIndexResponse. Maybe we can add parsing code for that one too (only parsing, without adding the rest of the API)? What do you think?

@catalin-ursachi
Copy link
Contributor Author

Thanks for the review; I'll write the changes as soon as I get the time (possibly Sunday). 🙂

@@ -378,6 +390,10 @@ static String endpoint(String[] indices, String[] types, String endpoint) {
return endpoint(String.join(",", indices), String.join(",", types), endpoint);
}

static String endpoint(String[] indices, String endpoint) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(see comment thread: #27019 (comment))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your comments, how about using the existing endpoint method then and calling it like this endpoint(indices, Strings.EMPTY_ARRAY, "") ?

@catalin-ursachi
Copy link
Contributor Author

Addressed all the review comments (still unsure about #27019 (comment), however). Will write the docs shortly.

@@ -378,6 +390,10 @@ static String endpoint(String[] indices, String[] types, String endpoint) {
return endpoint(String.join(",", indices), String.join(",", types), endpoint);
}

static String endpoint(String[] indices, String endpoint) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your comments, how about using the existing endpoint method then and calling it like this endpoint(indices, Strings.EMPTY_ARRAY, "") ?

*
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices.html">Indices API on elastic.co</a>
*/
public class IndicesClient {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we make this final?

@@ -221,6 +221,15 @@ public final void close() throws IOException {
}

/**
* Provides an {@link IndicesClient} which can be used to access the Indices API.
*
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices.html">Indices API on elastic.co</a>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you replace the version number with current like in the other links in this class?

/**
* A wrapper for the {@link RestHighLevelClient} that provides methods for accessing the Indices API.
*
* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices.html">Indices API on elastic.co</a>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you replace 5.6 with current?

}

private void createIndex(String index) throws IOException {
Response response = client().performRequest("PUT", "/" + index);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you don't need the "/" in front of the index name here. the client will do it for you if I remember correctly

assertEquals(200, response.getStatusLine().getStatusCode());
}

private boolean indexExists(String index) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we make this static?

}
}

private void createIndex(String index) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this be static?

@@ -61,4 +68,41 @@ protected void readAcknowledged(StreamInput in) throws IOException {
protected void writeAcknowledged(StreamOutput out) throws IOException {
out.writeBoolean(acknowledged);
}

@Override
public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no worries!

Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi @catalin-ursachi I did another review round and left a bunch of comments. Most of them are really minor though, this looks great already! Thanks a lot!

* See <a href="https://www.elastic.co/guide/en/elasticsearch/reference/5.6/indices.html">Indices API on elastic.co</a>
*/
public IndicesClient indices() {
return new IndicesClient(this);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you eagerly instantiate this and return always the same instance? What's the advantage of creating a new instance every time?

deleteIndexRequest.indicesOptions(IndicesOptions.fromOptions(randomBoolean(), randomBoolean(), randomBoolean(),
randomBoolean()));
}
expectedParams.put("ignore_unavailable", Boolean.toString(deleteIndexRequest.indicesOptions().ignoreUnavailable()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you want to share this same piece of code with testSearch by making a common private method?

Copy link
Contributor Author

@catalin-ursachi catalin-ursachi Oct 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's what I intended; however, the DeleteIndexRequest and the SearchRequest don't share any superclass; they define the indicesOptions methods separately. So a common method would have to take getter and setter lambdas as arguments; would that be preferable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the lambdas idea, maybe you can do the same with timeout.

@@ -61,4 +66,10 @@ protected void readAcknowledged(StreamInput in) throws IOException {
protected void writeAcknowledged(StreamOutput out) throws IOException {
out.writeBoolean(acknowledged);
}

protected static final ParseField ACKNOWLEDGED = new ParseField("acknowledged");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be protected?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed it doesn't.

private static final ParseField INDEX = new ParseField("index");

private static final ConstructingObjectParser<CreateIndexResponse, Void> PARSER = new ConstructingObjectParser<>("create_index",
true, a -> new CreateIndexResponse((boolean) a[0], (boolean) a[1], (String) a[2]));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rename a to args?

}

private static final ConstructingObjectParser<DeleteIndexResponse, Void> PARSER = new ConstructingObjectParser<>("delete_index",
true, a -> new DeleteIndexResponse((boolean) a[0]));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rename a to args here too?

* The left element is the actual {@link XContentBuilder} to serialize while the right element is the
* expected {@link CreateIndexResponse} after parsing.
*/
public static Tuple<XContentBuilder, CreateIndexResponse> randomCreateIndexResponse(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename this to createTestItem? also, can it be private?

builder.field("acknowledged", acknowledged);
builder.field("shards_acknowledged", shardsAcked);
builder.field("index", index);
builder.endObject();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for testing purposes, I guess it makes sense to make CreateIndexResponse and DeleteIndexResponse implement ToXContentObject (part of what I made you revert, sorry!). You could also add a TODO that explains that once all of the AcknowledgedResponses are migrated we can update the rest listener accordingly.

With such change we can then use toShuffledXContent and make this method return a CreateIndexResponse rather than a Tuple.

We could then add a simple testToXContent method as well like we have in GetResponseTests and friends.

Copy link
Contributor Author

@catalin-ursachi catalin-ursachi Oct 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be tempted to add a TODO here, asking for the test to be updated once AcknowledgedResponses implement ToXContent. Would that be okay, or would you rather I changed 'Create Index' and 'Delete Index' to implement ToXContent now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind the above; I've modified CreateIndexResponse and DeleteIndexResponse to implement ToXContentObject, and added a TODO in AcknowledgedRestListener.java.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ thanks I think there is some value in this from a testing point of view, thank you!

assertCreateIndexResponse(expectedCreateIndexResponse, parsedCreateIndexResponse);
}

public static void assertCreateIndexResponse(CreateIndexResponse expected, CreateIndexResponse actual) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be public?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, do we even need a method for this given that we call it only once?

* The left element is the actual {@link XContentBuilder} to serialize while the right element is the
* expected {@link DeleteIndexResponse} after parsing.
*/
public static Tuple<XContentBuilder, DeleteIndexResponse> randomDeleteIndexResponse(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would align this to CreateIndexResponse once you updated it according to my comments above, same direction here too.

}

/**
* Asynchronously deletes an index using the Delete Index api
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make api capital in all of its occurrences please?

@javanna
Copy link
Member

javanna commented Oct 24, 2017

retest this please

Copy link
Member

@javanna javanna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks a lot for your work @catalin-ursachi !!! I left a couple of minor comments, but those are the last things I could find, and very small ones. I will merge this once you addressed those.

@@ -281,6 +269,8 @@ public void testDeleteIndex() throws IOException {
assertNull(request.getEntity());
}



Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: may I ask you to remove these two empty lines?

expectedParams.put("expand_wildcards", "closed");
} else {
expectedParams.put("expand_wildcards", "none");
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should simplify this method. Given that we generate the random indices options as part of the method, we could reuse those rather than using the getter, then the getter argument is not needed. Also you could just return the indices options and that would remove the need for the setter argument too. Thoughts?

Copy link
Contributor Author

@catalin-ursachi catalin-ursachi Oct 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if we don't generate an IndicesOptions, the getter then gets the default for the given request. Without that, we'd have to pass in the default to this method.

So, we could change it to:

private static IndicesOptions createAndExpectRandomIndicesOptions(
    IndicesOptions default, Map<String, String> expectedParams) {
  if (randomBoolean()) {
    ...
  } else {
    //use default
    return null;
  }

And then handle the returned value depending on whether it's null or not. Is that nicer, though?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point! I missed the default case. I do not like the null invariant that we would introduce here, leave it as-is. Thanks for your patience ;)

@@ -365,7 +376,7 @@ public void searchAsync(SearchRequest searchRequest, ActionListener<SearchRespon
}

/**
* Executes a search using the Search Scroll api
* Executes a search using the Search Scroll API
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch :) thanks for adjusting these

}



Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may I ask you to remove these empty lines and leave just one?

include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[delete-index-request-timeout]
--------------------------------------------------
<1> Timeout to wait for primary shard to become available as a `TimeValue`
<2> Timeout to wait for primary shard to become available as a `String`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is rather the "Timeout to wait for the all the nodes to acknowledge the index deletion"

<2> Called in case of failure. The raised exception is provided as an argument

[[java-rest-high-delete-index-response]]
==== Delete Response
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make it "Delete Index Response" ?

[[java-rest-high-delete-index-response]]
==== Delete Response

The returned `DeleteResponse` allows to retrieve information about the executed
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DeleteIndexResponse rather than DeleteResponse


["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{doc-tests}/IndicesClientDocumentationIT.java[delete-index-response]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a little bullet point explaining what the acknowledged field holds? "Indicates whether all of the nodes have acknowledged the request or not"

@@ -1,4 +1,5 @@
include::_index.asciidoc[]
include::deleteindex.asciidoc[]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move this one line above?

}
}

private static <T> void setRandomTimeout(Function<String, T> setter, TimeValue defaultTimeout, Map<String, String> expectedParams) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here and in setRandomIndicesOptions, you could replace the Function with a Consumer. That works and makes the generics go away.

@catalin-ursachi
Copy link
Contributor Author

The failure looks unrelated to the changes; could you please retest?

@tvernum
Copy link
Contributor

tvernum commented Oct 25, 2017

@elasticmachine test this please

@javanna
Copy link
Member

javanna commented Oct 25, 2017

retest this please

@catalin-ursachi
Copy link
Contributor Author

No idea if merging in master fixed this (had it become incompatible with some external dependency?), or it was just a series of flukes; but, anyway.

@javanna
Copy link
Member

javanna commented Oct 25, 2017

@catalin-ursachi I looked at the failures too today and I am not sure what happened there, something broke badly but it was not certainly your fault, merging master in fixed it, happy that the build is green now, I will merge this in tomorrow.

@javanna javanna merged commit 8bf3324 into elastic:master Oct 26, 2017
@javanna
Copy link
Member

javanna commented Oct 26, 2017

Thanks a lot @catalin-ursachi , great job!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants