Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate REST access to System Indices #60945

Merged
merged 195 commits into from
Oct 6, 2020

Conversation

gwbrown
Copy link
Contributor

@gwbrown gwbrown commented Aug 11, 2020

This PR adds deprecation warnings when accessing System Indices via the REST layer. At this time, these warnings are only enabled for Snapshot builds by default, to allow projects external to Elasticsearch additional time to adjust their access patterns.

Deprecation warnings will be triggered by all REST requests which access registered System Indices, except for purpose-specific APIs which access System Indices as an implementation detail and a few specific APIs which will continue to allow access to system indices by default:

  • GET _cluster/health
  • GET {index}/_recovery
  • GET _cluster/allocation/explain
  • GET _cluster/state
  • POST _cluster/reroute
  • GET {index}/_stats
  • GET {index}/_segments
  • GET {index}/_shard_stores
  • GET _cat/[indices,aliases,health,recovery,shards,segments]

Deprecation warnings for accessing system indices take the form:

this request accesses system indices: [.some_system_index], but in a future major version, direct access to system indices will be prevented by default

gwbrown added 28 commits August 10, 2020 19:17
@gwbrown gwbrown added >enhancement :Core/Infra/Core Core issues without another label labels Aug 11, 2020
@gwbrown gwbrown requested a review from jaymode October 6, 2020 03:45
Copy link
Member

@jaymode jaymode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few minor suggestions and one question. Otherwise LGTM


public class TransportGetAliasesAction extends TransportMasterNodeReadAction<GetAliasesRequest, GetAliasesResponse> {
private static DeprecationLogger deprecationLogger = DeprecationLogger.getLogger(TransportGetAliasesAction.class);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
private static DeprecationLogger deprecationLogger = DeprecationLogger.getLogger(TransportGetAliasesAction.class);
private static final DeprecationLogger deprecationLogger = DeprecationLogger.getLogger(TransportGetAliasesAction.class);

@@ -103,8 +107,13 @@ public void setup() {
httpServerTransport.start();
}

@After
public void teardown() {
client.close();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
client.close();
IOUtils.close(client);

This allows us to guard in case the client is null to avoid unnecessary exceptions in case of odd setup failure. The IOUtils class I am referencing is from package org.elasticsearch.core.internal.io

@@ -798,7 +812,17 @@ private void wipeRollupJobs() throws IOException {
protected void refreshAllIndices() throws IOException {
boolean includeHidden = minimumNodeVersion().onOrAfter(Version.V_7_7_0);
Request refreshRequest = new Request("POST", "/_refresh");
refreshRequest.addParameter("expand_wildcards", "open,closed" + (includeHidden ? ",hidden" : ""));
refreshRequest.addParameter("expand_wildcards", "open" + (includeHidden ? ",hidden" : ""));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did closed get dropped here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to refresh a closed index appears to cause an exception, but that only came up after I switched a test which involves closing an index to use this instead of a direct POST _refresh call (to take advantage of the deprecation warning handling). This method is only used in a few tests, and I think specifying closed was always wrong, it just never mattered before.

@gwbrown gwbrown merged commit 91f4b58 into elastic:master Oct 6, 2020
@gwbrown gwbrown deleted the si/protection-header branch October 6, 2020 17:13
gwbrown added a commit that referenced this pull request Oct 6, 2020
This PR adds deprecation warnings when accessing System Indices via the REST layer. At this time, these warnings are only enabled for Snapshot builds by default, to allow projects external to Elasticsearch additional time to adjust their access patterns.

Deprecation warnings will be triggered by all REST requests which access registered System Indices, except for purpose-specific APIs which access System Indices as an implementation detail a few specific APIs which will continue to allow access to system indices by default:

- `GET _cluster/health`
- `GET {index}/_recovery`
- `GET _cluster/allocation/explain`
- `GET _cluster/state`
- `POST _cluster/reroute`
- `GET {index}/_stats`
- `GET {index}/_segments`
- `GET {index}/_shard_stores`
- `GET _cat/[indices,aliases,health,recovery,shards,segments]`

Deprecation warnings for accessing system indices take the form:
```
this request accesses system indices: [.some_system_index], but in a future major version, direct access to system indices will be prevented by default
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants