-
Notifications
You must be signed in to change notification settings - Fork 25.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add index block api #58094
Merged
Merged
Add index block api #58094
Changes from 21 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
ed6f7e5
Add API for adding index blocks
ywelsch 634bb38
Towards REST API
ywelsch 67511f8
Make metadatta block idempotent as well
ywelsch 877b465
more REST test changes
ywelsch f2c1934
Add docs
ywelsch 66be434
No HLRC yet
ywelsch 31dbfea
fix test
ywelsch 0a5c01d
Merge remote-tracking branch 'elastic/master' into add-index-block-api
ywelsch 0446032
chstyl
ywelsch 5ba6928
fix test
ywelsch a876465
Get basic stuff right
ywelsch 558c3b8
Merge remote-tracking branch 'elastic/master' into add-index-block-api
ywelsch db583d4
Use singular
ywelsch 97a6e93
Review comments
ywelsch 22e0195
Merge remote-tracking branch 'elastic/master' into add-index-block-api
ywelsch 85ab5be
checkstyleo
ywelsch 6d23195
change action name
ywelsch 27fa800
fix test
ywelsch 6953569
One more occurrence of close
ywelsch 39498f6
Fix docs CI
jrodewig a306955
Additional doc changes
jrodewig e20b9f7
Merge remote-tracking branch 'elastic/master' into add-index-block-api
ywelsch dff8526
Nits
ywelsch 1f9ccb0
Only for write blocks
ywelsch 0f39c54
API
ywelsch 0bf1c91
touched wrong class
ywelsch File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,146 @@ | ||
[[index-modules-blocks]] | ||
== Index blocks | ||
|
||
Index blocks limit the kind of operations that are available on a certain | ||
index. The blocks come in different flavours, allowing to block write, | ||
read, or metadata operations. The blocks can be set / removed using dynamic | ||
index settings, or can be added using a dedicated API, which also ensures | ||
that, once successfully returning to the user, all shards of the index are | ||
properly accounting for the block, for example that all in-flight writes to | ||
an index have been completed after adding the write block. | ||
|
||
[discrete] | ||
[[index-block-settings]] | ||
=== Index block settings | ||
|
||
The following _dynamic_ index settings determine the blocks present on an | ||
index: | ||
|
||
`index.blocks.read_only`:: | ||
|
||
Set to `true` to make the index and index metadata read only, `false` to | ||
allow writes and metadata changes. | ||
|
||
`index.blocks.read_only_allow_delete`:: | ||
|
||
Similar to `index.blocks.read_only`, but also allows deleting the index to | ||
make more resources available. The <<disk-based-shard-allocation,disk-based shard | ||
allocator>> may add and remove this block automatically. | ||
|
||
Deleting documents from an index to release resources - rather than deleting the index itself - can increase the index size over time. When `index.blocks.read_only_allow_delete` is set to `true`, deleting documents is not permitted. However, deleting the index itself releases the read-only index block and makes resources available almost immediately. | ||
|
||
IMPORTANT: {es} adds and removes the read-only index block automatically when the disk utilization falls below the high watermark, controlled by <<cluster-routing-flood_stage,cluster.routing.allocation.disk.watermark.flood_stage>>. | ||
|
||
`index.blocks.read`:: | ||
|
||
Set to `true` to disable read operations against the index. | ||
|
||
`index.blocks.write`:: | ||
|
||
Set to `true` to disable data write operations against the index. Unlike `read_only`, | ||
this setting does not affect metadata. For instance, you can close an index with a `write` | ||
block, but you cannot close an index with a `read_only` block. | ||
|
||
`index.blocks.metadata`:: | ||
|
||
Set to `true` to disable index metadata reads and writes. | ||
|
||
[discrete] | ||
[[add-index-block]] | ||
=== Add index block API | ||
|
||
Adds an index block to an index. | ||
|
||
[source,console] | ||
-------------------------------------------------- | ||
PUT /twitter/_block/write | ||
-------------------------------------------------- | ||
// TEST[setup:twitter] | ||
|
||
|
||
[discrete] | ||
[[add-index-block-api-request]] | ||
==== {api-request-title} | ||
|
||
`PUT /<index>/_block/<block>` | ||
|
||
|
||
[discrete] | ||
[role="child_attributes"] | ||
[[add-index-block-api-path-params]] | ||
==== {api-path-parms-title} | ||
|
||
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index] | ||
+ | ||
To add blocks to all indices, use `_all` or `*`. To disallow the adding | ||
of blocks to indices with `_all` or wildcard expressions, | ||
change the `action.destructive_requires_name` cluster setting to `true`. | ||
You can update this setting in the `elasticsearch.yml` file | ||
or using the <<cluster-update-settings,cluster update settings>> API. | ||
`<block>`:: | ||
(Required, string) | ||
Block type to add to the index. | ||
+ | ||
.Valid values for `<block>` | ||
[%collapsible%open] | ||
==== | ||
`metadata`:: | ||
Disable metadata changes, such as closing the index. | ||
|
||
`read`:: | ||
Disable read operations. | ||
|
||
`read_only`:: | ||
Disable write operations and metadata changes. | ||
|
||
`read_only_allow_delete`:: | ||
Disable write operations and metadata changes. | ||
Document deletion is disabled. | ||
However, index deletion is still allowed. | ||
|
||
`write`:: | ||
Disable write operations. However, metadata changes are still allowed. | ||
==== | ||
[discrete] | ||
[[add-index-block-api-query-params]] | ||
==== {api-query-parms-title} | ||
|
||
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices] | ||
+ | ||
Defaults to `true`. | ||
|
||
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards] | ||
+ | ||
Defaults to `open`. | ||
|
||
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable] | ||
|
||
include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=timeoutparms] | ||
|
||
[discrete] | ||
[[add-index-block-api-example]] | ||
==== {api-examples-title} | ||
|
||
The following example shows how to add an index block: | ||
|
||
[source,console] | ||
-------------------------------------------------- | ||
PUT /my_index/_block/write | ||
-------------------------------------------------- | ||
// TEST[s/^/PUT my_index\n/] | ||
|
||
The API returns following response: | ||
|
||
[source,console-result] | ||
-------------------------------------------------- | ||
{ | ||
"acknowledged" : true, | ||
"shards_acknowledged" : true, | ||
"indices" : { | ||
"my_index" : { | ||
"blocked" : true | ||
} | ||
} | ||
} | ||
-------------------------------------------------- | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
59 changes: 59 additions & 0 deletions
59
rest-api-spec/src/main/resources/rest-api-spec/api/indices.add_block.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
{ | ||
"indices.add_block":{ | ||
"documentation":{ | ||
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/indices-blocks.html", | ||
"description":"Adds a block to an index." | ||
}, | ||
"stability":"stable", | ||
"url":{ | ||
"paths":[ | ||
{ | ||
"path":"/{index}/_block/{block}", | ||
"methods":[ | ||
"PUT" | ||
], | ||
"parts":{ | ||
"index":{ | ||
"type":"list", | ||
"description":"A comma separated list of indices to add blocks to" | ||
ywelsch marked this conversation as resolved.
Show resolved
Hide resolved
|
||
}, | ||
"block":{ | ||
"type":"string", | ||
"description":"The block to add (one of read, write, read_only, metadata, read_only_allow_delete)" | ||
} | ||
} | ||
} | ||
] | ||
}, | ||
"params":{ | ||
"timeout":{ | ||
"type":"time", | ||
"description":"Explicit operation timeout" | ||
}, | ||
"master_timeout":{ | ||
"type":"time", | ||
"description":"Specify timeout for connection to master" | ||
}, | ||
"ignore_unavailable":{ | ||
"type":"boolean", | ||
"description":"Whether specified concrete indices should be ignored when unavailable (missing or closed)" | ||
}, | ||
"allow_no_indices":{ | ||
"type":"boolean", | ||
"description":"Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes `_all` string or when no indices have been specified)" | ||
}, | ||
"expand_wildcards":{ | ||
"type":"enum", | ||
"options":[ | ||
"open", | ||
"closed", | ||
"hidden", | ||
"none", | ||
"all" | ||
], | ||
"default":"open", | ||
"description":"Whether to expand wildcard expression to concrete indices that are open, closed or both." | ||
} | ||
} | ||
} | ||
} |
33 changes: 33 additions & 0 deletions
33
rest-api-spec/src/main/resources/rest-api-spec/test/indices.blocks/10_basic.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
--- | ||
"Basic test for index blocks": | ||
- skip: | ||
version: " - 7.99.99" | ||
reason: "index block APIs have only been made available in 8.0.0" | ||
- do: | ||
indices.create: | ||
index: test_index | ||
body: | ||
settings: | ||
number_of_replicas: 0 | ||
|
||
- do: | ||
indices.add_block: | ||
index: test_index | ||
block: write | ||
- is_true: acknowledged | ||
|
||
- do: | ||
catch: /cluster_block_exception/ | ||
index: | ||
index: test_index | ||
body: { foo: bar } | ||
|
||
- do: | ||
search: | ||
index: test_index | ||
|
||
- do: | ||
indices.put_settings: | ||
index: test_index | ||
body: | ||
index.blocks.write: false |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we call out that this only applies to write operations, not read operations?