Skip to content

Commit

Permalink
[DOCS] Move search pagination content to one page (#60515)
Browse files Browse the repository at this point in the history
  • Loading branch information
jrodewig authored Jul 31, 2020
1 parent 6bc1975 commit aec26b1
Show file tree
Hide file tree
Showing 21 changed files with 78 additions and 48 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ A multi-bucket aggregation that creates composite buckets from different sources
Unlike the other `multi-bucket` aggregation the `composite` aggregation can be used
to paginate **all** buckets from a multi-level aggregation efficiently. This aggregation
provides a way to stream **all** buckets of a specific aggregation similarly to what
<<request-body-search-scroll, scroll>> does for documents.
<<scroll-search-results, scroll>> does for documents.

The composite buckets are built from the combinations of the
values extracted/created for each document and each combination is considered as
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/docs/delete-by-query.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=routing]
`scroll`::
(Optional, <<time-units,time value>>)
Period to retain the <<scroll-search-context,search context>> for scrolling. See
<<request-body-search-scroll>>.
<<scroll-search-results>>.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=scroll_size]

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/docs/update-by-query.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=routing]
`scroll`::
(Optional, <<time-units,time value>>)
Period to retain the <<scroll-search-context,search context>> for scrolling. See
<<request-body-search-scroll>>.
<<scroll-search-results>>.

include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=scroll_size]

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/how-to/general.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Elasticsearch is designed as a search engine, which makes it very good at
getting back the top documents that match a query. However, it is not as good
for workloads that fall into the database domain, such as retrieving all
documents that match a particular query. If you need to do this, make sure to
use the <<request-body-search-scroll,Scroll>> API.
use the <<scroll-search-results,Scroll>> API.

[discrete]
[[maximum-document-size]]
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/index-modules.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ specific index module:
The maximum value of `from + size` for searches to this index. Defaults to
`10000`. Search requests take heap memory and time proportional to
`from + size` and this limits that memory. See
<<request-body-search-scroll,Scroll>> or <<request-body-search-search-after,Search After>> for a more efficient alternative
<<scroll-search-results,Scroll>> or <<search-after,Search After>> for a more efficient alternative
to raising this.

`index.max_inner_result_window`::
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/modules/cross-cluster-search.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ handling network delays:
By default, {es} reduces the number of network roundtrips between remote
clusters. This reduces the impact of network delays on search speed. However,
{es} can't reduce network roundtrips for large search requests, such as those
including a <<request-body-search-scroll, scroll>> or
including a <<scroll-search-results, scroll>> or
<<request-body-search-inner-hits,inner hits>>.
+
See <<ccs-min-roundtrips>> to learn how this option works.
Expand Down
16 changes: 13 additions & 3 deletions docs/reference/redirects.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -117,11 +117,11 @@ See <<request-body-search-script-fields>>.

[role="exclude",id="search-request-scroll"]
=== Scroll parameter for request body search API
See <<request-body-search-scroll>>.
See <<scroll-search-results>>.

[role="exclude",id="search-request-search-after"]
=== Search after parameter for request body search API
See <<request-body-search-search-after>>.
See <<search-after>>.

[role="exclude",id="search-request-search-type"]
=== Search type parameter for request body search API
Expand Down Expand Up @@ -874,7 +874,7 @@ See <<settings>>.
[role="exclude",id="scan-scroll"]
=== Scan and scroll

See <<request-body-search-scroll>>.
See <<scroll-search-results>>.

[role="exclude",id="mapping-dynamic-mapping"]
=== Dynamic mapping
Expand Down Expand Up @@ -1001,6 +1001,16 @@ See <<highlighting>>.
See <<how-es-highlighters-work-internally>>.
[role="exclude",id="request-body-search-scroll"]
==== Scroll
See <<scroll-search-results>>.
[role="exclude",id="request-body-search-search-after"]
==== Search After
See <<search-after>>.
[role="exclude",id="request-body-search-sort"]
==== Sort
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/search/async-search.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ query get skipped.
* `ccs_minimize_roundtrips` defaults to `false`, which is also the only
supported value

WARNING: Async search does not support <<request-body-search-scroll,scroll>>
WARNING: Async search does not support <<scroll-search-results,scroll>>
nor search requests that only include the <<search-suggesters,suggest section>>.
{ccs} is supported only with <<ccs-min-roundtrips,`ccs_minimize_roundtrips`>>
set to `false`.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/search/clear-scroll-api.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
++++

Clears the search context and results for a
<<request-body-search-scroll,scrolling search>>.
<<scroll-search-results,scrolling search>>.

////
[source,console]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
[discrete]
[[paginate-search-results]]
=== Paginate search results
== Paginate search results

By default, the search API returns the top 10 matching documents.
By default, the <<search-search,search API>> returns the top 10 matching documents.

To paginate through a larger set of results, you can use the search API's `size`
and `from` parameters. The `size` parameter is the number of matching documents
Expand Down Expand Up @@ -40,10 +39,14 @@ results must then be combined and sorted to ensure that the overall sort order
is correct.

As an alternative to deep paging, we recommend using
<<request-body-search-scroll,scroll>> or the
<<request-body-search-search-after,`search_after`>> parameter.
<<scroll-search-results,scroll>> or the
<<search-after,`search_after`>> parameter.

WARNING: {es} uses Lucene's internal doc IDs as tie-breakers. These internal
doc IDs can be completely different across replicas of the same
data. When paginating, you might occasionally see that documents with the same
sort values are not ordered consistently.

include::request/scroll.asciidoc[]

include::request/search-after.asciidoc[]
15 changes: 13 additions & 2 deletions docs/reference/search/request-body.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -139,9 +139,20 @@ include::request/rescore.asciidoc[]

include::request/script-fields.asciidoc[]

include::request/scroll.asciidoc[]
[[request-body-search-scroll]]
==== Scroll

include::request/search-after.asciidoc[]
See <<scroll-search-results>>.

[[_clear_scroll_api]]
===== Clear scroll API

See <<clear-scroll-api>>.

[[request-body-search-search-after]]
==== Search After

See <<search-after>>.

include::request/search-type.asciidoc[]

Expand Down
4 changes: 2 additions & 2 deletions docs/reference/search/request/collapse.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,8 @@ The `max_concurrent_group_searches` request parameter can be used to control
the maximum number of concurrent searches allowed in this phase.
The default is based on the number of data nodes and the default search thread pool size.

WARNING: `collapse` cannot be used in conjunction with <<request-body-search-scroll, scroll>>,
<<request-body-search-rescore, rescore>> or <<request-body-search-search-after, search after>>.
WARNING: `collapse` cannot be used in conjunction with <<scroll-search-results, scroll>>,
<<request-body-search-rescore, rescore>> or <<search-after, search after>>.

[discrete]
[[second-level-of-collapsing]]
Expand Down
15 changes: 10 additions & 5 deletions docs/reference/search/request/scroll.asciidoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
[[request-body-search-scroll]]
==== Scroll
[discrete]
[[scroll-search-results]]
=== Scroll search results

While a `search` request returns a single ``page'' of results, the `scroll`
API can be used to retrieve large numbers of results (or even all results)
Expand Down Expand Up @@ -102,8 +103,9 @@ GET /_search?scroll=1m
--------------------------------------------------
// TEST[setup:twitter]

[discrete]
[[scroll-search-context]]
===== Keeping the search context alive
==== Keeping the search context alive

A scroll returns all the documents which matched the search at the time of the
initial search request. It ignores any subsequent changes to these documents.
Expand Down Expand Up @@ -149,7 +151,9 @@ You can check how many search contexts are open with the
GET /_nodes/stats/indices/search
---------------------------------------

===== Clear scroll API
[discrete]
[[clear-scroll]]
==== Clear scroll

Search context are automatically removed when the `scroll` timeout has been
exceeded. However keeping scrolls open has a cost, as discussed in the
Expand Down Expand Up @@ -196,8 +200,9 @@ DELETE /_search/scroll/DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMN
---------------------------------------
// TEST[catch:missing]

[discrete]
[[sliced-scroll]]
===== Sliced Scroll
==== Sliced Scroll

For scroll queries that return a lot of documents it is possible to split the scroll in multiple slices which
can be consumed independently:
Expand Down
7 changes: 4 additions & 3 deletions docs/reference/search/request/search-after.asciidoc
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
[[request-body-search-search-after]]
==== Search After
[discrete]
[[search-after]]
=== Search after

Pagination of results can be done by using the `from` and `size` but the cost becomes prohibitive when the deep pagination is reached.
The `index.max_result_window` which defaults to 10,000 is a safeguard, search requests take heap memory and time proportional to `from + size`.
The <<request-body-search-scroll,Scroll>> api is recommended for efficient deep scrolling but scroll contexts are costly and it is not
The <<scroll-search-results,scroll>> API is recommended for efficient deep scrolling but scroll contexts are costly and it is not
recommended to use it for real time user requests.
The `search_after` parameter circumvents this problem by providing a live cursor.
The idea is to use the results from the previous page to help the retrieval of the next page.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/search/request/sort.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ GET /my-index-000001/_search

NOTE: `_doc` has no real use-case besides being the most efficient sort order.
So if you don't care about the order in which documents are returned, then you
should sort by `_doc`. This especially helps when <<request-body-search-scroll,scrolling>>.
should sort by `_doc`. This especially helps when <<scroll-search-results,scrolling>>.

[discrete]
=== Sort Values
Expand Down
10 changes: 5 additions & 5 deletions docs/reference/search/scroll-api.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<titleabbrev>Scroll</titleabbrev>
++++

Retrieves the next batch of results for a <<request-body-search-scroll,scrolling
Retrieves the next batch of results for a <<scroll-search-results,scrolling
search>>.

////
Expand Down Expand Up @@ -48,7 +48,7 @@ deprecated:[7.0.0]
==== {api-description-title}

You can use the scroll API to retrieve large sets of results from a single
<<request-body-search-scroll,scrolling search>> request.
<<scroll-search-results,scrolling search>> request.

The scroll API requires a scroll ID. To get a scroll ID, submit a
<<search-search,search API>> request that includes an argument for the
Expand All @@ -63,7 +63,7 @@ next batch of results for the request.
You can also use the scroll API to specify a new `scroll` parameter that extends
or shortens the retention period for the search context.

See <<request-body-search-scroll>>.
See <<scroll-search-results>>.

IMPORTANT: Results from a scrolling search reflect the state of the index at the
time of the initial search request. Subsequent indexing or document changes only
Expand All @@ -86,7 +86,7 @@ the <<scroll-api-scroll-id-param,`scroll_id` request body parameter>>.
`scroll`::
(Optional, <<time-units,time value>>)
Period to retain the <<scroll-search-context,search context>> for scrolling. See
<<request-body-search-scroll>>.
<<scroll-search-results>>.
+
This value overrides the duration set by the original search API request's
`scroll` parameter.
Expand Down Expand Up @@ -118,7 +118,7 @@ Defaults to `false`.
`scroll`::
(Optional, <<time-units,time value>>)
Period to retain the <<scroll-search-context,search context>> for scrolling. See
<<request-body-search-scroll>>.
<<scroll-search-results>>.
+
This value overrides the duration set by the original search API request's
`scroll` parameter.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/search/search-your-data.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -290,10 +290,10 @@ GET /*/_search
}
----

include::request/from-size.asciidoc[]
include::search-fields.asciidoc[]
include::request/collapse.asciidoc[]
include::request/highlighting.asciidoc[]
include::paginate-search-results.asciidoc[]
include::request/sort.asciidoc[]
include::{es-repo-dir}/async-search.asciidoc[]
include::{es-repo-dir}/modules/cross-cluster-search.asciidoc[]
Expand Down
20 changes: 10 additions & 10 deletions docs/reference/search/search.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -101,8 +101,8 @@ results must then be combined and sorted to ensure that the overall order is
correct.

As an alternative to deep paging, we recommend using
<<request-body-search-scroll,scroll>> or the
<<request-body-search-search-after,`search_after`>> parameter.
<<scroll-search-results,scroll>> or the
<<search-after,`search_after`>> parameter.
--

`ignore_throttled`::
Expand Down Expand Up @@ -159,7 +159,7 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=routing]
`scroll`::
(Optional, <<time-units,time value>>)
Period to retain the <<scroll-search-context,search context>> for scrolling. See
<<request-body-search-scroll>>.
<<scroll-search-results>>.
+
By default, this value cannot exceed `1d` (24 hours). You can change
this limit using the `search.max_keep_alive` cluster-level setting.
Expand All @@ -185,8 +185,8 @@ results must then be combined and sorted to ensure that the overall order is
correct.

As an alternative to deep paging, we recommend using
<<request-body-search-scroll,scroll>> or the
<<request-body-search-search-after,`search_after`>> parameter.
<<scroll-search-results,scroll>> or the
<<search-after,`search_after`>> parameter.

If the <<search-api-scroll-query-param,`scroll` parameter>> is specified, this
value cannot be `0`.
Expand Down Expand Up @@ -322,8 +322,8 @@ results must then be combined and sorted to ensure that the overall order is
correct.

As an alternative to deep paging, we recommend using
<<request-body-search-scroll,scroll>> or the
<<request-body-search-search-after,`search_after`>> parameter.
<<scroll-search-results,scroll>> or the
<<search-after,`search_after`>> parameter.
--

[[request-body-search-query]]
Expand Down Expand Up @@ -351,8 +351,8 @@ results must then be combined and sorted to ensure that the overall order is
correct.

As an alternative to deep paging, we recommend using
<<request-body-search-scroll,scroll>> or the
<<request-body-search-search-after,`search_after`>> parameter.
<<scroll-search-results,scroll>> or the
<<search-after,`search_after`>> parameter.

If the <<search-api-scroll-query-param,`scroll` parameter>> is specified, this
value cannot be `0`.
Expand Down Expand Up @@ -428,7 +428,7 @@ Identifier for the search and its <<scroll-search-context,search context>>.
+
You can use this scroll ID with the <<scroll-api,scroll API>> to retrieve the
next batch of search results for the request. See
<<request-body-search-scroll>>.
<<scroll-search-results>>.
+
This parameter is only returned if the <<search-api-scroll-query-param,`scroll`
query parameter>> is specified in the request.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/sql/endpoints/rest.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@ Which looks like:
Note that the `columns` object is only part of the first page.

You've reached the last page when there is no `cursor` returned
in the results. Like Elasticsearch's <<request-body-search-scroll,scroll>>,
in the results. Like Elasticsearch's <<scroll-search-results,scroll>>,
SQL may keep state in Elasticsearch to support the cursor. Unlike
scroll, receiving the last page is enough to guarantee that the
Elasticsearch state is cleared.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/sql/endpoints/translate.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ Which returns:
--------------------------------------------------

Which is the request that SQL will run to provide the results.
In this case, SQL will use the <<request-body-search-scroll,scroll>>
In this case, SQL will use the <<scroll-search-results,scroll>>
API. If the result contained an aggregation then SQL would use
the normal <<search-request-body,search>> API.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"clear_scroll":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/search-request-body.html#_clear_scroll_api",
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/current/clear-scroll-api.html",
"description":"Explicitly clears the search context for a scroll."
},
"stability":"stable",
Expand Down

0 comments on commit aec26b1

Please sign in to comment.