Skip to content

Commit

Permalink
Update indexing speed recommendations around the refresh interval. (#…
Browse files Browse the repository at this point in the history
…40690)

We now need to update recommendations now that we have introduced the concept
of "search idle" shards.
  • Loading branch information
jpountz committed Apr 2, 2019
1 parent 37758bb commit ba9abc5
Showing 1 changed file with 18 additions and 6 deletions.
24 changes: 18 additions & 6 deletions docs/reference/how-to/indexing-speed.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,24 @@ number of workers is. This can be tested by progressively increasing the
number of workers until either I/O or CPU is saturated on the cluster.

[float]
=== Increase the refresh interval

The default <<dynamic-index-settings,`index.refresh_interval`>> is `1s`, which
forces Elasticsearch to create a new segment every second.
Increasing this value (to say, `30s`) will allow larger segments to flush and
decreases future merge pressure.
=== Unset or increase the refresh interval

The operation that consists of making changes visible to search - called a
<<indices-refresh,refresh>> - is costly, and calling it often while there is
ongoing indexing activity can hurt indexing speed.

By default, Elasticsearch runs this operation every second, but only on
indices that have received one search request or more in the last 30 seconds.
This is the optimal configuration if you have no or very little search traffic
(e.g. less than one search request every 5 minutes) and want to optimize for
indexing speed.

On the other hand, if your index experiences regular search requests, this
default behavior means that Elasticsearch will refresh your index every 1
second. If you can afford to increase the amount of time between when a document
gets indexed and when it becomes visible, increasing the
<<dynamic-index-settings,`index.refresh_interval`>> to a larger value, e.g.
`30s`, might help improve indexing speed.

[float]
=== Disable refresh and replicas for initial loads
Expand Down

0 comments on commit ba9abc5

Please sign in to comment.