Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Fix outdated heap references #66646

Merged
merged 5 commits into from
Dec 21, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 14 additions & 8 deletions docs/reference/how-to/size-your-shards.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ set of large shards uses fewer resources than many small shards.

Segments play a big role in a shard's resource usage. Most shards contain
several segments, which store its index data. {es} keeps segment metadata in
<<heap-size,heap memory>> so it can be quickly retrieved for searches. As a
JVM heap memory so it can be quickly retrieved for searches. As a
shard grows, its segments are <<index-modules-merge,merged>> into fewer, larger
segments. This decreases the number of segments, which means less metadata is
kept in heap memory.
Expand Down Expand Up @@ -141,22 +141,28 @@ and may tax node resources.
==== Aim for 20 shards or fewer per GB of heap memory

The number of shards a node can hold is proportional to the node's
<<heap-size,heap memory>>. For example, a node with 30GB of heap memory should
heap memory. For example, a node with 30GB of heap memory should
have at most 600 shards. The further below this limit you can keep your nodes,
the better. If you find your nodes exceeding more than 20 shards per GB,
consider adding another node. You can use the <<cat-shards,cat shards API>> to
check the number of shards per node.
consider adding another node.

To check the current size of each node's heap, use the <<cat-nodes,cat nodes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this 👍

API>>.

[source,console]
----
GET _cat/shards
GET _cat/nodes?v&h=heap.current
----
// TEST[setup:my_index]

To use compressed pointers and save memory, we
recommend each node have a maximum heap size of 32GB or 50% of the node's
available memory, whichever is lower. See <<heap-size>>.
You can use the <<cat-shards,cat shards API>> to check the number of shards per
node.

[source,console]
----
GET _cat/shards
----
// TEST[setup:my_index]

[discrete]
[[avoid-node-hotspots]]
Expand Down
17 changes: 8 additions & 9 deletions docs/reference/setup/bootstrap-checks.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -64,15 +64,14 @@ force execution of the bootstrap checks independent of the node configuration.

=== Heap size check

If a JVM is started with unequal initial and max heap size, it can be
prone to pauses as the JVM heap is resized during system usage. To avoid
these resize pauses, it's best to start the JVM with the initial heap
size equal to the maximum heap size. Additionally, if
<<bootstrap-memory_lock,`bootstrap.memory_lock`>> is enabled, the JVM
will lock the initial size of the heap on startup. If the initial heap
size is not equal to the maximum heap size, after a resize it will not
be the case that all of the JVM heap is locked in memory. To pass the
heap size check, you must configure the <<heap-size,heap size>>.
By default, {es} automatically sizes JVM heap based on a node's
<<node-roles,roles>> and total memory. If you manually override the default
sizing and start the JVM with different initial and max heap sizes, the JVM may
pause as it resizes the heap during system usage. If you enable
<<bootstrap-memory_lock,`bootstrap.memory_lock`>>, the JVM locks the initial heap
size on startup. If the initial heap size is not equal to the maximum heap size,
some JVM heap may not be locked after a resize. To avoid these issues, start the
JVM with an initial heap size equal to the maximum heap size.

=== File descriptor check

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/setup/important-settings.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ of items which *must* be considered before using your cluster in production:
* <<node-name,Node name setting>>
* <<network.host,Network host settings>>
* <<discovery-settings,Discovery settings>>
* <<heap-size,Heap size settings>>
* <<heap-size-settings,Heap size settings>>
* <<heap-dump-path,JVM heap dump path setting>>
* <<gc-logging,GC logging settings>>
* <<es-tmpdir,Temporary directory settings>>
Expand Down
6 changes: 3 additions & 3 deletions docs/reference/setup/important-settings/heap-size.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ production environments. If needed, you can override default sizing by manually
NOTE: Automatic heap sizing requires the <<jvm-version,bundled JDK>> or, if using
a custom JRE location, a Java 14 or later JRE.

NOTE: When running in a container, such as <<docker>>, total memory is defined as
the amount of memory visible to the container, not the total system memory on
the host.
NOTE: When running in a container, such as <<docker,Docker>>, total memory is
defined as the amount of memory visible to the container, not the total system
memory on the host.
39 changes: 21 additions & 18 deletions docs/reference/setup/install/docker.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ include::docker-compose.yml[]
--------------------------------------------
endif::[]

NOTE: This sample `docker-compose.yml` file uses the `ES_JAVA_OPTS`
environment variable to manually set the heap size to 512MB. We do not recommend
using `ES_JAVA_OPTS` in production. See <<docker-set-heap-size>>.

This sample Docker Compose file brings up a three-node {es} cluster.
Node `es01` listens on `localhost:9200` and `es02` and `es03` talk to `es01` over a Docker network.

Expand Down Expand Up @@ -286,24 +290,23 @@ published ports with `--publish-all` is recommended,
unless you are pinning one container per host.

[[docker-set-heap-size]]
===== Set the heap size

To configure the heap size, you can bind mount a <<jvm-options,JVM options>>
file under `/usr/share/elasticsearch/config/jvm.options.d` that includes your
desired <<heap-size,heap size>> settings. Note that while the default root
`jvm.options` file sets a default heap of 1 GB, any value you set in a
bind-mounted JVM options file will override it.

While setting the heap size via bind-mounted JVM options is the recommended
method, you can also configure this by using the `ES_JAVA_OPTS` environment
variable to set the heap size. For example, to use 16 GB, specify
`-e ES_JAVA_OPTS="-Xms16g -Xmx16g"` with `docker run`. Note that while the
default root `jvm.options` file sets a default heap of 1 GB, any value you set
in `ES_JAVA_OPTS` will override it. The `docker-compose.yml` file above sets the heap size to 512 MB.

IMPORTANT: You must <<heap-size,configure the heap size>> even if you are
https://docs.docker.com/config/containers/resource_constraints/#limit-a-containers-access-to-memory[limiting
memory access] to the container.
===== Manually set the heap size

By default, {es} automatically sizes JVM heap based on a nodes's
<<node-roles,roles>> and the total memory available to the node's container. We
recommend this default sizing for most production environments. If needed, you
can override default sizing by manually setting JVM heap size.

To manually set the heap size in production, bind mount a <<jvm-options,JVM
options>> file under `/usr/share/elasticsearch/config/jvm.options.d` that
includes your desired <<setting-jvm-heap-size,heap size>> settings.

For testing, you can also manually set the heap size using the `ES_JAVA_OPTS`
environment variable. For example, to use 16GB, specify `-e
ES_JAVA_OPTS="-Xms16g -Xmx16g"` with `docker run`. The `ES_JAVA_OPTS` variable
overrides all other JVM options. The `ES_JAVA_OPTS` variable overrides all other
JVM options. We do not recommend using `ES_JAVA_OPTS` in production. The
`docker-compose.yml` file above sets the heap size to 512MB.

===== Pin deployments to a specific image version

Expand Down
15 changes: 11 additions & 4 deletions docs/reference/setup/install/zip-windows.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -203,11 +203,18 @@ The Elasticsearch service can be configured prior to installation by setting the
NOTE: At its core, `elasticsearch-service.bat` relies on https://commons.apache.org/proper/commons-daemon/[Apache Commons Daemon] project
to install the service. Environment variables set prior to the service installation are copied and will be used during the service lifecycle. This means any changes made to them after the installation will not be picked up unless the service is reinstalled.

NOTE: On Windows, the <<heap-size,heap size>> can be configured as for
any other Elasticsearch installation when running Elasticsearch from the
command line, or when installing Elasticsearch as a service for the
first time. To adjust the heap size for an already installed service,
[NOTE]
====
By default, {es} automatically sizes JVM heap based on a node's
<<node-roles,roles>> and total memory. We recommend this default sizing for most
production environments. If needed, you can override default sizing by manually
setting the heap size.

When installing {es} on Windows as a service for the first time or running {es}
from the command line, you can manually set the heap size as described in
<<setting-jvm-heap-size>>. To resize the heap for an already installed service,
use the service manager: `bin\elasticsearch-service.bat manager`.
====

NOTE: The service automatically configures a private temporary directory for use
by Elasticsearch when it is running. This private temporary directory is
Expand Down
8 changes: 4 additions & 4 deletions docs/reference/setup/jvm-options.asciidoc
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
[[jvm-options]]
=== Setting JVM options

You should rarely need to change Java Virtual Machine (JVM) options. If you do,
the most likely change is setting the <<heap-size,heap size>>. The remainder of
this document explains in detail how to set JVM options. You can set options
either with `jvm.options` files or with the `ES_JAVA_OPTS` environment variable.
You should rarely need to change Java Virtual Machine (JVM) options. {es}
includes default JVM options that work well for most production environments. If
needed, you can override these default options using `jvm.options` files or the
`ES_JAVA_OPTS` environment variable.

The preferred method of setting or overriding JVM options is via JVM options
files. When installing from the tar or zip distributions, the root `jvm.options`
Expand Down