Introduce Nodes APIs

In this commit we are introducing a new section in REST API Reference for Nodes APIs. We are adding documentation for "Nodes hot threads" and "Nodes info". Because Nodes APIs share common request parameters we also add more detailed section into parent index page (notice that the nodes-filter parameter was moved from "Cluster stats" page and was improved, and relevant links were updated). There are more Nodes APIs that still need to be documented, this is just a good start. We updated nav_order parameter for some pages as well. Relevant ticket: opensearch-project#424 Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
lukas-vlcek · Apr 22, 2022 · 2df0f84 · 2df0f84
1 parent cb02f71
commit 2df0f84
Show file tree

Hide file tree

Showing 5 changed files with 306 additions and 2 deletions.
diff --git a/_opensearch/rest-api/cluster-stats.md b/_opensearch/rest-api/cluster-stats.md
@@ -31,7 +31,7 @@ All cluster stats parameters are optional.
 
 Parameter | Type | Description
 :--- | :--- | :---
-&lt;node-filters&gt; | List | A comma-separated list of node-filters that OpenSearch uses to filter results. Available options are `all`, `_local`, `_master`, a node name or ID, `master:true`, `master:false`, `data:true`, `data:false`, `ingest:true`, `ingest:false`, `voting_only:true`, `voting_only:false`, `ml:true`, `ml:false`, `coordinating_only:true`, `coordinating_only:false`, and &lt;custom node attributes&gt; : &lt;attribute values&gt; pairs.
+&lt;node-filters&gt; | List | A comma-separated list of [node-filters](../nodes-apis/index/#node-filters) that OpenSearch uses to filter results.
 
 
 ## Response

diff --git a/_opensearch/rest-api/ingest-apis/index.md b/_opensearch/rest-api/ingest-apis/index.md
@@ -3,7 +3,7 @@ layout: default
 title: Ingest APIs
 parent: REST API reference
 has_children: true
-nav_order: 3
+nav_order: 4
 redirect_from:
   - /opensearch/rest-api/ingest-apis/
 ---

diff --git a/_opensearch/rest-api/nodes-apis/index.md b/_opensearch/rest-api/nodes-apis/index.md
@@ -0,0 +1,72 @@
+---
+layout: default
+title: Nodes APIs
+parent: REST API reference
+has_children: true
+nav_order: 5
+---
+
+# Nodes APIs
+
+The nodes API makes it possible to retrieve information about individual cluster nodes.
+
+---
+
+Many nodes APIs support common parameters like `{timeout}`, and `{node-filters}`.
+
+## Timeout
+
+The `{timeout}` parameter can be used to change the default time limit for node response.
+
+Parameter | Type      | Description
+:--- |:----------| :---
+`{timeout}` | TimeValue | Default value is `30s`.
+
+## Node filters
+
+The `{node-filters}` parameter can be used to filter target set of nodes that will be included in the response.
+
+Parameter | Type   | Description
+:--- |:-------| :---
+<code><nobr>{node-filters} | String | A comma-separated list of resolution mechanisms that OpenSearch uses to identify cluster nodes.
+
+Node filters support several node resolution mechanisms:
+
+- pre-defined constants: `_local`, `_cluster_manager` (or deprecated `_master`) or `_all`
+- exact match for `nodeID`
+- a simple case-sensitive wildcard pattern matching for: `node-name`, `host-name` or `host-IP-address`
+- node roles (where `<bool>` value is set either to `true` or `false`):
+  - `cluster_manager:<bool>` (or deprecated `master:<bool>`)
+  - `data:<bool>`
+  - `ingest:<bool>`
+  - `voting_only:<bool>`
+  - `ml:<bool>`
+  - `coordinating_only:<bool>`
+- a simple case-sensitive wildcard pattern matching for node attributes: <br>`<node attribute*>:<attribute value*>` (the wildcard matching pattern can be used in both the key and value at the same time)
+
+The resolution mechanisms are applied sequentially in the order specified by the client and each mechanism specification may either add or remove nodes.
+
+### Example
+
+Get statistics from elected cluster-manager node only:
+```text
+GET /_nodes/_cluster_manager/stats
+```
+
+Get statistics from nodes that are data-only nodes:
+```text
+GET /_nodes/data:true/stats
+```
+#### Order of resolution mechanisms matters
+
+The order of resolution mechanisms is applied sequentially and each can add or remove nodes, this means that the following two examples yield different results.
+
+Get statistics from all the nodes but the cluster-manager node:
+```text
+GET /_nodes/_all,cluster_namager:false/stats
+```
+
+However, if we switch the resolution mechanisms then the result will include all the cluster nodes including the cluster manager node. 
+```text
+GET /_nodes/cluster_namager:false,_all/stats
+```
diff --git a/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md b/_opensearch/rest-api/nodes-apis/nodes-hot-threads.md
@@ -0,0 +1,101 @@
+---
+layout: default
+title: Nodes hot threads
+parent: Nodes APIs
+grand_parent: REST API reference
+nav_order: 10
+---
+
+# Nodes hot threads
+
+This REST API provide information about busy JVM threads for selected cluster nodes.
+It provides a unique view of what activity each node spends time on.
+
+## Example
+
+```json
+GET /_nodes/hot_threads
+```
+
+## Path and HTTP methods
+
+```text
+GET /_nodes/hot_threads
+GET /_nodes/{nodeId}/hot_threads
+```
+
+## URL parameters
+
+You can include the following URL parameters in your request. All parameters are optional.
+
+Parameter | Type      | Description
+:--- |:----------| :---
+nodeId | String    | A comma-separated list of nodeIds to filter results. Supports [node filters](../index/#node-filters).<br>Defaults to `_all`.
+snapshots | Integer   | Number of samples of thread stacktraces.<br>Defaults to `10`.
+interval | TimeValue | Interval between consecutive samples.<br>Defaults to `500ms`.
+threads | Integer   | A number of top bussiest threads to return information about. Defaults to `3`.
+ignore_idle_threads | Boolean   | Don’t show threads that are in known-idle states, such as waiting on a socket select or pulling from an empty task queue.<br>Defaults to `true`.
+type | String    | Supported thread types are `cpu`, `wait`, or `block`.<br>Defaults to `cpu`.
+timeout | TimeValue | A request [timeout](../index/#timeout).<br>Defaults to `30s`.
+
+## Response
+
+Unlike majority of OpenSearch responses this response is in text format.
+It consists of one section per each cluster node included in the response.
+
+Each section starts with a single line containing the following segments:
+
+Line segment | Description
+:--- |:-------
+<code>:::&nbsp;</code>  | Line start (a distinct visual symbol).
+`{global-eu-35}` | Node name.
+`{uFPbKLDOTlOmdnwUlKW8sw}` | NodeId.
+`{OAM8OT5CQAyasWuIDeVyUA}` | EphemeralId.
+`{global-eu-35.local}` | Host name.
+`{[gdv2:a284:2acv:5fa6:0:3a2:7260:74cf]:9300}` | Host address.
+`{dimr}` | Node roles (d=data, i=ingest, m=cluster&nbsp;manager, r=remote&nbsp;cluster&nbsp;client).
+`{zone=west-a2, shard_indexing_pressure_enabled=true}` | Node attributes.
+
+Then follows information about threads of selected type.
+
+```text
+::: {global-eu-35}{uFPbKLDOTlOmdnwUlKW8sw}{OAM8OT5CQAyasWuIDeVyUA}{global-eu-35.local}{[gdv2:a284:2acv:5fa6:0:3a2:7260:74cf]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true}
+   Hot threads at 2022-04-01T15:15:27.658Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
+   
+    0.1% (645micros out of 500ms) cpu usage by thread 'opensearch[global-eu-35][transport_worker][T#7]'
+     4/10 snapshots sharing following 3 elements
+       io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
+       io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
+       java.base@11.0.14.1/java.lang.Thread.run(Thread.java:829)
+::: {global-eu-62}{4knOxAdERlOB19zLQIT1bQ}{HJuZs2HiQ_-8Elj0Fvi_1g}{global-eu-62.local}{[gdv2:a284:2acv:5fa6:0:3a2:bba6:fe3f]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true}
+   Hot threads at 2022-04-01T15:15:27.659Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
+      
+   18.7% (93.4ms out of 500ms) cpu usage by thread 'opensearch[global-eu-62][transport_worker][T#3]'
+     6/10 snapshots sharing following 3 elements
+       io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
+       io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
+       java.base@11.0.14.1/java.lang.Thread.run(Thread.java:829)
+::: {global-eu-44}{8WW3hrkcTwGvgah_L8D_jw}{Sok7spHISFyol0jFV6i0kw}{global-eu-44.local}{[gdv2:a284:2acv:5fa6:0:3a2:9120:e79e]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true}
+   Hot threads at 2022-04-01T15:15:27.659Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
+   
+   42.6% (212.7ms out of 500ms) cpu usage by thread 'opensearch[global-eu-44][write][T#5]'
+     2/10 snapshots sharing following 43 elements
+       java.base@11.0.14.1/sun.nio.ch.IOUtil.write1(Native Method)
+       java.base@11.0.14.1/sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:254)
+       io.netty.channel.nio.NioEventLoop.wakeup(NioEventLoop.java:787)
+       io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:846)
+       io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:815)
+       io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:989)
+       io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:796)
+       io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:758)
+       io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1020)
+       io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:311)
+       org.opensearch.transport.netty4.Netty4TcpChannel.sendMessage(Netty4TcpChannel.java:159)
+       app//org.opensearch.transport.OutboundHan...
+```
+
+## Required permissions
+
+If you use the security plugin, make sure you have the appropriate permissions:
+`cluster:monitor/nodes/hot_threads`
+{: .note }
diff --git a/_opensearch/rest-api/nodes-apis/nodes-info.md b/_opensearch/rest-api/nodes-apis/nodes-info.md
@@ -0,0 +1,131 @@
+---
+layout: default
+title: Nodes info
+parent: Nodes APIs
+grand_parent: REST API reference
+nav_order: 20
+---
+
+# Nodes info
+
+Represents mostly static information about cluster nodes.
+Such as host system information, JVM, processor type, or specific
+node information like node settings, thread pools settings, installed plugins, and more.
+
+## Example
+
+```json
+# Get information from all cluster nodes
+GET /_nodes
+
+# Get thread pool information from the cluster manager node only
+GET /_nodes/master:true/thread_pool?pretty
+```
+
+## Path and HTTP methods
+
+```text
+GET /_nodes
+GET /_nodes/{nodeId}
+GET /_nodes/{metrics}
+GET /_nodes/{nodeId}/{metrics}
+# or full path equivalent
+GET /_nodes/{nodeId}/info/{metrics}
+```
+
+## URL parameters
+
+You can include the following URL parameters in your request. All parameters are optional.
+
+Parameter | Type   | Description
+:--- |:-------| :---
+nodeId | String | A comma-separated list of nodeIds to filter results. Supports [node filters](../index/#node-filters).<br>Defaults to `_all`.
+metrics | String | A comma-separated list of metric groups that will be included in the response. For example `jvm,thread_pools`.<br>Defaults to all metrics.
+timeout | TimeValue | A request [timeout](../index/#timeout).<br>Defaults to `30s`.
+
+The following are listed all available metric groups:
+
+Metric | Description
+:--- |:----
+`settings` | A node settings. This is combination of the default settings, custom settings from [configuration file](../../../configuration/#configuration-file) and dynamically [updated settings](../../../configuration/#update-cluster-settings-using-the-api).
+`os` | Static information about host os, including version, processor architecture and available/allocated processors.
+`process` | Contains process id.
+`jvm` | Detailed static information about running JVM, including arguments.
+`thread_pool` | Configured options for all individual thread pools (type, size ... etc).
+`transport` | Mostly static information about transport layer.
+`http` | Mostly static information about http layer.
+`plugins` | Information about installed plugins and modules.
+`ingest` | Information about ingets pipelines and available ingest processors.
+`aggregations` | Information about available [aggregations](../../../aggregations).
+`indices` | Static index settings configured at the node level.
+
+## Response
+
+The response contains basic node identification and build info for every node
+matching the `{nodeId}` request parameter.
+
+Metric | Description
+:--- |:----
+name | A node name.
+transport_address | A node transport address.
+host | A node host address.
+ip | A node host ip address.
+version | A node OpenSearch version.
+build_type | A build type, like `rpm`,`docker`, `zip` ... etc.
+build_hash | A git commit hash of the build.
+roles | A node roles.
+attributes | A node attributes.
+
+On top of that it will contain one or more metric groups depending on `{metrics}` request parameter.
+
+```json
+GET /_nodes/master:true/process,transport?pretty
+
+{
+  "_nodes": {
+    "total": 1,
+    "successful": 1,
+    "failed": 0
+  },
+  "cluster_name": "opensearch",
+  "nodes": {
+    "VC0d4RgbTM6kLDwuud2XZQ": {
+      "name": "node-m1-23",
+      "transport_address": "127.0.0.1:9300",
+      "host": "127.0.0.1",
+      "ip": "127.0.0.1",
+      "version": "1.3.1",
+      "build_type": "tar",
+      "build_hash": "c4c0672877bf0f787ca857c7c37b775967f93d81",
+      "roles": [
+        "data",
+        "ingest",
+        "master",
+        "remote_cluster_client"
+      ],
+      "attributes": {
+        "shard_indexing_pressure_enabled": "true"
+      },
+      "process" : {
+        "refresh_interval_in_millis": 1000,
+        "id": 44584,
+        "mlockall": false
+      },
+      "transport": {
+        "bound_address": [
+          "[::1]:9300",
+          "127.0.0.1:9300"
+        ],
+        "publish_address": "127.0.0.1:9300",
+        "profiles": { }
+      }
+    }
+  }
+}
+```
+
+## Required permissions
+
+If you use the security plugin, make sure you have the appropriate permissions:
+`cluster:monitor/nodes/info`
+{: .note }