From 9813405a4e9d326c4dc1864c58f0dcd2f1e930ef Mon Sep 17 00:00:00 2001 From: Josh Mock Date: Tue, 18 Jun 2024 18:13:34 -0500 Subject: [PATCH 1/5] Documentation for OpenTelemetry support --- docs/changelog.asciidoc | 19 ++++++ docs/observability.asciidoc | 113 ++++++++++++++++++++++-------------- 2 files changed, 87 insertions(+), 45 deletions(-) diff --git a/docs/changelog.asciidoc b/docs/changelog.asciidoc index 20de84fc7..cc5d0aaef 100644 --- a/docs/changelog.asciidoc +++ b/docs/changelog.asciidoc @@ -1,6 +1,25 @@ [[changelog-client]] == Release notes +[discrete] +=== 8.15.0 + +[discrete] +==== Features + +[discrete] +===== Support for Elasticsearch `v8.15.0` + +You can find all the API changes +https://www.elastic.co/guide/en/elasticsearch/reference/8.15/release-notes-8.15.0.html[here]. + +[discrete] +===== OpenTelemetry zero-code instrumentation support + +For those that use an observability service that supports OpenTelemetry spans, the client will now automatically generate traces for each Elasticsearch request it makes. +See https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/observability.html#_opentelemetry[the docs] +for more information. + [discrete] === 8.14.0 diff --git a/docs/observability.asciidoc b/docs/observability.asciidoc index 8ae57bcad..c6e189b1c 100644 --- a/docs/observability.asciidoc +++ b/docs/observability.asciidoc @@ -1,24 +1,52 @@ [[observability]] === Observability -The client does not provide a default logger, but instead it offers an event -emitter interface to hook into internal events, such as `request` and -`response`. +To observe and measure Elasticsearch client usage, several client features are provided. -Correlating those events can be hard, especially if your applications have a -large codebase with many events happening at the same time. +First, as of 8.15.0, the client provides native support for OpenTelemetry, which allows you to send client usage data to any endpoint that supports OpenTelemetry without having to make any changes to your JavaScript codebase. -To help you with this, the client offers you a correlation id system and other -features. Let's see them in action. +Also, rather than providing a default logger, the clientoffers an event +emitter interface to hook into internal events, such as `request` and +`response`, allowing you to log the events you care about, or otherwise react +to client usage however you might need. +Correlating events can be hard, especially if your applications have a large codebase with many events happening at the same time. To help you with this, the client provides a correlation ID system, and other +features. + +All of these observability features are documented below. + +[discrete] +==== OpenTelemetry + +The client supports OpenTelemetry's https://opentelemetry.io/docs/zero-code/js/[zero-code +instrumentation] to enable tracking each client request as an +https://opentelemetry.io/docs/concepts/signals/traces/#spans[OpenTelemetry span]. These spans +follow all of the https://opentelemetry.io/docs/specs/semconv/database/elasticsearch/[semantic +OpenTelemetry conventions for Elasticsearch] except for `db.query.text`. + +To start sending Elasticsearch trace data to your OpenTelemetry endpoint, follow +https://opentelemetry.io/docs/zero-code/js/[OpenTelemetry's zero-code instrumentation guide], +or the following steps: + +1. Install `@opentelemetry/api` and `@opentelemetry/auto-instrumentations-node` as Node.js dependencies +2. Export the following environment variables with the appropriate values: + - `OTEL_EXPORTER_OTLP_ENDPOINT` + - `OTEL_EXPORTER_OTLP_HEADERS` + - `OTEL_RESOURCE_ATTRIBUTES` + - `OTEL_SERVICE_NAME` +3. `require` the Node.js auto-instrumentation library at startup: +[source,bash] +---- +node --require '@opentelemetry/auto-instrumentations-node/register' index.js +---- [discrete] ==== Events -The client is an event emitter, this means that you can listen for its event and -add additional logic to your code, without need to change the client internals -or your normal usage. You can find the events names by access the `events` key -of the client. +The client is an event emitter. This means that you can listen for its events to +add additional logic to your code, without needing to change the client's internals +or how you use the client. You can find the events' names by accessing the `events` key +of the client: [source,js] ---- @@ -26,9 +54,8 @@ const { events } = require('@elastic/elasticsearch') console.log(events) ---- - -The event emitter functionality can be useful if you want to log every request, -response and error that is happening during the use of the client. +The event emitter functionality can be useful if you want to log every request, +response or error that is created by the client: [source,js] ---- @@ -48,7 +75,6 @@ client.diagnostic.on('response', (err, result) => { }) ---- - The client emits the following events: [cols=2*] |=== @@ -59,7 +85,7 @@ a|Emitted before starting serialization and compression. If you want to measure client.diagnostic.on('serialization', (err, result) => { console.log(err, result) }) ----- +----e |`request` a|Emitted before sending the actual request to {es} _(emitted multiple times in case of retries)_. @@ -108,7 +134,7 @@ client.diagnostic.on('resurrect', (err, result) => { |=== -The values of `result` in `serialization`, `request`, `deserialization`, +The values of `result` in `serialization`, `request`, `deserialization`, `response` and `sniff` are: [source,ts] @@ -135,7 +161,6 @@ meta: { }; ---- - While the `result` value in `resurrect` is: [source,ts] @@ -152,10 +177,10 @@ request: { [discrete] ===== Events order -The event order is described in the following graph, in some edge cases, the +The event order is described in the following graph, in some edge cases, the order is not guaranteed. -You can find in -https://github.com/elastic/elasticsearch-js/blob/main/test/acceptance/events-order.test.js[`test/acceptance/events-order.test.js`] +You can find in +https://github.com/elastic/elasticsearch-js/blob/main/test/acceptance/events-order.test.js[`test/acceptance/events-order.test.js`] how the order changes based on the situation. [source] @@ -175,12 +200,11 @@ serialization └─▶ response ---- - [discrete] -==== Correlation id +==== Correlation ID -Correlating events can be hard, especially if there are many events at the same -time. The client offers you an automatic (and configurable) system to help you +Correlating events can be hard, especially if there are many events at the same +time. The client offers you an automatic (and configurable) system to help you handle this problem. [source,js] @@ -211,8 +235,7 @@ client.search({ }).then(console.log, console.log) ---- - -By default the id is an incremental integer, but you can configure it with the +By default the ID is an incremental integer, but you can configure it with the `generateRequestId` option: [source,js] @@ -231,7 +254,7 @@ const client = new Client({ ---- -You can also specify a custom id per request: +You can also specify a custom ID per request: [source,js] ---- @@ -247,8 +270,8 @@ client.search({ [discrete] ==== Context object -Sometimes, you might need to make some custom data available in your events, you -can do that via the `context` option of a request: +Sometimes, you might need to make some custom data available in your events, you +can do that via the `context` option of a request: [source,js] ---- @@ -283,7 +306,7 @@ client.search({ ---- The context object can also be configured as a global option in the client -configuration. If you provide both, the two context objects will be shallow +configuration. If you provide both, the two context objects will be shallow merged, and the API level object will take precedence. [source,js] @@ -323,9 +346,9 @@ client.search({ [discrete] ==== Client name -If you are using multiple instances of the client or if you are using multiple -child clients _(which is the recommended way to have multiple instances of the -client)_, you might need to recognize which client you are using. The `name` +If you are using multiple instances of the client or if you are using multiple +child clients _(which is the recommended way to have multiple instances of the +client)_, you might need to recognize which client you are using. The `name` options help you in this regard. [source,js] @@ -374,15 +397,15 @@ child.search({ [discrete] ==== X-Opaque-Id support -To improve observability, the client offers an easy way to configure the -`X-Opaque-Id` header. If you set the `X-Opaque-Id` in a specific request, this -allows you to discover this identifier in the -https://www.elastic.co/guide/en/elasticsearch/reference/current/logging.html#deprecation-logging[deprecation logs], -helps you with https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html#_identifying_search_slow_log_origin[identifying search slow log origin] +To improve observability, the client offers an easy way to configure the +`X-Opaque-Id` header. If you set the `X-Opaque-Id` in a specific request, this +allows you to discover this identifier in the +https://www.elastic.co/guide/en/elasticsearch/reference/current/logging.html#deprecation-logging[deprecation logs], +helps you with https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html#_identifying_search_slow_log_origin[identifying search slow log origin] as well as https://www.elastic.co/guide/en/elasticsearch/reference/current/tasks.html#_identifying_running_tasks[identifying running tasks]. -The `X-Opaque-Id` should be configured in each request, for doing that you can -use the `opaqueId` option, as you can see in the following example. The +The `X-Opaque-Id` should be configured in each request, for doing that you can +use the `opaqueId` option, as you can see in the following example. The resulting header will be `{ 'X-Opaque-Id': 'my-search' }`. [source,js] @@ -401,10 +424,10 @@ client.search({ }).then(console.log, console.log) ---- -Sometimes it may be useful to prefix all the `X-Opaque-Id` headers with a -specific string, in case you need to identify a specific client or server. For -doing this, the client offers a top-level configuration option: -`opaqueIdPrefix`. In the following example, the resulting header will be +Sometimes it may be useful to prefix all the `X-Opaque-Id` headers with a +specific string, in case you need to identify a specific client or server. For +doing this, the client offers a top-level configuration option: +`opaqueIdPrefix`. In the following example, the resulting header will be `{ 'X-Opaque-Id': 'proxy-client::my-search' }`. [source,js] From 24078888ccfc4d56817b6288e68ec30b737db648 Mon Sep 17 00:00:00 2001 From: Josh Mock Date: Thu, 20 Jun 2024 11:21:32 -0500 Subject: [PATCH 2/5] Update docs/observability.asciidoc Co-authored-by: Miguel Grinberg --- docs/observability.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/observability.asciidoc b/docs/observability.asciidoc index c6e189b1c..b967efff7 100644 --- a/docs/observability.asciidoc +++ b/docs/observability.asciidoc @@ -5,7 +5,7 @@ To observe and measure Elasticsearch client usage, several client features are p First, as of 8.15.0, the client provides native support for OpenTelemetry, which allows you to send client usage data to any endpoint that supports OpenTelemetry without having to make any changes to your JavaScript codebase. -Also, rather than providing a default logger, the clientoffers an event +Also, rather than providing a default logger, the client offers an event emitter interface to hook into internal events, such as `request` and `response`, allowing you to log the events you care about, or otherwise react to client usage however you might need. From 67ae85a5b5ef218d868ebb2dba20f349b50338d2 Mon Sep 17 00:00:00 2001 From: Josh Mock Date: Thu, 20 Jun 2024 11:39:11 -0500 Subject: [PATCH 3/5] Fix docs typo --- docs/observability.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/observability.asciidoc b/docs/observability.asciidoc index b967efff7..9436d457f 100644 --- a/docs/observability.asciidoc +++ b/docs/observability.asciidoc @@ -85,7 +85,7 @@ a|Emitted before starting serialization and compression. If you want to measure client.diagnostic.on('serialization', (err, result) => { console.log(err, result) }) -----e +---- |`request` a|Emitted before sending the actual request to {es} _(emitted multiple times in case of retries)_. From 10b301b21f95665e128535a4f4b8a0b7927a21e1 Mon Sep 17 00:00:00 2001 From: Josh Mock Date: Thu, 20 Jun 2024 12:04:21 -0500 Subject: [PATCH 4/5] Fix bad link references in asciidoc changelog --- docs/changelog.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/changelog.asciidoc b/docs/changelog.asciidoc index cc5d0aaef..b5da7a455 100644 --- a/docs/changelog.asciidoc +++ b/docs/changelog.asciidoc @@ -17,7 +17,7 @@ https://www.elastic.co/guide/en/elasticsearch/reference/8.15/release-notes-8.15. ===== OpenTelemetry zero-code instrumentation support For those that use an observability service that supports OpenTelemetry spans, the client will now automatically generate traces for each Elasticsearch request it makes. -See https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/observability.html#_opentelemetry[the docs] +See {jsclient}/observability.html#_opentelemetry[the docs] for more information. [discrete] @@ -217,7 +217,7 @@ https://www.elastic.co/guide/en/elasticsearch/reference/8.9/release-notes-8.9.0. [discrete] ===== Allow document to be overwritten in `onDocument` iteratee of bulk helper https://github.com/elastic/elasticsearch-js/pull/1732[#1732] -In the https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/client-helpers.html#bulk-helper[bulk helper], documents could not be modified before being sent to Elasticsearch. It is now possible to https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/client-helpers.html#_modifying_a_document_before_operation[modify a document] before sending it. +In the {jsclient}/client-helpers.html#bulk-helper[bulk helper], documents could not be modified before being sent to Elasticsearch. It is now possible to {jsclient}/client-helpers.html#_modifying_a_document_before_operation[modify a document] before sending it. [discrete] ==== Fixes From 4361112d7d76ba834a6d4befe344621a9e83a832 Mon Sep 17 00:00:00 2001 From: Josh Mock Date: Thu, 20 Jun 2024 14:17:09 -0500 Subject: [PATCH 5/5] Drop link to 8.15 changelog For now. Link just doesn't work yet. --- docs/changelog.asciidoc | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/changelog.asciidoc b/docs/changelog.asciidoc index b5da7a455..d3474617d 100644 --- a/docs/changelog.asciidoc +++ b/docs/changelog.asciidoc @@ -7,12 +7,6 @@ [discrete] ==== Features -[discrete] -===== Support for Elasticsearch `v8.15.0` - -You can find all the API changes -https://www.elastic.co/guide/en/elasticsearch/reference/8.15/release-notes-8.15.0.html[here]. - [discrete] ===== OpenTelemetry zero-code instrumentation support