Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark remote sampling API as stable #599

Merged
merged 2 commits into from
Jan 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/_client_libs/client-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ weight: 5
---

{{< warning >}}
Jaeger clients are being retired. Please refer to this [notice](../client-libraries/#deprecating-jaeger-clients).
Jaeger clients have been retired. Please refer to this [notice](../client-libraries/#deprecating-jaeger-clients).
{{< /warning >}}

The table below provides a feature matrix for the existing client libraries. Cells marked with `?` indicate that it's not known if the given client supports the given feature and additional research & documentation update is required.
Expand Down
2 changes: 1 addition & 1 deletion content/_client_libs/client-libraries.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ children:
---

{{< warning >}}
Jaeger clients are being retired.
Jaeger clients have been retired. Please use the OpenTelemetry SDKs.
{{< /warning >}}

## Deprecating Jaeger clients
Expand Down
57 changes: 47 additions & 10 deletions content/docs/1.41/apis.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,31 +19,36 @@ Agent and Collector are the two components of the Jaeger backend that can receiv

### OpenTelemetry Protocol (stable)

Since v1.35, the Jaeger backend can receive trace data from the OpenTelemetry SDKs in their native [OpenTelemetry Protocol (OTLP)][otlp]. That means that the OpenTelemetry SDKs no longer need to be configured with Jaeger exporters, nor the OpenTelemetry Collectors need to be deployed between the OpenTelemetry SDKs and the Jaeger backend.
Since v1.35, the Jaeger backend can receive trace data from the OpenTelemetry SDKs in their native [OpenTelemetry Protocol (OTLP)][otlp]. It is no longer necessary to configure the OpenTelemetry SDKs with Jaeger exporters, nor deploy the OpenTelemetry Collectors between the OpenTelemetry SDKs and the Jaeger backend.

The OTLP data is accepted in these formats: (1) binary gRPC, (2) Protobuf over HTTP, (3) JSON over HTTP. For more details on the OTLP receiver see the [official documentation][otlp-rcvr] (note that not all configuration options are supported in the Jaeger collector, and only tracing data is accepted, since Jaeger does not store other telemetry types).

[otlp-rcvr]: https://github.com/open-telemetry/opentelemetry-collector/blob/main/receiver/otlpreceiver/README.md

### Thrift over UDP (stable)

The Agent can only receive spans over UDP in Thrift format. The primary API is a UDP packet that contains a Thrift-encoded `Batch` struct defined in [jaeger.thrift][jaeger.thrift] IDL file, located in the [jaeger-idl][jaeger-idl] repository. Most Jaeger Clients use Thrift's `compact` encoding, however some client libraries do not support it (notably, Node.js) and use Thrift's `binary` encoding (sent to a different UDP port). The Agent's API is defined by [agent.thrift][agent.thrift] IDL file.
The Agent can only receive spans over UDP in Thrift format. The primary API is a UDP packet that contains a Thrift-encoded `Batch` struct defined in [jaeger.thrift] IDL file, located in the [jaeger-idl] repository. Most Jaeger Clients use Thrift's `compact` encoding, however some client libraries do not support it (notably, Node.js) and use Thrift's `binary` encoding (sent to a different UDP port). The Agent's API is defined by [agent.thrift] IDL file.

For legacy reasons, the Agent also accepts spans over UDP in Zipkin format, however, only very old versions of Jaeger clients can send data in that format and it is officially deprecated.

### Protobuf via gRPC (stable)

In a typical Jaeger deployment, Agents receive spans from Clients and forward them to Collectors. Since Jaeger v1.11 the official and recommended protocol between Agents and Collectors is `jaeger.api_v2.CollectorService` gRPC endpoint defined in [collector.proto][collector.proto] IDL file.
In a typical Jaeger deployment, Agents receive spans from Clients and forward them to Collectors. Since Jaeger v1.11 the official and recommended protocol between Agents and Collectors is `jaeger.api_v2.CollectorService` gRPC endpoint defined in [collector.proto] IDL file. The same endpoint can be used to submit trace data from SDKs directly to the Collector.

### Thrift over HTTP (stable)

In some cases it is not feasible to deploy Jaeger Agent next to the application, for example, when the application code is running as a serverless function. In these scenarios the Jaeger Clients can be configured to submit spans directly to the Collectors over HTTP/HTTPS.
In some cases it is not feasible to deploy Jaeger Agent next to the application, for example, when the application code is running as a serverless function. In these scenarios the SDKs can be configured to submit spans directly to the Collectors over HTTP/HTTPS.

The same [jaeger.thrift][jaeger.thrift] payload can be submitted in HTTP POST request to `/api/traces` endpoint, for example, `https://jaeger-collector:14268/api/traces`. The `Batch` struct needs to be encoded using Thrift's `binary` encoding, and the HTTP request should specify the content type header:
The same [jaeger.thrift] payload can be submitted in HTTP POST request to `/api/traces` endpoint, for example, `https://jaeger-collector:14268/api/traces`. The `Batch` struct needs to be encoded using Thrift's `binary` encoding, and the HTTP request should specify the content type header:

```
Content-Type: application/vnd.apache.thrift.binary
```

### JSON over HTTP (n/a)

There is no official Jaeger JSON format that can be accepted by the collector. In the future the OpenTelemetry JSON may be supported.
There is no official Jaeger JSON format that can be accepted by the collector.
Jaeger does accept the OpenTelemetry protocol via JSON (see [above](#opentelemetry-protocol-stable)).

### Zipkin Formats (stable)

Expand All @@ -58,7 +63,7 @@ Traces saved in the storage can be retrieved by calling Jaeger Query Service.

### gRPC/Protobuf (stable)

The recommended way for programmatically retrieving traces and other data is via `jaeger.api_v2.QueryService` gRPC endpoint defined in [query.proto][query.proto] IDL file.
The recommended way for programmatically retrieving traces and other data is via `jaeger.api_v2.QueryService` gRPC endpoint defined in [query.proto] IDL file.

### HTTP JSON (internal)

Expand All @@ -68,11 +73,43 @@ Jaeger UI communicates with Jaeger Query Service via JSON API. For example, a tr

When using the `grpc-plugin` storage type (a.k.a. [storage plugin](../deployment/#storage-plugin)), Jaeger components can use custom storage backends as long as those backends implement the gRPC [Remote Storage API][storage.proto].

## Clients configuration (internal)
## Remote Sampling Configuration (stable)

This API supports Jaeger's [Remote Sampling](../sampling/#remote-sampling) protocol, defined in [sampling.proto] IDL file.

Both the Jaeger Agent and Jaeger Collector implement the API. See [Remote Sampling](../sampling/#remote-sampling) for details on how to configure the Collector with sampling strategies. The Agent is merely acting as a proxy to the Collector.

The following table lists different endpoints and formats that can be used to query for sampling strategies. The official HTTP/JSON endpoints use standard [Protobuf-to-JSON mapping](https://developers.google.com/protocol-buffers/docs/proto3#json).

Component | Port | Endpoint | Format | Notes
--------- | ----- | ----------------- | --------- | -----
Collector | 14268 | `/api/sampling` | HTTP/JSON | Recommended for most SDKs
Collector | 14250 | [sampling.proto] | gRPC | For SDKs that want to use gRPC (e.g. OpenTelemetry Java SDK)
Agent | 5778 | `/sampling` | HTTP/JSON | Recommended for most SDKs if the Agent is used in a deployment
Agent | 5778 | `/` (deprecated) | HTTP/JSON | Legacy format, with enums encoded as numbers. **Not recommended.**

Client libraries not only submit finished spans to Jaeger backend, but also periodically poll the Agents for various configurations, such as sampling strategies. The schema for the payload is defined by [sampling.thrift][sampling.thrift], encoded as JSON using Thrift's built-in JSON generation capabilities.
**Examples**

Agent acts as a proxy by retrieving sampling configuration from the Collector using `jaeger.api_v2.SamplingManager` gRPC endpoint defined in [sampling.proto][sampling.proto] IDL file.
Run all-in-one in one terminal:
```shell
$ go run ./cmd/all-in-one \
--sampling.strategies-file=cmd/all-in-one/sampling_strategies.json
```

Query different endpoints in another terminal:
```shell
# Collector
$ curl "http://localhost:14268/api/sampling?service=foo"
{"strategyType":"PROBABILISTIC","probabilisticSampling":{"samplingRate":1}}

# Agent
$ curl "http://localhost:5778/sampling?service=foo"
{"strategyType":"PROBABILISTIC","probabilisticSampling":{"samplingRate":1}}

# Agent, legacy endpoint / (not recommended)
$ curl "http://localhost:5778/?service=foo"
{"strategyType":0,"probabilisticSampling":{"samplingRate":1}}
```

## Service dependencies graph (internal)

Expand Down
17 changes: 12 additions & 5 deletions content/docs/1.41/sampling.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,25 @@ Jaeger libraries implement consistent upfront (or head-based) sampling. For exam

## Client Sampling Configuration

{{< warning >}}
This section only applies to classic Jaeger SDKs, which are now deprecated.
We recommend using the [OpenTelemetry SDKs](https://opentelemetry.io).
{{< /warning >}}

When using configuration object to instantiate the tracer, the type of sampling can be selected via `sampler.type` and `sampler.param` properties. Jaeger libraries support the following samplers:

* **Constant** (`sampler.type=const`) sampler always makes the same decision for all traces. It either samples all traces (`sampler.param=1`) or none of them (`sampler.param=0`).
* **Probabilistic** (`sampler.type=probabilistic`) sampler makes a random sampling decision with the probability of sampling equal to the value of `sampler.param` property. For example, with `sampler.param=0.1` approximately 1 in 10 traces will be sampled.
* **Rate Limiting** (`sampler.type=ratelimiting`) sampler uses a leaky bucket rate limiter to ensure that traces are sampled with a certain constant rate. For example, when `sampler.param=2.0` it will sample requests with the rate of 2 traces per second.
* **Remote** (`sampler.type=remote`, which is also the default) sampler consults Jaeger agent for the appropriate sampling strategy to use in the current service. This allows controlling the sampling strategies in the services from a [central configuration](#collector-sampling-configuration) in Jaeger backend, or even dynamically (see [Adaptive Sampling](#adaptive-sampling)).
* **Remote** (`sampler.type=remote`, which is also the default) sampler consults Jaeger agent for the appropriate sampling strategy to use in the current service. This allows controlling the sampling strategies in the services from a central configuration in Jaeger backend (see [Remote Sampling](#remote-sampling)), or even dynamically (see [Adaptive Sampling](#adaptive-sampling)).

## Remote Sampling

## Collector Sampling Configuration
If your client SDKs are configured to use remote sampling configuration (see [Remote Sampling API][remote-sampling-api]) then sampling rates can be centrally controlled via the Jaeger Collectors. In this setup a sampling strategy configuration is served to the client SDK that describes endpoints and their sampling probabilities. This configuration can be generated by the Jaeger Collector in two different ways: [periodically loaded from a file](#file-based-sampling-configuration) or [dynamically calculated based on traffic](#adaptive-sampling). The method of generation is controlled by the environment variable `SAMPLING_CONFIG_TYPE` which can be set to either `file` (default) or `adaptive`.

If your clients are configured to use remote sampling then sampling rates can be centrally controlled via the collectors. In a remote sampling setup a json document is served to the Jaeger client that describes endpoints and their sampling probabilities. This document can be generated two different ways: [periodically loaded from a file](#file-sampling) or [dynamically based on traffic](#adaptive-sampling). The method of document generation is controlled by the environment variable `SAMPLING_CONFIG_TYPE` which can be set to either `file`(default) or `adaptive`.
[remote-sampling-api]: ../apis#remote-sampling-configuration-stable

### File Sampling
### File-based Sampling Configuration

Collectors can be instantiated with the `--sampling.strategies-file` option that points to a file containing sampling strategies to be served to Jaeger clients. The option's value can contain a path to a JSON file, which will be automatically reloaded if its contents change, or an HTTP URL from where the file will be periodically retrieved, with reload frequency controlled by the `--sampling.strategies-reload-interval` option.

Expand Down Expand Up @@ -89,4 +96,4 @@ Adaptive sampling requires a storage backend to store the observed traffic data

By default adaptive sampling will attempt to use the backend specified by `SPAN_STORAGE_TYPE` to store data. However, a second type of backend can also be specified by using `SAMPLING_STORAGE_TYPE`. For instance, `SPAN_STORAGE_TYPE=elasticsearch SAMPLING_STORAGE_TYPE=cassandra ./jaeger-collector` will run the collector in a mode where it attempts to store its span data in the configured elasticsearch cluster and its adaptive sampling data in the configured cassandra cluster. Note that this feature can not be used to store span and adaptive sampling data in two different backends of the same type.

Read [this blog post](https://medium.com/jaegertracing/adaptive-sampling-in-jaeger-50f336f4334) for more details on adaptive sampling engine.
Read [this blog post](https://medium.com/jaegertracing/adaptive-sampling-in-jaeger-50f336f4334) for more details on adaptive sampling engine.
Loading