Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify MeterProvider configurable cardinality limits #2960

Merged
merged 37 commits into from
May 8, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
61264c7
Specify MetricReader configurable limits
Nov 18, 2022
7ba02d1
Add PR number and spec-matrix entry
Nov 18, 2022
16f6170
edits
Nov 18, 2022
82fb36d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Nov 28, 2022
0f78e17
changelog order
Nov 28, 2022
26b2ce9
corrections, update spec matrix
Nov 28, 2022
57d35a5
Update specification/metrics/sdk.md
jmacd Nov 30, 2022
e755ee6
Update specification/metrics/sdk.md
jmacd Nov 30, 2022
00c44f6
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Dec 12, 2022
8c87107
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
Dec 12, 2022
20c7555
revision
Dec 13, 2022
dcbcaf3
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Dec 16, 2022
7cdc7ab
toc
Dec 16, 2022
f12c8b5
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Jan 4, 2023
df16924
Apply suggestions from code review
jmacd Jan 6, 2023
1d7a117
Merge branch 'main' into jmacd/metrics_limits
reyang Jan 17, 2023
15bce36
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Feb 9, 2023
d116a28
Update with ideas from Jan Spec SIG
Feb 9, 2023
a92c4c7
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
Feb 9, 2023
c72fae4
refine text
Feb 10, 2023
bfbf348
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Mar 3, 2023
6cd74e0
Revise based on prototype in otel-launcher-go
Mar 3, 2023
0e59093
revise spelling
Mar 3, 2023
c4a5a61
revise spelling
Mar 3, 2023
7e2bf6d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Mar 27, 2023
81ae9e7
clarify wording; give MetricReader control over the default
Mar 27, 2023
88c1557
reword
Mar 27, 2023
fbf66c7
use aggregation_cardinality_limit
Mar 28, 2023
e53c22d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 17, 2023
f1f47f4
clarify attribute SETs h/t @pirgeo
Apr 17, 2023
a0a6596
Update specification/metrics/sdk.md
jmacd Apr 17, 2023
7099f07
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 19, 2023
7505081
update changelog
Apr 19, 2023
9569ef5
lint
Apr 19, 2023
1a98c25
Update specification/metrics/sdk.md
jmacd May 4, 2023
4aaad07
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
May 5, 2023
cee0c74
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
May 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ release.

### Metrics

- Recommended limits applied by default at the MetricProvider level to protect
metrics pipelines against excessive data production from a single
instrumentation library or instrument.
([#2960](https://github.com/open-telemetry/opentelemetry-specification/pull/2960))

### Logs

### Resource
Expand Down
1 change: 1 addition & 0 deletions spec-compliance-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ formats is required. Implementing more than one format is optional.
| The API provides a way to set and get a global default `MeterProvider`. | X | + | + | + | + | | | | | + | - | |
| It is possible to create any number of `MeterProvider`s. | X | + | + | + | + | | | | | + | + | |
| `MeterProvider` provides a way to get a `Meter`. | | + | + | + | + | | | | | + | - | |
| `MeterProvider` provides a way to set cardinality limits. | X | | | | | | | | | | | |
| `get_meter` accepts name, `version` and `schema_url`. | | + | + | + | + | | | | | + | - | |
| `get_meter` accepts `attributes`. | | | | | | | | | | | | |
| When an invalid `name` is specified a working `Meter` implementation is returned as a fallback. | | + | + | + | + | | | | | + | - | |
Expand Down
49 changes: 49 additions & 0 deletions specification/metrics/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ linkTitle: SDK
* [Observations inside asynchronous callbacks](#observations-inside-asynchronous-callbacks)
* [Resolving duplicate instrument registration conflicts](#resolving-duplicate-instrument-registration-conflicts)
- [Attribute limits](#attribute-limits)
- [Cardinality limits](#cardinality-limits)
- [Exemplar](#exemplar)
* [ExemplarFilter](#exemplarfilter)
* [ExemplarReservoir](#exemplarreservoir)
Expand Down Expand Up @@ -70,6 +71,12 @@ metrics produced by any `Meter` from the `MeterProvider`. The [tracing SDK
specification](../trace/sdk.md#additional-span-interfaces) has provided some
suggestions regarding how to implement this efficiently.

A `MeterProvider` SHOULD provide a way to specify the configurable
[cardinality limits](#cardinality-limits) to be applied to `Meter`
instances. SDKs that support configurable limits MUST provide an
option to configure the default limits to be applied to all `Meter`
instances by default.

### Meter Creation

New `Meter` instances are always created through a `MeterProvider`
Expand Down Expand Up @@ -586,6 +593,48 @@ Attributes which belong to Metrics are exempt from the
time. Attribute truncation or deletion could affect identity of metric time
series and the topic requires further analysis.

## Cardinality limits

**Status**: [Experimental](../document-status.md)

MeterProviders SHOULD support being configured with limits to be
applied to metrics produced from a single metric instrument. This
mechanism supports protecting metrics pipelines from excessive data
production in cases when the number of timeseries produced by
application code grows large, which can happen due to several factors.

Whether because of single attributes having many distinct values or
because of combinatorial expansion among many attributes, these limits
help protect the overall system from individual sources of excessive
metrics instrumentation.

When the limit is reached by an the individual metric instrument, the
entire batch of metrics for the instrument MUST be dropped and an
jmacd marked this conversation as resolved.
Show resolved Hide resolved
error reported to the user indicating:

- Detail about the limit that was exceeded (e.g., the library name and
version, the instrument, and the configured limit).
- Recommend the user to configure a [View](#view) that filters the
offending instrument to limit cardinality or otherwise correct the problem.

Note that limit expressed here refers to exported data for a single
instrument and collection interval. The number of timeseries referred
to here corresponds with the number of distinct data points produced
by the instrument over one collection interval.

Under some circumstances, depending on the configured aggregation
temporality, the MeterProvider may be able to recover after these
limits are reached.
jmacd marked this conversation as resolved.
Show resolved Hide resolved

### Specific cardinality limits

There is only one configurable limit: the number of timeseries per
instrument, with recommended default value 2000.
jmacd marked this conversation as resolved.
Show resolved Hide resolved

SDKs SHOULD NOT support unlimited cardinality. There is no unset
jmacd marked this conversation as resolved.
Show resolved Hide resolved
jmacd marked this conversation as resolved.
Show resolved Hide resolved
value for this configurable limit. Users that wish to configure
"unlimited" cardinality should instead configure a very large limit.

## Exemplar

**Status**: [Feature-freeze](../document-status.md)
Expand Down