From bdc607d7ce7cc0c4efa8764a4d3529b120cfb8fb Mon Sep 17 00:00:00 2001 From: jmacd Date: Thu, 10 Oct 2019 14:49:02 -0700 Subject: [PATCH 01/12] Edits, expansion --- specification/api-metrics-user.md | 411 +++++++++++++++++++++++++++++- 1 file changed, 407 insertions(+), 4 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index 4f33f742f58..b38d12ba9cc 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -1,6 +1,409 @@ # Metric User-facing API -This document is a placeholder pending active discussion. It will -eventually contain portions that were removed from -https://github.com/open-telemetry/opentelemetry-specification/blob/3d2b8ecf410f62172a22a9fbff88304724d4cc78/specification/api-metrics.md -after further discussion. +TODO Table of contents. + +## Overview + +Metric instruments are the entry point for application and framework +developers to instrument their code using counters, gauges, and +measures. + +### Metric names + +Metric instruments have names, which are how we refer to them in +external systems. Metric names conform to the following syntax: + +1. They are non-empty strings +1. They are case-insensitive +1. The first character must be non-numeric, non-space, non-punctuation +1. Subsequent characters must be belong to the alphanumeric characters, '_', '.', and '-'. + +Metrics names belong to a namespace by virtue of a "Named" `Meter` +instance. A "Named" `Meter` refers to the requirement that every +`Meter` instance must have an associated `component` label, determined +statically in the code. The `component` label value of the associated +`Meter` serves as its namespace, allowing the same metric name to be +used in multiple libraries of code, unambiguously, within the same +application. + +Metric instruments are defined using a `Meter` instance, using a +variety of `New` methods specific to the kind of metric and type of +input (integer or floating point). The Meter will return an error +when a metric name is already registered with a different kind for the +same component name. Metric systems are expected to automatically +prefix exported metrics by the `component` namespace in a manner +consistent with the target system. For example, a Prometheus exporter +SHOULD use the component followed by `_` as the [application +prefix](https://prometheus.io/docs/practices/naming/#metric-names). + +### Format of a metric event + +Regardless of the instrument kind or method of input, metric events +include the instrument descriptor, a numerical value, and an optional +set of labels. The descriptor, discussed in detail below, contains +the metric name and various optional settings. + +Labels are key:value pairs associated with events describing various +dimensions or categories that describe the event. A "label key" +refers to the key component while "label value" refers to the +correlated value component of a label. Label refers to the pair of +label key and value. + +Metric events always have an associated `component` label, by virtue +of the named `Meter` used in their definition. Other labels are +passed in to the metric event in the form of a `LabelSet` argument, +using several input methods discussed below. + +### New constructors + +The `Meter` interface allows creating of a registered metric +instrument using methods specific to each kind of metric. There are +six constructors representing the three kinds of instrument taking +either floating point or integer inputs, see the detailed design below. + +Binding instruments to a single `Meter` instance has two benefits: + +1. Instruments can be exported freom the zero state, prior to first use, with no explicit `Register` call +1. The component name provided by the named `Meter` satisfies a namespace requirement + +The recommended practice is to define structures to contain the +instruments in use and keep references only to the instruments that +are specifically needed. + +We recognize that many existing metric systems support allocating +metric instruments statically and providing the `Meter` interface at +the time-of-use. In this example, typical of statsd clients, existing +code may not be structured with a convenient place to store new metric +instruments. Where this becomes a burden, it may be acceptable to use +the global `Meter` as a workaround. + +The situation is similar for users of Prometheus clients, where +instruments are allocated statically and there is an implicit global. +Such code may not have access to the appropriate `Meter` where +instruments are defined. Where this becomes a burden, it may be +acceptable to use the global `Meter` as a workaround. + +#### Metric instrument descriptors + +An instrument `Descriptor` is a structure describing all the +configurable aspects of an instrument that the user can elect. +Although the API provides a common `Descriptor` type--not the +SDK--users do not construct these directly. Users pass all common +configuration options to the appropriate `Meter.New` method, which +itself will use a helper method provided by the API to build the new +`Descriptor`. Users can access the descriptor of the built +instrument, in any case. + +Applications are expected to construct long-lived instruments. +Instruments are considered permanent, there is no method to delete +them forget the metrics they produce in the SDK. The structure of the +descriptor and various options are given in detail below. + +#### Metric instrument constructor example code + +In this Golang example, a struct holding four instruments is built +using the provided `Meter`. These calls give examples of the kind of +configuration options available for descriptors as well. + +```golang +type instruments struct { + counter1 metric.Int64Counter + counter2 metric.Float64Counter + gauge3 metric.Int64Gauge + measure4 metric.Float64Measure +} + +func newInstruments(metric.Meter meter) *instruments { + return &instruments{ + counter1: meter.NewCounter("counter1", metric.WithKeys("client.service")) + counter2: meter.NewCounter("counter2", metric.WithNonMonotonic(true)) + gauge3: meter.NewGauge("gauge3", metric.WithUnit(unit.Bytes)). + measure4: meter.NewMeasure("measure4", metric.WithDescription("Measure of ...")). + } +} +``` + +Code will be structured to call `newInstruments` somewhere in a +constructor and keep the `instruments` reference for use at runtime. +Here's an example of building a server with configured instruments and +a single metric operation. + +```golang +type server struct { + meter metric.Meter + instruments *instruments + + // ... other fields +} + +func newServer(meter metric.Meter) *server { + return &server{ + meter: meter, + instruments: newInstruments(meter), + + // ... other fields + } +} + +// ... + +func (s *server) operate(ctx context.Context) { + // ... other work + + s.instruments.counter1.Add(ctx, 1, s.meter.Labels( + label1.String("..."), + label2.String("..."))) +} +``` + +### Metric calling conventions + +This API is factored into three core concepts: instruments, handles, +and label sets. In doing so, we provide several ways of capturing +measurements that are semantically equivalent and generate equivalent +metric events, but offer varying degrees of performance and +convenience. + +#### Metric handle calling convention + +As described above, metric events consist of an instrument, a set of +labels, and a numerical value. The performance of a metric API +depends on the work done to enter a new measurement. One approach to +reduce cost is to pre-aggregate results, so that subsequent events in +the same collection period for the same label set combine into the +same working memory. + +This approach requires locating an entry for the instrument and label +set in a table of some kind, finding the place where a group of metric +events are being aggregated. This lookup can be successfully +precomputed, giving rise to the Handle calling convention. + +In situations where performance is a requirement and a metric is +repeatedly used with the same set of labels, the developer may elect +to use _instrument handles_ as an optimization. For handles to be a +benefit, it requires that a specific instrument will be re-used with +specific labels. + +To obtain a handle given an instrument and label set, use the +`GetHandle()` method to return an interface that supports the `Add()`, +`Set()`, or `Record()` method of the instrument in question. + +A high-performace metrics SDK will take steps to ensure that +operations on handles are very fast. Application developers are +required to delete handles when they are no-longer in use. + +```golang +func (s *server) processStream(ctx context.Context) { + + streamLabels = []core.KeyValue{ + labelA.String("..."), + labelB.String("..."), + } + counter2Handle := s.instruments.counter2.GetHandle(streamLabels) + + for _, item := <-s.channel { + // ... other work + + // High-performance metric calling convention: use of handles. + counter2Handle.Add(ctx, item.size()) + } + + counter2Handle.Delete() +} +``` + +#### Direct metric calling convention + +When convenience is more important than performance, or there is no +re-use to potentially optimize, users may elect to operate directly on +metric instruments, supplying a label set at the call site. + +For example, to update a single counter: + +```golang +func (s *server) method(ctx context.Context) { + // ... other work + + s.instruments.counter1.Add(ctx, 1, s.meter.Labels(...)) +} +``` + +This method offers the greatest convenience possible. If performance +becomes a problem, one option is to use handles as described above. +Another performance option, in some cases, is to just re-use the +labels. In the example here, `meter.Labels(...)` constructs a +re-usable label set which may be a useful performance optimization, as +discussed next. + +#### Label set calling convention + +A significant factor in the cost of metrics export is that labels, +which arrive as an unordered list of keys and values, must be +canonicalized in some way before they can be used for lookup. +Canonicalizing labels can be an expensive operation as it may require +sorting or de-duplicating by some other means, possibly even +serializing, the set of labels to produce a valid map key. + +The operation of converting an unordered set of labels into a +canonicalized set of labels, useful for pre-aggregation, is expensive +enough that we give it first-class treatment in the API. The +`meter.Labels(...)` API canonicalizes labels, returning an opaque +`LabelSet` object, another form of pre-computation available to the +user. + +Re-usable `LabelSet` objects provide a potential optimization for +scenarios where handles might not be effective. For example, if the +label set will be re-used but only used once per metric, handles do +not offer any optimization. It may be best to pre-compute a +canonicalized `LabelSet` once and re-use it with the direct calling +convention. + +```golang +func (s *server) method(ctx context.Context) { + // ... other work + + labelSet := s.meter.Labels(...) + + s.instruments.counter1.Add(ctx, 1, labelSet) + + // ... more work + + s.instruments.gauge1.Set(ctx, 10, labelSet) + + // ... more work + + s.instruments.measure1.Record(ctx, 100, labelSet) +} +``` + +#### RecordBatch calling convention + +There is one final API for entering measurements, which is like the +direct access calling convention but supports multiple simultaneous +measurements. The use of a RecordBatch API supports entering multiple +measurements, implying a semantically atomic update to several +instruments. + +The preceding example could be rewritten: + +```golang +func (s *server) method(ctx context.Context) { + // ... other work + + labelSet := s.meter.Labels(...) + + // ... more work + + s.meter.RecordBatch(ctx, labelSet, []metric.Measurement{ + { s.instruments.counter1, 1 }, + { s.instruments.gauge1, 10 }, + { s.instruments.measure1, 100 }, + }) +} +``` + +Using the RecordBatch calling convention is identical to the sequence +of direct calls in the preceding example, only because the values are +entered in a single call to the SDK, the SDK is able to ensure that +a metrics exporter does not see a partial update. + +## Detailed specification + +See the [SDK-facing Metrics API](api-metrics-meter.md) specification +for an in-depth summary of each method in the Metrics API. + +### Instrument construction + +Instruments are constructed using the appropriate `New` method for the +kind of instrument (Counter, Gauge, Measure) and for the type of input +(integer or floating point). + +| `Meter` method | Kind of instrument | +|-------------------------------------|--------------------| +| `NewIntCounter(name, options...)` | An integer counter | +| `NewFloatCounter(name, options...)` | A floating point counter | +| `NewIntGauge(name, options...)` | An integer gauge | +| `NewFloatGauge(name, options...)` | A floating point gauge | +| `NewIntMeasure(name, options...)` | An integer measure | +| `NewFloatMeasure(name, options...)` | A floating point measure | +| `NewIntObserver(name, callback, options...)` | An integer observer | +| `NewFloatObserver(name, callback, options...)` | A floating point observer | + +As in all OpenTelemetry specifications, these names are examples. +Each language committee will decide on the appropriate names based on +conventions in that language. + +#### Recommended label keys + +Instruments may be defined with a recommended set of label keys. This +setting may be used by SDKs as a good default for grouping exported +metrics. Recommended label keys are usually selected by the developer +for exhibiting low cardinality. + +SDKs should consider grouping exported metric data by the recommended +label keys of each instrument, unless superceded by another form of +configuration. + +#### Instrument options + +Instruments provide several optional settings, summarized here. The +kind of instrument and input value type are implied by the constructor +that it used, and the metric name is the only required field. + +| Option | Option name | Explanation | +|-----------------------|--------------------| +| Description | WithDescription(string) | Descriptive text documenting the instrument. | +| Unit | WithUnit(string) | Units specified according to the [UCUM](http://unitsofmeasure.org/ucum.html). | +| Recommended label keys | WithRecommendedKeys(list) | Recommended grouping keys for this instrument | +| NonMonotonic | WithNonMonotonic(boolean) | Configure a counter that accepts negative updates | +| Monotonic | WithMonotonic(boolean) | Configure a gauge that accepts only monotonic increasing udpates | +| Signed | WithSigned(boolean) | Configure a measure that accepts positive and negative updates. | + +See the Metric API [specification overview](api-metrics.md) for more +information about the kind-specific monotonic, non-monotonic, and +signed options. + +### Metric Descriptor + +A metric instrument is completely described by its descriptor, through +options passed to the constructor. The complete contents of a metric +`Descriptor` are: + +- **Name** The unique name of this metric +- **Kind** An enumeration, one of `CounterKind`, `GaugeKind`, `ObserverKind`, or `MeasureKind` +- **Keys** The recommended label keys +- **Meter** A reference to its `Meter`, providing access to the `component` label for the instrument +- **ID** A unique identifier associated with new instruments +- **Description** A string describing the meaning and use of this instrument +- **Unit** The unit of measurement, optional +- _Kind-specific options_ + - **NonMonotonic** (Counter): add positive and negative values + - **Monotonic** (Gauge): set a monotonic counter value + - **Signed** (Measure): record positive and negative values + +### Instrument handle calling convention + +Counter, gauge, and measure instruments each support allocating +handles for the high-performance calling convention. The +`Instrument.GetHandle(LabelSet)` method returns an interface which +implements the `Add()`, `Set()` or `Record()` method, respectively, +for counter, gauge, and measure instruments. + +Instrument handles support a `Delete` method, allowing users to +discard handles that are no longer used. + +### Instrument direct calling convention + +Counter, gauge, and measure instruments support the appropriate +`Add()`, `Set()`, and `Record()` method for submitting individual +metric events. + +### Observer metric + +Observer metrics do not support handles or direct calls. The +`Meter.NewFloatObseerver` and `Meter.NewIntObserver` methods take +callbacks that generate measurements on demand, allowing the SDK to +arrange to call them only as needed, periodically, for an exporter. + +Observer instruments are automatically registered on creation, since +the corresponding `Meter` handles construction. From 2edc872dd70faf0d65ab1456a098cb24bfdf7fab Mon Sep 17 00:00:00 2001 From: jmacd Date: Fri, 11 Oct 2019 17:00:43 -0700 Subject: [PATCH 02/12] Minor fix for AloisReitbauer's feedback on PR250 --- specification/api-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/api-metrics.md b/specification/api-metrics.md index c7e9326051b..670e4650d6f 100644 --- a/specification/api-metrics.md +++ b/specification/api-metrics.md @@ -4,7 +4,7 @@ TODO: Table of contents ## Overview -The user-facing metrics API supports reporting diagnostic measurements +The user-facing metrics API supports producing diagnostic measurements using three basic kinds of instrument. "Metrics" are the thing being produced--mathematical, statistical summaries of certain observable behavior in the program. "Instruments" are the devices used by the From 599d2ac3348217db5f76b417196256256a7fc1a2 Mon Sep 17 00:00:00 2001 From: jmacd Date: Mon, 14 Oct 2019 14:41:36 -0700 Subject: [PATCH 03/12] Describe metrics event context; make use of global Meter recommended for allocating static metrics --- specification/api-metrics-user.md | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index b38d12ba9cc..6d220e5658f 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -47,12 +47,14 @@ Labels are key:value pairs associated with events describing various dimensions or categories that describe the event. A "label key" refers to the key component while "label value" refers to the correlated value component of a label. Label refers to the pair of -label key and value. +label key and value. Labels are passed in to the metric event in the +form of a `LabelSet` argument, using several input methods discussed +below. -Metric events always have an associated `component` label, by virtue -of the named `Meter` used in their definition. Other labels are -passed in to the metric event in the form of a `LabelSet` argument, -using several input methods discussed below. +Metric events always have an associated component name, the name +passed when constructing the corresponding `Meter`. Metric events are +associated with the current (implicit or explicit) OpenTelemetry +context, including distributed correlation context and span context. ### New constructors @@ -63,7 +65,7 @@ either floating point or integer inputs, see the detailed design below. Binding instruments to a single `Meter` instance has two benefits: -1. Instruments can be exported freom the zero state, prior to first use, with no explicit `Register` call +1. Instruments can be exported from the zero state, prior to first use, with no explicit `Register` call 1. The component name provided by the named `Meter` satisfies a namespace requirement The recommended practice is to define structures to contain the @@ -74,14 +76,16 @@ We recognize that many existing metric systems support allocating metric instruments statically and providing the `Meter` interface at the time-of-use. In this example, typical of statsd clients, existing code may not be structured with a convenient place to store new metric -instruments. Where this becomes a burden, it may be acceptable to use -the global `Meter` as a workaround. +instruments. Where this becomes a burden, it is recommended to use +the global meter factory to construct a static named `Meter` in order +to construct metric instruments. The situation is similar for users of Prometheus clients, where instruments are allocated statically and there is an implicit global. Such code may not have access to the appropriate `Meter` where -instruments are defined. Where this becomes a burden, it may be -acceptable to use the global `Meter` as a workaround. +instruments are defined. Where this becomes a burden, it is +recommended to use the global meter factory to construct a static +named `Meter` in order to construct metric instruments. #### Metric instrument descriptors From 12b941faa57afdb83f8d40830c18f3fcf665ee09 Mon Sep 17 00:00:00 2001 From: jmacd Date: Mon, 14 Oct 2019 23:11:07 -0700 Subject: [PATCH 04/12] Feedback part 1 --- specification/api-metrics-user.md | 101 ++++++++++++++++++------------ 1 file changed, 62 insertions(+), 39 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index 6d220e5658f..b9bafc568bb 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -74,40 +74,27 @@ are specifically needed. We recognize that many existing metric systems support allocating metric instruments statically and providing the `Meter` interface at -the time-of-use. In this example, typical of statsd clients, existing +the time of use. In this example, typical of statsd clients, existing code may not be structured with a convenient place to store new metric instruments. Where this becomes a burden, it is recommended to use -the global meter factory to construct a static named `Meter` in order -to construct metric instruments. +the global meter factory to construct a static named `Meter`, to +construct metric instruments. The situation is similar for users of Prometheus clients, where instruments are allocated statically and there is an implicit global. Such code may not have access to the appropriate `Meter` where instruments are defined. Where this becomes a burden, it is recommended to use the global meter factory to construct a static -named `Meter` in order to construct metric instruments. - -#### Metric instrument descriptors - -An instrument `Descriptor` is a structure describing all the -configurable aspects of an instrument that the user can elect. -Although the API provides a common `Descriptor` type--not the -SDK--users do not construct these directly. Users pass all common -configuration options to the appropriate `Meter.New` method, which -itself will use a helper method provided by the API to build the new -`Descriptor`. Users can access the descriptor of the built -instrument, in any case. +named `Meter`, to construct metric instruments. Applications are expected to construct long-lived instruments. -Instruments are considered permanent, there is no method to delete -them forget the metrics they produce in the SDK. The structure of the -descriptor and various options are given in detail below. +Instruments are considered permanent for the lifetime of a SDK, there +is no method to delete them. SDKs should #### Metric instrument constructor example code In this Golang example, a struct holding four instruments is built -using the provided `Meter`. These calls give examples of the kind of -configuration options available for descriptors as well. +using the provided, non-global `Meter` instance. ```golang type instruments struct { @@ -119,10 +106,10 @@ type instruments struct { func newInstruments(metric.Meter meter) *instruments { return &instruments{ - counter1: meter.NewCounter("counter1", metric.WithKeys("client.service")) - counter2: meter.NewCounter("counter2", metric.WithNonMonotonic(true)) - gauge3: meter.NewGauge("gauge3", metric.WithUnit(unit.Bytes)). - measure4: meter.NewMeasure("measure4", metric.WithDescription("Measure of ...")). + counter1: meter.NewCounter("counter1", ...), // Optional parameters + counter2: meter.NewCounter("counter2", ...), // are discussed below. + gauge3: meter.NewGauge("gauge3", ...), + measure4: meter.NewMeasure("measure4", ...), } } ``` @@ -168,14 +155,18 @@ measurements that are semantically equivalent and generate equivalent metric events, but offer varying degrees of performance and convenience. +This section applies to calling conventions for counter, gauge, and +measure instruments. Observer instruments, a special class of gauge +defined by callbacks, are discussed in a dedicated section below. + #### Metric handle calling convention As described above, metric events consist of an instrument, a set of -labels, and a numerical value. The performance of a metric API -depends on the work done to enter a new measurement. One approach to -reduce cost is to pre-aggregate results, so that subsequent events in -the same collection period for the same label set combine into the -same working memory. +labels, and a numerical value, plus associated context. The +performance of a metric API depends on the work done to enter a new +measurement. One approach to reduce cost is to pre-aggregate results, +so that subsequent events happening in the same collection period, for +the same label set, combine into the same working memory. This approach requires locating an entry for the instrument and label set in a table of some kind, finding the place where a group of metric @@ -186,23 +177,26 @@ In situations where performance is a requirement and a metric is repeatedly used with the same set of labels, the developer may elect to use _instrument handles_ as an optimization. For handles to be a benefit, it requires that a specific instrument will be re-used with -specific labels. +specific labels. If an instrument will be used with the same label +set more than once, obtaining an instrument handle corresponding to +the label set ensures the highest performance available. To obtain a handle given an instrument and label set, use the `GetHandle()` method to return an interface that supports the `Add()`, `Set()`, or `Record()` method of the instrument in question. -A high-performace metrics SDK will take steps to ensure that -operations on handles are very fast. Application developers are -required to delete handles when they are no-longer in use. +Instrument handles may consume SDK resources indefinitely. Handles +support a `Delete` method that will allow these resources to be +reclaimed after all the corresponding metric events have been +collected. ```golang func (s *server) processStream(ctx context.Context) { - streamLabels = []core.KeyValue{ + streamLabels := s.meter.Labels( labelA.String("..."), labelB.String("..."), - } + ) counter2Handle := s.instruments.counter2.GetHandle(streamLabels) for _, item := <-s.channel { @@ -219,8 +213,9 @@ func (s *server) processStream(ctx context.Context) { #### Direct metric calling convention When convenience is more important than performance, or there is no -re-use to potentially optimize, users may elect to operate directly on -metric instruments, supplying a label set at the call site. +re-use to potentially optimize with instrument handles, users may +elect to operate directly on metric instruments, supplying a label set +at the call site. For example, to update a single counter: @@ -236,8 +231,8 @@ This method offers the greatest convenience possible. If performance becomes a problem, one option is to use handles as described above. Another performance option, in some cases, is to just re-use the labels. In the example here, `meter.Labels(...)` constructs a -re-usable label set which may be a useful performance optimization, as -discussed next. +re-usable label set which may be an important performance +optimization, as discussed next. #### Label set calling convention @@ -262,6 +257,11 @@ not offer any optimization. It may be best to pre-compute a canonicalized `LabelSet` once and re-use it with the direct calling convention. +Constructing an instrument handle is considered the higher-performance +option, when the handle will be used more than once. Still, consider +re-using the result of `Meter.Labels(...)` when constructing more than +one instrument handle. + ```golang func (s *server) method(ctx context.Context) { // ... other work @@ -280,6 +280,29 @@ func (s *server) method(ctx context.Context) { } ``` +##### Option: Ordered LabelSet construction + +As a language-level decision, APIs may support _ordered_ LabelSet +construction, in which a pre-defined set of ordered label keys is +defined such that values can be supplied in order. For example, + +```golang + +var rpcLabelKeys = meter.OrderedLabelKeys("a", "b", "c") + +for _, input := range stream { + labels := rpcLabelKeys.Values(1, 2, 3) // a=1, b=2, c=3 + + // ... +} +``` + +This is specified as a language-optional feature because its safety, +and therefore its value as an input for monitoring, depends on the +availability of type-checking in the source language. Passing +unordered labels (i.e., a list of bound keys and values) to the +`Meter.Labels(...)` constructor is considered the safer alternative. + #### RecordBatch calling convention There is one final API for entering measurements, which is like the From c3dd27b088212c626220538219ffb8bf579f5eb2 Mon Sep 17 00:00:00 2001 From: jmacd Date: Tue, 15 Oct 2019 00:12:14 -0700 Subject: [PATCH 05/12] Feedback part 2 --- specification/api-metrics-user.md | 52 ++++++++++++++++++++++++++++--- 1 file changed, 48 insertions(+), 4 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index b9bafc568bb..e67e55dec35 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -329,10 +329,54 @@ func (s *server) method(ctx context.Context) { } ``` -Using the RecordBatch calling convention is identical to the sequence -of direct calls in the preceding example, only because the values are -entered in a single call to the SDK, the SDK is able to ensure that -a metrics exporter does not see a partial update. +Using the RecordBatch calling convention is semantically identical to +the sequence of direct calls in the preceding example, with the +addition of atomicity. Because values are entered in a single call, +the SDK is potentially able to implement an atomic update, from the +point-of-view of the exporter. + +#### Interaction with distrubuted correlation context + +The `LabelSet` type introduced above applies strictly to "local" +labels, meaning provided in a call to `meter.Labels(...)`. The +application explicitly declares these labels, whereas distributed +correlation context labels are implicitly associated with the event. + +There is a clear intention to pre-aggregate metrics within the SDK, +using the contents of a `LabelSet` to derive grouping keys. There are +two available options for users to apply distributed correlation +context to the local grouping function used for metrics +pre-aggregation: + +1. The distributed context, whether implicit or explicit, is +associated with every metric event. The SDK could _automatically_ +project selected label keys from the distributed correlation into the +metric event. This would require some manner of dynamic mapping from +`LabelSet` to grouping key during aggregation. +1. The user can explicitly perform the same projection of distributed +correlation into a `LabelSet` by extracting from the correlation +context and including it in the call to `metric.Labels(...)`. + +An example of an explicit projection follows. + +```golang +import "go.opentelemetry.io/api/distributedcontext" + +func (s *server) doThing(ctx context.Context) { + var doLabels []core.KeyValue{ + key1.String("..."), + key2.String("..."), + } + + correlations := distributedcontext.FromContext() + if val, ok := correlations.Value(key3); ok { + doLabels = append(doLabels, key3.Value(val)) + } + labels := s.meter.Labels(doLabels) + + // ... +} +``` ## Detailed specification From 4cca9297b88b38729b16e2a4c42fce976c06386e Mon Sep 17 00:00:00 2001 From: jmacd Date: Tue, 15 Oct 2019 00:35:35 -0700 Subject: [PATCH 06/12] Feedback part 3 --- specification/api-metrics-user.md | 127 ++++++++++++++---------------- 1 file changed, 60 insertions(+), 67 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index e67e55dec35..dae0902f8ff 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -280,6 +280,13 @@ func (s *server) method(ctx context.Context) { } ``` +##### Missing label keys + +When the SDK interprets a `LabelSet` in the context of aggregating +values for an exporter, and where there are grouping keys that are +missing, the SDK is required to consider these values _explicitly +unspecified_, a distinct value type of the exported data model. + ##### Option: Ordered LabelSet construction As a language-level decision, APIs may support _ordered_ LabelSet @@ -335,49 +342,6 @@ addition of atomicity. Because values are entered in a single call, the SDK is potentially able to implement an atomic update, from the point-of-view of the exporter. -#### Interaction with distrubuted correlation context - -The `LabelSet` type introduced above applies strictly to "local" -labels, meaning provided in a call to `meter.Labels(...)`. The -application explicitly declares these labels, whereas distributed -correlation context labels are implicitly associated with the event. - -There is a clear intention to pre-aggregate metrics within the SDK, -using the contents of a `LabelSet` to derive grouping keys. There are -two available options for users to apply distributed correlation -context to the local grouping function used for metrics -pre-aggregation: - -1. The distributed context, whether implicit or explicit, is -associated with every metric event. The SDK could _automatically_ -project selected label keys from the distributed correlation into the -metric event. This would require some manner of dynamic mapping from -`LabelSet` to grouping key during aggregation. -1. The user can explicitly perform the same projection of distributed -correlation into a `LabelSet` by extracting from the correlation -context and including it in the call to `metric.Labels(...)`. - -An example of an explicit projection follows. - -```golang -import "go.opentelemetry.io/api/distributedcontext" - -func (s *server) doThing(ctx context.Context) { - var doLabels []core.KeyValue{ - key1.String("..."), - key2.String("..."), - } - - correlations := distributedcontext.FromContext() - if val, ok := correlations.Value(key3); ok { - doLabels = append(doLabels, key3.Value(val)) - } - labels := s.meter.Labels(doLabels) - - // ... -} -``` - ## Detailed specification See the [SDK-facing Metrics API](api-metrics-meter.md) specification @@ -408,12 +372,15 @@ conventions in that language. Instruments may be defined with a recommended set of label keys. This setting may be used by SDKs as a good default for grouping exported -metrics. Recommended label keys are usually selected by the developer -for exhibiting low cardinality. +metrics, where used with pre-aggregation. The recommended label keys +are usually selected by the developer for exhibiting low cardinality, +importance for monitoring purposes, and _an intention to provide these +variables locally_. SDKs should consider grouping exported metric data by the recommended label keys of each instrument, unless superceded by another form of -configuration. +configuration. Recommended keys that are missing will be considered +explicitly unspecified, as for missing `LabelSet` keys in general. #### Instrument options @@ -425,33 +392,14 @@ that it used, and the metric name is the only required field. |-----------------------|--------------------| | Description | WithDescription(string) | Descriptive text documenting the instrument. | | Unit | WithUnit(string) | Units specified according to the [UCUM](http://unitsofmeasure.org/ucum.html). | -| Recommended label keys | WithRecommendedKeys(list) | Recommended grouping keys for this instrument | -| NonMonotonic | WithNonMonotonic(boolean) | Configure a counter that accepts negative updates | -| Monotonic | WithMonotonic(boolean) | Configure a gauge that accepts only monotonic increasing udpates | +| Recommended label keys | WithRecommendedKeys(list) | Recommended grouping keys for this instrument. | +| Monotonic | WithMonotonic(boolean) | Configure a counter or gauge that accepts only monotonic/non-monotonic updates. | | Signed | WithSigned(boolean) | Configure a measure that accepts positive and negative updates. | See the Metric API [specification overview](api-metrics.md) for more information about the kind-specific monotonic, non-monotonic, and signed options. -### Metric Descriptor - -A metric instrument is completely described by its descriptor, through -options passed to the constructor. The complete contents of a metric -`Descriptor` are: - -- **Name** The unique name of this metric -- **Kind** An enumeration, one of `CounterKind`, `GaugeKind`, `ObserverKind`, or `MeasureKind` -- **Keys** The recommended label keys -- **Meter** A reference to its `Meter`, providing access to the `component` label for the instrument -- **ID** A unique identifier associated with new instruments -- **Description** A string describing the meaning and use of this instrument -- **Unit** The unit of measurement, optional -- _Kind-specific options_ - - **NonMonotonic** (Counter): add positive and negative values - - **Monotonic** (Gauge): set a monotonic counter value - - **Signed** (Measure): record positive and negative values - ### Instrument handle calling convention Counter, gauge, and measure instruments each support allocating @@ -469,6 +417,49 @@ Counter, gauge, and measure instruments support the appropriate `Add()`, `Set()`, and `Record()` method for submitting individual metric events. +### Interaction with distrubuted correlation context + +The `LabelSet` type introduced above applies strictly to "local" +labels, meaning provided in a call to `meter.Labels(...)`. The +application explicitly declares these labels, whereas distributed +correlation context labels are implicitly associated with the event. + +There is a clear intention to pre-aggregate metrics within the SDK, +using the contents of a `LabelSet` to derive grouping keys. There are +two available options for users to apply distributed correlation +context to the local grouping function used for metrics +pre-aggregation: + +1. The distributed context, whether implicit or explicit, is +associated with every metric event. The SDK could _automatically_ +project selected label keys from the distributed correlation into the +metric event. This would require some manner of dynamic mapping from +`LabelSet` to grouping key during aggregation. +1. The user can explicitly perform the same projection of distributed +correlation into a `LabelSet` by extracting from the correlation +context and including it in the call to `metric.Labels(...)`. + +An example of an explicit projection follows. + +```golang +import "go.opentelemetry.io/api/distributedcontext" + +func (s *server) doThing(ctx context.Context) { + var doLabels []core.KeyValue{ + key1.String("..."), + key2.String("..."), + } + + correlations := distributedcontext.FromContext() + if val, ok := correlations.Value(key3); ok { + doLabels = append(doLabels, key3.Value(val)) + } + labels := s.meter.Labels(doLabels) + + // ... +} +``` + ### Observer metric Observer metrics do not support handles or direct calls. The @@ -478,3 +469,5 @@ arrange to call them only as needed, periodically, for an exporter. Observer instruments are automatically registered on creation, since the corresponding `Meter` handles construction. + +TODO \ No newline at end of file From 26cfbb7bfe7a484aa5c3b26774884fbc9587f69e Mon Sep 17 00:00:00 2001 From: jmacd Date: Tue, 15 Oct 2019 00:38:52 -0700 Subject: [PATCH 07/12] Feedback part 4 --- specification/api-metrics-user.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index dae0902f8ff..879ef12cf5f 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -39,8 +39,8 @@ prefix](https://prometheus.io/docs/practices/naming/#metric-names). ### Format of a metric event Regardless of the instrument kind or method of input, metric events -include the instrument descriptor, a numerical value, and an optional -set of labels. The descriptor, discussed in detail below, contains +include the instrument, a numerical value, and an optional +set of labels. The instrument, discussed in detail below, contains the metric name and various optional settings. Labels are key:value pairs associated with events describing various From 30693b49d91b3381cac2e1be8e8bbc8a91739ecb Mon Sep 17 00:00:00 2001 From: jmacd Date: Tue, 15 Oct 2019 13:18:49 -0700 Subject: [PATCH 08/12] Revert api-metrics.md fix --- specification/api-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/api-metrics.md b/specification/api-metrics.md index 670e4650d6f..c7e9326051b 100644 --- a/specification/api-metrics.md +++ b/specification/api-metrics.md @@ -4,7 +4,7 @@ TODO: Table of contents ## Overview -The user-facing metrics API supports producing diagnostic measurements +The user-facing metrics API supports reporting diagnostic measurements using three basic kinds of instrument. "Metrics" are the thing being produced--mathematical, statistical summaries of certain observable behavior in the program. "Instruments" are the devices used by the From 84865af59d53ade6e19048d100a23d449a07c69f Mon Sep 17 00:00:00 2001 From: jmacd Date: Tue, 15 Oct 2019 18:09:01 -0700 Subject: [PATCH 09/12] Comment on convenience overload to bypass LabelSet --- specification/api-metrics-user.md | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index 879ef12cf5f..b5c2400f536 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -89,7 +89,7 @@ named `Meter`, to construct metric instruments. Applications are expected to construct long-lived instruments. Instruments are considered permanent for the lifetime of a SDK, there -is no method to delete them. SDKs should +is no method to delete them. #### Metric instrument constructor example code @@ -232,7 +232,7 @@ becomes a problem, one option is to use handles as described above. Another performance option, in some cases, is to just re-use the labels. In the example here, `meter.Labels(...)` constructs a re-usable label set which may be an important performance -optimization, as discussed next. +optimization. #### Label set calling convention @@ -287,6 +287,27 @@ values for an exporter, and where there are grouping keys that are missing, the SDK is required to consider these values _explicitly unspecified_, a distinct value type of the exported data model. +##### Option: Convenience method to bypass `meter.Labels(...)` + +As a language-optional feature, the direct and handle calling +convention APIs may support alternate convenience methods to pass raw +labels at the call site. These may be offered as overloaded methods +for `Add()`, `Set()`, and `Record()` (direct calling convention) or +`GetHandle()` (handle calling convention), in both cases bypassing a +call to `meter.Labels(...)`. For example: + +```java + public void method() { + // pass raw labels, no explicit `LabelSet` + s.instruments.counter1.add(1, labelA.value(...), labelB.value(...)) + + // ... or + + // pass raw labels, no explicit `LabelSet` + handle := s.instruments.gauge1.getHandle(labelA.value(...), labelB.value(...)) + } +``` + ##### Option: Ordered LabelSet construction As a language-level decision, APIs may support _ordered_ LabelSet @@ -417,7 +438,7 @@ Counter, gauge, and measure instruments support the appropriate `Add()`, `Set()`, and `Record()` method for submitting individual metric events. -### Interaction with distrubuted correlation context +### Interaction with distributed correlation context The `LabelSet` type introduced above applies strictly to "local" labels, meaning provided in a call to `meter.Labels(...)`. The From 1342cdc714a94620ae0cf8189f47aeff9f1b6e6e Mon Sep 17 00:00:00 2001 From: jmacd Date: Tue, 15 Oct 2019 18:10:39 -0700 Subject: [PATCH 10/12] Change signed/non-negative to absolute --- specification/api-metrics-user.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index b5c2400f536..cf2e45130fc 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -415,11 +415,10 @@ that it used, and the metric name is the only required field. | Unit | WithUnit(string) | Units specified according to the [UCUM](http://unitsofmeasure.org/ucum.html). | | Recommended label keys | WithRecommendedKeys(list) | Recommended grouping keys for this instrument. | | Monotonic | WithMonotonic(boolean) | Configure a counter or gauge that accepts only monotonic/non-monotonic updates. | -| Signed | WithSigned(boolean) | Configure a measure that accepts positive and negative updates. | +| Absolute | WithAbsolute(boolean) | Configure a measure that does or does not accept negative updates. | See the Metric API [specification overview](api-metrics.md) for more -information about the kind-specific monotonic, non-monotonic, and -signed options. +information about the kind-specific monotonic and absolute options. ### Instrument handle calling convention From de28b1f85cce810e58594d4c957128aafa830361 Mon Sep 17 00:00:00 2001 From: jmacd Date: Tue, 15 Oct 2019 23:28:07 -0700 Subject: [PATCH 11/12] Remove observer instrument from this spec for v0.2 --- specification/api-metrics-meter.md | 7 ++-- specification/api-metrics-user.md | 62 +++++++++++++++--------------- 2 files changed, 34 insertions(+), 35 deletions(-) diff --git a/specification/api-metrics-meter.md b/specification/api-metrics-meter.md index f43b780a557..e82c3acdd71 100644 --- a/specification/api-metrics-meter.md +++ b/specification/api-metrics-meter.md @@ -1,6 +1,5 @@ # Metric SDK-facing API -This document is a placeholder pending active discussion. It will -eventually contain portions that were removed from -https://github.com/open-telemetry/opentelemetry-specification/blob/3d2b8ecf410f62172a22a9fbff88304724d4cc78/specification/api-metrics.md -after further discussion. +This document will be updated as part of the v0.2 milestone with a +detailed list of `Meter` API methods. These methods will match the +user-facing API specified [here](api-metrics-user.md). \ No newline at end of file diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index cf2e45130fc..7ed00f7bba3 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -1,6 +1,11 @@ # Metric User-facing API -TODO Table of contents. +Note: This specification for the v0.2 OpenTelemetry milestone does not +cover the Observer gauge instrument discussed in the +[overview](api-metrics.md). Observer instruments will be added in the +v0.3 milestone. + +TODO: Add a table of contents. ## Overview @@ -156,20 +161,27 @@ metric events, but offer varying degrees of performance and convenience. This section applies to calling conventions for counter, gauge, and -measure instruments. Observer instruments, a special class of gauge -defined by callbacks, are discussed in a dedicated section below. - -#### Metric handle calling convention +measure instruments. As described above, metric events consist of an instrument, a set of labels, and a numerical value, plus associated context. The performance of a metric API depends on the work done to enter a new -measurement. One approach to reduce cost is to pre-aggregate results, -so that subsequent events happening in the same collection period, for -the same label set, combine into the same working memory. +measurement. One approach to reduce cost is to aggregate intermediate +results in the SDK, so that subsequent events happening in the same +collection period, for the same label set, combine into the same +working memory. + +In this document, the term "aggregation" is used to describe the +process of coalescing metric events for a complete set of labels, +whereas "grouping" is used to describe further coalescing aggregate +metric data into a reduced number of key dimensions. SDKs may be +designed to perform aggregation and/or grouping in the process, with +various trade-offs in terms of complexity and performance. + +#### Metric handle calling convention This approach requires locating an entry for the instrument and label -set in a table of some kind, finding the place where a group of metric +set in a table of some kind, finding the location where a metric events are being aggregated. This lookup can be successfully precomputed, giving rise to the Handle calling convention. @@ -282,8 +294,8 @@ func (s *server) method(ctx context.Context) { ##### Missing label keys -When the SDK interprets a `LabelSet` in the context of aggregating -values for an exporter, and where there are grouping keys that are +When the SDK interprets a `LabelSet` in the context of grouping +aggregated values for an exporter, and where there are keys that are missing, the SDK is required to consider these values _explicitly unspecified_, a distinct value type of the exported data model. @@ -349,11 +361,11 @@ func (s *server) method(ctx context.Context) { // ... more work - s.meter.RecordBatch(ctx, labelSet, []metric.Measurement{ - { s.instruments.counter1, 1 }, - { s.instruments.gauge1, 10 }, - { s.instruments.measure1, 100 }, - }) + s.meter.RecordBatch(ctx, labelSet, + s.instruments.counter1.Measurement(1), + s.instruments.gauge1.Measurement(10), + s.instruments.measure2.Measurement(123.45), + ) } ``` @@ -361,7 +373,9 @@ Using the RecordBatch calling convention is semantically identical to the sequence of direct calls in the preceding example, with the addition of atomicity. Because values are entered in a single call, the SDK is potentially able to implement an atomic update, from the -point-of-view of the exporter. +exporter's point of view. Calls to `RecordBatch` may potentially +reduce costs because the SDK can enqueue a single bulk update, or take +a lock only once, for example. ## Detailed specification @@ -382,8 +396,6 @@ kind of instrument (Counter, Gauge, Measure) and for the type of input | `NewFloatGauge(name, options...)` | A floating point gauge | | `NewIntMeasure(name, options...)` | An integer measure | | `NewFloatMeasure(name, options...)` | A floating point measure | -| `NewIntObserver(name, callback, options...)` | An integer observer | -| `NewFloatObserver(name, callback, options...)` | A floating point observer | As in all OpenTelemetry specifications, these names are examples. Each language committee will decide on the appropriate names based on @@ -479,15 +491,3 @@ func (s *server) doThing(ctx context.Context) { // ... } ``` - -### Observer metric - -Observer metrics do not support handles or direct calls. The -`Meter.NewFloatObseerver` and `Meter.NewIntObserver` methods take -callbacks that generate measurements on demand, allowing the SDK to -arrange to call them only as needed, periodically, for an exporter. - -Observer instruments are automatically registered on creation, since -the corresponding `Meter` handles construction. - -TODO \ No newline at end of file From adc68a32ddfc32e688cb31b511cc4fe15071e335 Mon Sep 17 00:00:00 2001 From: jmacd Date: Thu, 17 Oct 2019 10:03:04 -0700 Subject: [PATCH 12/12] Remove Delete for 0.2 --- specification/api-metrics-user.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/specification/api-metrics-user.md b/specification/api-metrics-user.md index 7ed00f7bba3..3fdcdcd4514 100644 --- a/specification/api-metrics-user.md +++ b/specification/api-metrics-user.md @@ -197,10 +197,7 @@ To obtain a handle given an instrument and label set, use the `GetHandle()` method to return an interface that supports the `Add()`, `Set()`, or `Record()` method of the instrument in question. -Instrument handles may consume SDK resources indefinitely. Handles -support a `Delete` method that will allow these resources to be -reclaimed after all the corresponding metric events have been -collected. +Instrument handles may consume SDK resources indefinitely. ```golang func (s *server) processStream(ctx context.Context) { @@ -217,8 +214,6 @@ func (s *server) processStream(ctx context.Context) { // High-performance metric calling convention: use of handles. counter2Handle.Add(ctx, item.size()) } - - counter2Handle.Delete() } ``` @@ -440,9 +435,6 @@ handles for the high-performance calling convention. The implements the `Add()`, `Set()` or `Record()` method, respectively, for counter, gauge, and measure instruments. -Instrument handles support a `Delete` method, allowing users to -discard handles that are no longer used. - ### Instrument direct calling convention Counter, gauge, and measure instruments support the appropriate