Exporting metrics to backends with strict schemas #703
Labels
area:sdk
Related to the SDK
release:after-ga
Not required before GA release, and not going to work on before GA
spec:metrics
Related to the specification/metrics directory
What are you trying to achieve?
Writing a metrics exporter for a telemetry backend where exported metrics must match a schema (Google Cloud Monitoring). If the instrument is updated, metrics will fail to write because they won't match the schema.
Additional context.
Google Cloud Monitoring uses schemas for metrics and their labels called MetricDescriptors. If data written to a metric doesn't match this schema, the backend will raise an error. For example, writing a metric that is missing a label or has an extra label will fail. If an instrument is updated (e.g. adding labels, changing the value kind), the exporter needs to be able to detect this and create a new metric with a new MetricDescriptor, otherwise the backend will drop the data.
Potential Solutions
Versioning instrument names - Prefix metric names with a version string like
v1.*
and bump the version when the instrument is updated. This can be done already, but I think a mention in Metric naming conventions oteps#108 would be warranted. The standard instruments proposed in Standard system metrics and semantic conventions oteps#119 would also need version prefixes. One downside is that the user must remember to bump the version.Versioning instruments as part of spec - This would be the same as 1. except version would be an additional field when creating an instrument instead of a prefix in the name.
The exporter could compute a hash of the fields it uses to create a MetricDescriptor (label keys, label types, instrument type, value type, aggregator, maybe more) and use that hash in the metric name, e.g.
custom.googleapis.com/{hash}/{instrument_name}
.This approach is automatic. If there are other backends with strict schemas besides Cloud Monitoring, then it might be worth having this as part of the SDK that can be configured to pass the hash to exporters?
@james-bebbington @AndrewAXue
The text was updated successfully, but these errors were encountered: