Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exporting metrics to backends with strict schemas #703

Closed
aabmass opened this issue Jul 14, 2020 · 2 comments
Closed

Exporting metrics to backends with strict schemas #703

aabmass opened this issue Jul 14, 2020 · 2 comments
Labels
area:sdk Related to the SDK release:after-ga Not required before GA release, and not going to work on before GA spec:metrics Related to the specification/metrics directory

Comments

@aabmass
Copy link
Member

aabmass commented Jul 14, 2020

What are you trying to achieve?
Writing a metrics exporter for a telemetry backend where exported metrics must match a schema (Google Cloud Monitoring). If the instrument is updated, metrics will fail to write because they won't match the schema.

Additional context.
Google Cloud Monitoring uses schemas for metrics and their labels called MetricDescriptors. If data written to a metric doesn't match this schema, the backend will raise an error. For example, writing a metric that is missing a label or has an extra label will fail. If an instrument is updated (e.g. adding labels, changing the value kind), the exporter needs to be able to detect this and create a new metric with a new MetricDescriptor, otherwise the backend will drop the data.

Potential Solutions

  1. Versioning instrument names - Prefix metric names with a version string like v1.* and bump the version when the instrument is updated. This can be done already, but I think a mention in Metric naming conventions oteps#108 would be warranted. The standard instruments proposed in Standard system metrics and semantic conventions oteps#119 would also need version prefixes. One downside is that the user must remember to bump the version.

  2. Versioning instruments as part of spec - This would be the same as 1. except version would be an additional field when creating an instrument instead of a prefix in the name.

  3. The exporter could compute a hash of the fields it uses to create a MetricDescriptor (label keys, label types, instrument type, value type, aggregator, maybe more) and use that hash in the metric name, e.g. custom.googleapis.com/{hash}/{instrument_name}.

    This approach is automatic. If there are other backends with strict schemas besides Cloud Monitoring, then it might be worth having this as part of the SDK that can be configured to pass the hash to exporters?

@james-bebbington @AndrewAXue

@aabmass aabmass added the spec:metrics Related to the specification/metrics directory label Jul 14, 2020
@jkwatson
Copy link
Contributor

I'd be strongly opposed to option 1. Metric names should not be polluted with version numbers.

@justinfoote
Copy link
Member

This feels like a matter of massaging the OpenTelemetry data to match a specific vendor's backend, so I think the logic belongs squarely in the exporter.
I'd vote for number 3.

@carlosalberto carlosalberto added area:sdk Related to the SDK release:after-ga Not required before GA release, and not going to work on before GA labels Jul 23, 2020
@aabmass aabmass closed this as completed Aug 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:sdk Related to the SDK release:after-ga Not required before GA release, and not going to work on before GA spec:metrics Related to the specification/metrics directory
Projects
None yet
Development

No branches or pull requests

4 participants