Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added external label functionality to prw exporter #27

Closed
wants to merge 4 commits into from
Closed

added external label functionality to prw exporter #27

wants to merge 4 commits into from

Conversation

amanbrar1999
Copy link

Description:
This PR adds the concept of Prometheus "external labels" to the Prometheus Remote Write Exporter. These labels must be kept isolated from OTLP metric data point labels because they do not undergo the same type of sanitization. The difference is external labels are allowed to start with "__". This satisfies the use case of Cortex HA deduplication and hence resolves the linked issue.

Link to tracking Issue: open-telemetry#1892

Testing:
Unit test cases have been added wherever changes were made and manual testing was done to verify Cortex HA deduplication and that nothing changed in metric graphs other than labels.

Documentation:
The example configuration in testdata/config.yaml demonstrates how to use the external label config

@amanbrar1999 amanbrar1999 marked this pull request as ready for review October 30, 2020 14:08
JasonXZLiu
JasonXZLiu reviewed Oct 30, 2020
View reviewed changes
exporter/prometheusremotewriteexporter/helper.go
name := el.Key
// The reserved prefix "__" is allowed in an external label
if name[:2] == "__" {
name = "__" + sanitize(name[2:])
Copy link

@JasonXZLiu JasonXZLiu Oct 30, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's being sanitized here? I'm not sure if we should be altering the external labels as they act as a global override. We can just specify to the user what's an allowed external label.

Copy link
Author

@amanbrar1999 amanbrar1999 Oct 30, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how external labels are handled in Prometheus. Some sanitization still must be done, for example if the external label entered by the user is "__host.name__", "." is not a valid character in a prometheus label

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That being said, external labels are not quite an override for all restrictions on a label, just for the "__" restriction

JasonXZLiu
JasonXZLiu reviewed Oct 30, 2020
View reviewed changes
Copy link

@JasonXZLiu JasonXZLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably want to update the README for the RWExporter just to say this functionality exists and how to use it.

alolita
alolita approved these changes Oct 31, 2020
View reviewed changes
Copy link

@alolita alolita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to update readme usage else #lgtm

@amanbrar1999
Copy link
Author

PR has been made upstrean, so closing this

JamesJHPark pushed a commit that referenced this pull request Nov 3, 2021
Update to collector v0.23.0 (#27)
3d80039
JamesJHPark pushed a commit that referenced this pull request Nov 3, 2021
@jpkrohling @bogdandrutu
@joe-elliott @jiwen624 @Ashmita152 @fangyi-zhou @crearys @pmalek-sumo @secat @sodabrew @pmalek @KKelvinLo @himanshu007-creator @u5surf @zchee @Aneurysm9 @loomis-relativity
Move opentelemetry-collector-builder to core (open-telemetry#4286)
4063a46 
* Initial commit

* Add CODEOWNERS file (#2)

* Add CODEOWNERS file

* Update CODEOWNERS

* Moved from github.com/observatorium/opentelemetry-collector-builder (#3)

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* fixed panics (#6)

Signed-off-by: Joe Elliott <number101010@gmail.com>

* Replace master with main in CI and mergify files (#8)

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Bump to OpenTelemetry Collector 0.20.0 (#10)

Closes #9

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Explicitly enable Go modules in quickstart instructions (#13)

* Update to collector v0.21.0 (#17)

Fixes #16

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Update to collector v0.22.0 (#19)

* Download go modules before building (#20)

Fixes #14

* Add version command (#25)

Signed-off-by: Ashmita Bohara <ashmita.bohara152@gmail.com>

* Pass errors from cobra Execute back to main for correct exit code (#28)

* pass errors from cobra execute back to main

* print the error

* Update to collector v0.23.0 (#27)

* Generate a warning if the builder and collector base version mismatch (#30)

* Generate a warning if the builder and collector base version mismatch

* Show current default version in the warning message

* Update to OpenTelemetry Collector 0.24.0

* Don't use %w formatting with log.Fatal (#35)

* Update to OpenTelemetry Collector 0.25.0 (#36)

Signed-off-by: Serge Catudal <serge.catudal@gmail.com>

* Update to 0.26.0 and update BuildInfo (#39)

* Sync build and CI Go versions at latest 1.16 (#34)

* Sync build and CI Go versions at latest 1.16

* Run go mod tidy

* Set go binary to use in the compilation phase in tests

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

Co-authored-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Add option to generate go code only (no compile) (#40)

* Issue#24 Add option to generate go code only (no compile)

* Update cmd/root.go logging

Suggested by @jpkkrohling

Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>

* remove verbose help .. created by corba

* suggestion by jpkrohling to keep generateandcompile

* lint error: remove unused var

* reword cmd option and add back help message for default

Co-authored-by: Juraci Paixão Kröhling <juraci.github@kroehling.de>

* Don't reuse exec.Cmd (#42)

* Update to OpenTelemetry Collector 0.27.0 (#43)

* Add CI Badge (#47)

* Update to Collector v0.28.0 (#49)

* Update to Collector v0.28.0

Closes #48

Addresses the breaking API change in
open-telemetry#3163,
besides the usual version number changes.

Signed-off-by: Fangyi Zhou <me@fangyi.io>

* Use `go mod tidy` instead of `go mod download`

It appears that this magically resolves the go.mod file issue.
https://stackoverflow.com/questions/67203641/missing-go-sum-entry-for-module-providing-package-package-name

Signed-off-by: Fangyi Zhou <me@fangyi.io>

* Account for go mod download in go1.17 not updating go.sum (#50)

* Update to collector v0.29.0 (#54)

* Update replaces.builder.yaml

* Update nocore.builder.yaml

* Update config.go

* Update README.md

* Update main.go

* Update to collector v0.30.0 (#57)

* cmd: fix module flag default value to github.com/open-telemetry (#58)

Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>

* Update to collector v0.31.0 (#60)

* Update to v0.33.0 (#62)

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

* Add excludes support to generated go.mod (#63)

Signed-off-by: Anthony J Mirabella <a9@aneurysm9.com>

Co-authored-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Small cleanup for the builder files (#64)

Signed-off-by: Bogdan Drutu <bogdandrutu@gmail.com>

* Support building with Go 1.17 (#66)

* Support building with Go 1.17
Fixes #65

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Update workflows to use Go 1.17

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Add gosec exceptions for exec.Command

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Update to OpenTelemetry core 0.34.0 (#68)

Fixes #67

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Upgrade to OpenTelemetry Collector 0.35.0 (#70)

Signed-off-by: Fangyi Zhou <me@fangyi.io>

* Upgrade to OpenTelemetry Collector 0.36.0 (#76)

* Generate custom service code for Windows (#75)

* update main to include windows service code

* use main version from tag 0.35.0

* update main function

* align with upstream v0.36.0 tag

* dummy change to trigger build

* Revert "dummy change to trigger build"

This reverts commit 629d499461da2d2c240bf1e495b5fe0558e3547f.

* Remove Core from Module type (#77)

Fixes #15

Signed-off-by: yugo-horie <u5.horie@gmail.com>

* release 0.37.0 (#78)

* release 0.37.0

* update use of NewCommand

* Move builder to subdirectory

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

Co-authored-by: Bogdan Drutu <lazy@splunk.com>
Co-authored-by: Bogdan Drutu <bogdandrutu@gmail.com>
Co-authored-by: Joe Elliott <joe.elliott@grafana.com>
Co-authored-by: Eric Yang <jiwen624@gmail.com>
Co-authored-by: Brian Gibbins <eroteme@supernought.co.uk>
Co-authored-by: Ashmita <ashmita.bohara152@gmail.com>
Co-authored-by: Fangyi Zhou <me@fangyi.io>
Co-authored-by: Shaun Creary <65406540+crearys@users.noreply.github.com>
Co-authored-by: Patryk Małek <69143962+pmalek-sumo@users.noreply.github.com>
Co-authored-by: Serge Catudal <serge.catudal@gmail.com>
Co-authored-by: Aaron Stone <aaron@serendipity.cx>
Co-authored-by: Patryk Małek <pmalek@sumologic.com>
Co-authored-by: Aaron Stone <aaron.stone@udacity.com>
Co-authored-by: Kelvin Lo <kello@live.ca>
Co-authored-by: Himanshu <addyjeridiq@gmail.com>
Co-authored-by: Y.Horie <u5.horie@gmail.com>
Co-authored-by: Koichi Shiraishi <zchee.io@gmail.com>
Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
Co-authored-by: Cal Loomis <68860480+loomis-relativity@users.noreply.github.com>
Co-authored-by: alrex <aboten@lightstep.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JasonXZLiu JasonXZLiu JasonXZLiu approved these changes

@alolita alolita alolita approved these changes

@alvinlin123 alvinlin123 Awaiting requested review from alvinlin123

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@amanbrar1999 @alolita @JasonXZLiu