Skip to content
This repository was archived by the owner on Aug 23, 2023. It is now read-only.

remove tilde from name values when indexing tags #1371

Merged
merged 2 commits into from
Jul 5, 2019

Conversation

replay
Copy link
Contributor

@replay replay commented Jun 26, 2019

I'll wait for this PR to get merged before this can be merged: raintank/schema@80d1297

This is related to grafana/grafana#15261
And it is the implementation of graphite-project/graphite-web#2458

// used as a tag value. This is important when we index
// metric names as values of the tag "name"
func SanitizeNameAsTagValue(name string) string {
if !strings.Contains(name, "~") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why all the complexity in this function. Cant you just use strings.ReplaceAll(name, "~", "")?

Copy link
Contributor Author

@replay replay Jun 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely can, but since this is a function that may get called very often (potentially millions of times in one query) I wanted to keep it as efficient as possible. let me benchmark which version is more efficient

Copy link
Contributor Author

@replay replay Jun 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the definition of what is valid has just changed, so this function will get updated anyway
graphite-project/carbon#858 (comment)

Copy link
Contributor Author

@replay replay Jun 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a new version. I benchmarked it and compared it with an alternative version which is simpler by relying on strings:

func SanitizeNameAsTagValue(name string) (string, error) {
	if len(name) == 0 {
		return name, fmt.Errorf("Invalid name of length 0")
	}

	if name[0] != '~' {
		return name, nil
	}

	name = strings.TrimLeft(name, "~")

	if len(name) == 0 {
		return "", fmt.Errorf("Invalid name, requiring other characters than ~")
	}

	return name, nil
}

The first one is with the latest commit that I pushed (raintank/schema@f065be1) and the second is with the above simplified version:

replay@vbox:~/go/src/github.com/raintank/schema$ go test -run=none -bench=BenchmarkNameSanitized
goos: linux
goarch: amd64
pkg: github.com/raintank/schema
BenchmarkNameSanitizedAsTagValueWithValidValue-3        300000000                5.02 ns/op
BenchmarkNameSanitizedAsTagValueWithInvalidValue-3      200000000                6.10 ns/op
PASS
ok      github.com/raintank/schema      3.867s
replay@vbox:~/go/src/github.com/raintank/schema$ go test -run=none -bench=BenchmarkNameSanitized
goos: linux
goarch: amd64
pkg: github.com/raintank/schema
BenchmarkNameSanitizedAsTagValueWithValidValue-3        300000000                5.09 ns/op
BenchmarkNameSanitizedAsTagValueWithInvalidValue-3      20000000                84.8 ns/op
PASS
ok      github.com/raintank/schema      3.810s

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I don't think it makes sense for this method to have the potential to return errors, if the returned value is "" then the caller should just have to deal with it, i'll change that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it again so the method does not return errors and compared the latest version (raintank/schema@1adcebf) with a simpler version relying on strings. it's quite a difference:

replay@vbox:~/go/src/github.com/raintank/schema$ go test -run=none -bench=BenchmarkNameSanitized
goos: linux
goarch: amd64
pkg: github.com/raintank/schema
BenchmarkNameSanitizedAsTagValueWithValidValue-3        300000000                4.41 ns/op
BenchmarkNameSanitizedAsTagValueWithInvalidValue-3      300000000                5.74 ns/op
PASS
ok      github.com/raintank/schema      4.095s
replay@vbox:~/go/src/github.com/raintank/schema$ go test -run=none -bench=BenchmarkNameSanitized
goos: linux
goarch: amd64
pkg: github.com/raintank/schema
BenchmarkNameSanitizedAsTagValueWithValidValue-3        20000000                70.4 ns/op
BenchmarkNameSanitizedAsTagValueWithInvalidValue-3      20000000                82.0 ns/op
PASS
ok      github.com/raintank/schema      3.220s

@replay replay force-pushed the remove_tilde_when_indexing_names_as_tag_values branch from b38778c to 323e4de Compare July 4, 2019 20:59
@replay replay force-pushed the remove_tilde_when_indexing_names_as_tag_values branch from 323e4de to d1368da Compare July 4, 2019 21:06
@replay replay changed the title [WIP] remove tilde from name values when indexing tags remove tilde from name values when indexing tags Jul 4, 2019
@replay
Copy link
Contributor Author

replay commented Jul 4, 2019

This should be ready to review now

Copy link
Contributor

@Dieterbe Dieterbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nits, lgtm

@replay replay force-pushed the remove_tilde_when_indexing_names_as_tag_values branch from c2f0a0f to 23d3f14 Compare July 5, 2019 12:34
@replay replay merged commit b7ab32f into master Jul 5, 2019
@Dieterbe Dieterbe deleted the remove_tilde_when_indexing_names_as_tag_values branch July 5, 2019 13:44
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants