Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sanitize names when using them as tag value #858

Conversation

replay
Copy link
Contributor

@replay replay commented Jun 27, 2019

@codecov-io
Copy link

codecov-io commented Jun 27, 2019

Codecov Report

Merging #858 into master will decrease coverage by <.01%.
The diff coverage is 48%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #858      +/-   ##
==========================================
- Coverage   50.15%   50.15%   -0.01%     
==========================================
  Files          37       37              
  Lines        3483     3505      +22     
  Branches      495      501       +6     
==========================================
+ Hits         1747     1758      +11     
- Misses       1625     1632       +7     
- Partials      111      115       +4
Impacted Files Coverage Δ
lib/carbon/util.py 49.82% <48%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8af0b4f...3195cdd. Read the comment docs.

@DanCech
Copy link
Member

DanCech commented Jun 27, 2019

The biggest concern here is the impact that this will have on anyone who is currently using series with a ~ character in the name, since those series will be changed. We could reduce that impact if we reduce the scope here to only stripping leading ~ signs, since technically that is the only place that it actually impacts the tagging system.

@replay
Copy link
Contributor Author

replay commented Jun 27, 2019

Yeah, we could do that. Then i think I should update the docs accordingly, and update the validation function in raintank/schema as well to keep everything consistent

@replay
Copy link
Contributor Author

replay commented Jun 27, 2019

Another question: What do we do if a metric name is exactly ~ and nothing else?
We have a rule that says tag values must be >=1 char long in https://github.com/graphite-project/graphite-web/blob/master/docs/tags.rst#carbon
We could just add it to the index with tag name="", but that would only be queriable via regex because the query name= has another meaning.
In the case of Metrictank we could also consider that metric to have "invalid tags", and there's already a setting to define whether metrics with invalid tags should still get indexed via their other tags and the name or not, but in the case of carbon I'm not sure if there's a nice way to reject a metric name.

@DanCech
Copy link
Member

DanCech commented Jun 27, 2019

The only real option available here would be to drop it as invalid afaik

@replay
Copy link
Contributor Author

replay commented Jun 27, 2019

@replay replay force-pushed the sanitize_names_when_indexing_as_tag_value branch 2 times, most recently from 44b21c5 to d700f69 Compare June 27, 2019 18:28
@replay replay changed the title [WIP] sanitize names when using them as tag value sanitize names when using them as tag value Aug 16, 2019
@replay replay force-pushed the sanitize_names_when_indexing_as_tag_value branch from d700f69 to 15da096 Compare August 16, 2019 14:54
@replay
Copy link
Contributor Author

replay commented Aug 16, 2019

Currently, if the resulting string is "", it would raise an Exception which gets caught here:

metric = TaggedSeries.parse(metric).path

So it would ignore the metric and log an error. I think that's probably what we want

@replay replay force-pushed the sanitize_names_when_indexing_as_tag_value branch from 15da096 to dc86a7c Compare August 16, 2019 15:06
@replay replay force-pushed the sanitize_names_when_indexing_as_tag_value branch from a7cbbde to 7afddc0 Compare August 16, 2019 17:48
@replay
Copy link
Contributor Author

replay commented Aug 20, 2019

I've pushed a few more tests, so the test coverage report is happy

Co-Authored-By: Dan Cech <dan@aussiedan.com>
@replay replay force-pushed the sanitize_names_when_indexing_as_tag_value branch from 9d7d37b to f1cc19f Compare December 17, 2019 17:44
@DanCech DanCech requested a review from deniszh December 17, 2019 18:13
@deniszh deniszh merged commit c5d7cc8 into graphite-project:master Jan 19, 2020
deniszh pushed a commit to deniszh/carbon that referenced this pull request Jan 20, 2020
* sanitize names when using them as tag value

related: graphite-project/graphite-web#2458

* implement updated rules

* copy tag and value validation from graphite

* better error messages

* add tests for tag validation

* Better exception messages

Co-Authored-By: Dan Cech <dan@aussiedan.com>

Co-authored-by: Dan Cech <dan@aussiedan.com>
deniszh pushed a commit to deniszh/carbon that referenced this pull request Jan 20, 2020
* sanitize names when using them as tag value

related: graphite-project/graphite-web#2458

* implement updated rules

* copy tag and value validation from graphite

* better error messages

* add tests for tag validation

* Better exception messages

Co-Authored-By: Dan Cech <dan@aussiedan.com>

Co-authored-by: Dan Cech <dan@aussiedan.com>
deniszh pushed a commit to deniszh/carbon that referenced this pull request Feb 21, 2020
* sanitize names when using them as tag value

related: graphite-project/graphite-web#2458

* implement updated rules

* copy tag and value validation from graphite

* better error messages

* add tests for tag validation

* Better exception messages

Co-Authored-By: Dan Cech <dan@aussiedan.com>

Co-authored-by: Dan Cech <dan@aussiedan.com>
deniszh added a commit that referenced this pull request Feb 29, 2020
…858_pr-858_pr-858_pr-874_pr-875_pr-880_pr-881_pr-882

[1.1.x] s390x support for travis (#869) | Fix #871: Adjust aggregator-rules input_pattern match greediness to support numeric matching after captured field (#872) | sanitize names when using them as tag value (#858) | sanitize names when us
@stale
Copy link

stale bot commented Apr 13, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Apr 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants