model: sanitize/de-dot keys for labels, custom, and marks #4465
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Replace reserved label key characters with '_', alleviating clients from having to do this. Also, optimise how we merge global and event-specific labels. Instead of adding all global labels and then using utility.DeepUpdate, de-dot the label keys first and range over the two maps to create a second map. Because the labels keys cannot have dots, we do not need to deep merge.
Sanitize the keys of "custom" context using the same logic applied to labels. Also, drop the model.Custom type and just use common.MapStr. This eliminates some unnecessary pointer indirection.
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
Steps errors
Expand to view the steps failures
|
simitt
approved these changes
Nov 30, 2020
2 tasks
axw
added a commit
to axw/apm-server
that referenced
this pull request
Dec 14, 2020
* model: sanitize label keys Replace reserved label key characters with '_', alleviating clients from having to do this. Also, optimise how we merge global and event-specific labels. Instead of adding all global labels and then using utility.DeepUpdate, de-dot the label keys first and range over the two maps to create a second map. Because the labels keys cannot have dots, we do not need to deep merge. * model: sanitize "custom" attribute keys Sanitize the keys of "custom" context using the same logic applied to labels. Also, drop the model.Custom type and just use common.MapStr. This eliminates some unnecessary pointer indirection. * model: sanitize "marks" keys * model/modeldecoder: drop label key regexp rules * Add changelog entry
axw
added a commit
that referenced
this pull request
Dec 14, 2020
) * model: sanitize label keys Replace reserved label key characters with '_', alleviating clients from having to do this. Also, optimise how we merge global and event-specific labels. Instead of adding all global labels and then using utility.DeepUpdate, de-dot the label keys first and range over the two maps to create a second map. Because the labels keys cannot have dots, we do not need to deep merge. * model: sanitize "custom" attribute keys Sanitize the keys of "custom" context using the same logic applied to labels. Also, drop the model.Custom type and just use common.MapStr. This eliminates some unnecessary pointer indirection. * model: sanitize "marks" keys * model/modeldecoder: drop label key regexp rules * Add changelog entry
|
Tested with BC1 - sent key |
12 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation/summary
Sanitize/de-dot keys for labels, custom, and marks, in the server when transforming to documents. Reserved characters (
*,., and") will be replaced with_.Remove the validation from the modeldecoder types, as agents will no longer be required to sanitize the keys. This changes various
patternPropertiestoadditionalPropertiesin the generated JSON Schema.Finally, remove the
model.Labelsandmodels.Customtypes and usecommon.MapStrdirectly (and without an extra level of pointer indirection). Use the knowledge that label keys cannot have dots to avoid the need for deep updates, replacing the relatively expensiveutility.DeepUpdatecalls with shallow map merging.Checklist
I have considered changes for:
- [ ] documentation- [ ] logging (add log lines, choose appropriate log selector, etc.)- [ ] metrics and monitoring (create issue for Kibana team to add metrics to visualizations, e.g. Kibana#44001)- [ ] telemetry- [ ] Elasticsearch Service (https://cloud.elastic.co)- [ ] Elastic Cloud Enterprise (https://www.elastic.co/products/ece)- [ ] Elastic Cloud on Kubernetes (https://www.elastic.co/elastic-cloud-kubernetes)How to test these changes
make testRelated issues
Closes #4225