Add timeseries schema to CockroachDB on Nexus startup #6005
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Adds tables representing the timeseries schema. These currently track the core data about a schema that is fixed across versions; the set of fields, also across all versions; and then the mapping between the schema and their fields in each particular version. - Add database models for these - Add a build.rs to `oximeter` which sets up a const fn that returns all schema known at compile-time. Nexus uses this to determine which schema need to be loaded during population time. - Add datastore methods for loading all those schema during population, and adds a populator using it. - Adds more datatstore methods for fetching, listing, and upserting timeseries schema. There are a number of tests around these, but they are currently unused outside the datastore crate itself (other than the populator). The existing Nexus methods for those operations go through ClickHouse directly now, but they'll eventually be redirected to CockroachDB when the remainder of the switch is completed. - Closes #1294
bnaecker
commented
Jul 5, 2024
| -- | ||
| -- Names are derived from the concatenation of the target and metric names, | ||
| -- joined with a colon (':'). | ||
| timeseries_name STRING(128) PRIMARY KEY, |
Collaborator
Author
There was a problem hiding this comment.
We should probably add these limits into the schema generation mechanisms. Those should check the length of the data loaded from TOML, and fail if they exceed these limits. It would be unfortunate to have a "valid" name or description, which is then truncated in the database.
Collaborator
Author
There was a problem hiding this comment.
I went ahead and did this in #6009. We should review and merge that first, and then I'll update the limits here with those values once we're happy with them.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
oximeterwhich sets up a const fn that returns all schema known at compile-time. Nexus uses this to determine which schema need to be loaded during population time.