-
Notifications
You must be signed in to change notification settings - Fork 132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Generic Histogram for time-based data #2785
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
philrz
reviewed
Jun 28, 2023
philrz
reviewed
Jun 29, 2023
Co-authored-by: Phil Rzewski <phil@brimdata.io>
philrz
approved these changes
Jun 29, 2023
This was referenced Jun 30, 2023
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR depends on the Zealot PR.
When time series data is loaded into the app, we want to display a summary of that data in the form of a histogram.
Features include:
We need to store bits of data to properly render the histogram. We will store these settings per pool.
When we run the query.
To specific the domain we are searching, we can use the pool range on the primary key, but this might not be what the user chose, and the primary key might not be the time type. We should instead perform a “max and min” query in addition to the histogram query to set the range of the query. max(x) min(x) will be the range. We can run the query after the base query and if nothing comes up, we can assume that we cannot render the histogram. There is no value for x axis.
Questions?
How should we represent a “type”
A type has a certain number of fields, it has the names of those fields, the types, etc. If the type does not have a name, we will show the first three columns, then the number of columns after that. If it has a name we will show the name.
<ts,_path,id, +23>
<ts,_path,thing, +11>
<ts,_path,, +11>
Cool Ideas
It would be cool to filter based on a type. Although this should be done in the “types” tab. The histogram is for narrowing in on a range, the types tab will be used for the types.
Jittery Histogram
In order to avoid the flash of blank screen, we keep the old data until some new data arrives. Then we swap it out. We also collect the new data using the flush settings.
Some flush settings you want to optimize for the first 30 items. Others you really want to wait until you have lots of the data before you do something. Ideally all of it.
So the table needs first 30 records or 1s. But the histogram flush needs to be 1s or Infinity records. Every 1s it updates unless it’s done then it goes immediately.
Also fixes #2783
Also fixes #2782
Also fixes brimdata/super#2088