Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic Histogram for time-based data #2785

Merged
merged 116 commits into from
Jun 29, 2023
Merged

Generic Histogram for time-based data #2785

merged 116 commits into from
Jun 29, 2023

Conversation

jameskerr
Copy link
Member

@jameskerr jameskerr commented Jun 19, 2023

This PR depends on the Zealot PR.

When time series data is loaded into the app, we want to display a summary of that data in the form of a histogram.

Features include:

  1. The histogram can be swiped and a pin will be added to the query that changes the range on the desired key.
  2. The time field used in the histogram will default to the pool key, but can be specified manually in the options modal next to the histogram.
  3. The grouping field that creates the bar stacks can also be changed in the options modal next to the histogram, and it will default to typeof(this).
  4. If the histogram is unable to be rendered due to some problem, the app will display an error message in place of the histogram.

We need to store bits of data to properly render the histogram. We will store these settings per pool.

  1. What is the field for the x axis
  2. What is the field for the group

When we run the query.

To specific the domain we are searching, we can use the pool range on the primary key, but this might not be what the user chose, and the primary key might not be the time type. We should instead perform a “max and min” query in addition to the histogram query to set the range of the query. max(x) min(x) will be the range. We can run the query after the base query and if nothing comes up, we can assume that we cannot render the histogram. There is no value for x axis.

Questions?

How should we represent a “type”

A type has a certain number of fields, it has the names of those fields, the types, etc. If the type does not have a name, we will show the first three columns, then the number of columns after that. If it has a name we will show the name.

<ts,_path,id, +23>
<ts,_path,thing, +11>
<ts,_path,, +11>

Cool Ideas

It would be cool to filter based on a type. Although this should be done in the “types” tab. The histogram is for narrowing in on a range, the types tab will be used for the types.

Jittery Histogram

In order to avoid the flash of blank screen, we keep the old data until some new data arrives. Then we swap it out. We also collect the new data using the flush settings.

Some flush settings you want to optimize for the first 30 items. Others you really want to wait until you have lots of the data before you do something. Ideally all of it.

So the table needs first 30 records or 1s. But the histogram flush needs to be 1s or Infinity records. Every 1s it updates unless it’s done then it goes immediately.

Also fixes #2783
Also fixes #2782
Also fixes brimdata/super#2088

@philrz

This comment was marked as resolved.

@philrz

This comment was marked as resolved.

@philrz

This comment was marked as resolved.

@philrz

This comment was marked as resolved.

CODE.md Outdated Show resolved Hide resolved
CODE.md Outdated Show resolved Hide resolved
CODE.md Outdated Show resolved Hide resolved
jameskerr and others added 2 commits June 29, 2023 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants