Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic Histogram for time-based data #2785

Merged
merged 116 commits into from
Jun 29, 2023
Merged
Show file tree
Hide file tree
Changes from 114 commits
Commits
Show all changes
116 commits
Select commit Hold shift + click to select a range
c382589
Initialize the app with nextjs
jameskerr Apr 5, 2023
d03956f
Context menus working over ipc
jameskerr Apr 7, 2023
285146b
Fix detect file types, add cors
jameskerr Apr 7, 2023
7d1aef8
Create Pool operation works
jameskerr Apr 10, 2023
3ddf928
yarn lint works
jameskerr Apr 14, 2023
7cb4959
Were able to build an optimized version
jameskerr Apr 14, 2023
62235c7
Domain folders
jameskerr Apr 20, 2023
10f6d2b
Create a when clause compiler
jameskerr Apr 21, 2023
4ee58c4
Pcaps completely work now
jameskerr Apr 23, 2023
8738836
Converted the nav menus
jameskerr Apr 23, 2023
66ad04c
Typescript errors fixed
jameskerr Apr 24, 2023
eda54f7
Lint errors fixed
jameskerr Apr 24, 2023
664afb1
Correlations work again
jameskerr Apr 24, 2023
50e3eaf
Remove tsbuildinfo
jameskerr Apr 24, 2023
8bfa532
Add jest-jsdom
jameskerr Apr 24, 2023
a891998
Configurations work through the plugin api now
jameskerr Apr 24, 2023
ce8e9a5
No typescript errors
jameskerr Apr 24, 2023
3bc7fa7
All the unit tests pass
jameskerr Apr 25, 2023
94da1af
First e2e test passing
jameskerr Apr 25, 2023
9152aad
Export e2e tests work
jameskerr Apr 25, 2023
2f1dc62
Background window works
jameskerr Apr 25, 2023
c8b25e2
e2e tests work with local server
jameskerr Apr 25, 2023
9005f4d
Removed all dynamic requires so that we can bundle the node code
jameskerr Apr 27, 2023
41da783
Fix start scripts
jameskerr Apr 27, 2023
a4e090b
Fixing tests
jameskerr May 1, 2023
3ea2512
Tests are green
jameskerr May 1, 2023
ec54f9f
Lint works
jameskerr May 1, 2023
00e0676
Got a prod build working!
jameskerr May 2, 2023
1d3fae0
End to end tests passing
jameskerr May 2, 2023
57a10b9
Fix tests
jameskerr May 2, 2023
90e4cbb
Merge main
jameskerr May 2, 2023
18ad5e4
Point at the updated packages
jameskerr May 2, 2023
20adc11
Update our zed deps
jameskerr May 3, 2023
aeaf4b1
Merge into main
jameskerr May 3, 2023
30eedb1
Update zed deps
jameskerr May 3, 2023
5f0f7e5
Converted about window to nextjs
jameskerr May 3, 2023
958c527
Remove old entry points
jameskerr May 3, 2023
ba0c793
Error handling bad correlation queries
jameskerr May 3, 2023
67db882
Update zed deps
jameskerr May 3, 2023
4396829
Update yarn lock
jameskerr May 3, 2023
4a116b3
End of line auto
jameskerr May 4, 2023
21cf465
Explicitly include files into the app build
jameskerr May 4, 2023
64c5c2d
update next
jameskerr May 4, 2023
cc177c9
update paths to point to /installers
jameskerr May 4, 2023
fd4e450
Don't use app dir
jameskerr May 4, 2023
df30dbe
Merge main
jameskerr May 4, 2023
d9b62b3
Remove code
jameskerr May 4, 2023
7c8513b
Bundle depdencies for main process
jameskerr May 5, 2023
8ceba5e
This now points to the zeps for the zed binary
jameskerr May 5, 2023
0735198
Update zealot deps
jameskerr May 5, 2023
790dab9
Add migration for brimcap root
jameskerr May 5, 2023
373579e
Upload only the interesting artifacts
jameskerr May 5, 2023
a628145
Add migration for suricata data
jameskerr May 5, 2023
610a594
Update build script
jameskerr May 5, 2023
9531193
Fix typo
jameskerr May 5, 2023
024e286
Clean up build workflow
jameskerr May 5, 2023
02dca9c
Suricata update working
jameskerr May 8, 2023
aca7df8
Merge remote-tracking branch 'origin/main' into untangle
jameskerr May 8, 2023
e31bcab
Fix context menus for values
jameskerr May 8, 2023
4f5c7bf
Only allow bound commands in menus
jameskerr May 8, 2023
23e1af3
Handle nested menus
jameskerr May 8, 2023
57f9320
Use the clicks
jameskerr May 8, 2023
dc04200
Remote queries working again
jameskerr May 8, 2023
7550fe9
Fix about window
jameskerr May 9, 2023
6042186
Move progress transform into the loop
jameskerr May 9, 2023
d011a12
Fix welcome page
jameskerr May 9, 2023
6cc65c9
Fix reset state
jameskerr May 9, 2023
59c34ef
Merge remote-tracking branch 'origin/main' into untangle
jameskerr May 9, 2023
57a36e1
Add exit mock
jameskerr May 10, 2023
1df8d51
Merge remote-tracking branch 'origin/main' into untangle
jameskerr May 24, 2023
35f2f03
Remove duplicate function
jameskerr May 24, 2023
a9795eb
Merge remote-tracking branch 'origin/main' into untangle
jameskerr May 25, 2023
1ebd20e
Use the open lib, not the shell.openExternal
jameskerr May 25, 2023
a7f9bb8
Persisted query panels state
jameskerr May 30, 2023
e0367ad
Made a useQuery hook that feels good
jameskerr Jun 1, 2023
e2b79ad
Dialog position looks good
jameskerr Jun 5, 2023
dcbc585
Got the outside click working
jameskerr Jun 6, 2023
72c943a
the form works and saves values
jameskerr Jun 6, 2023
b1a0a01
Use the pool settings in the histogram query
jameskerr Jun 6, 2023
bdbbd91
histogram bar width and histogram tooltip
jameskerr Jun 13, 2023
885a2ba
Tooltip data works
jameskerr Jun 13, 2023
ffb186a
Fix type errors
jameskerr Jun 13, 2023
b4530de
Fixing more type errors
jameskerr Jun 14, 2023
277236c
Updated typescript and got type errors fixed
jameskerr Jun 14, 2023
a3ad748
Styled the axises
jameskerr Jun 15, 2023
f08db85
Undo resizable histogram
jameskerr Jun 16, 2023
5d91f64
Cleaned up histogram code
jameskerr Jun 16, 2023
e0e8b54
Tooltip bug
jameskerr Jun 16, 2023
ce0342b
Fix up the tooltip
jameskerr Jun 17, 2023
f1a8b37
Update node and point to local zed-js
jameskerr Jun 19, 2023
fdb6c1d
Make the panel a flex parent
jameskerr Jun 19, 2023
2aa3237
Add max cardinality to the histogram
jameskerr Jun 19, 2023
059a3b6
Tableau 10 works better
jameskerr Jun 19, 2023
87f52e5
Merge remote-tracking branch 'origin/main' into histogram-update
jameskerr Jun 19, 2023
70931f3
Keep zealot the same
jameskerr Jun 19, 2023
9f4bf98
Configure color scale in the brimcap plugin
jameskerr Jun 20, 2023
95e4713
A living document for code patterns
jameskerr Jun 20, 2023
7f53c88
Show pool name in history entries
jameskerr Jun 22, 2023
c3a9bce
Use data transition and clean up run-query
jameskerr Jun 22, 2023
82ac06b
Update the histogram
jameskerr Jun 23, 2023
033d0e1
Update zealot dependencies
jameskerr Jun 27, 2023
ab7a624
Fix a text overflow issue
jameskerr Jun 27, 2023
1873241
Increase the size of the tooltip
jameskerr Jun 27, 2023
ab92ec7
Use a dediecated function for fetching the first page
jameskerr Jun 27, 2023
61a4558
Separate nulls from missing in histogram title
jameskerr Jun 27, 2023
0395b5d
Remove stacked bar idea
jameskerr Jun 27, 2023
6750630
Cleanup event listeners between test runs
jameskerr Jun 28, 2023
5f5e722
Fix yarn lint
jameskerr Jun 28, 2023
a66740f
Fix e2e tests
jameskerr Jun 28, 2023
c55714a
Merge remote-tracking branch 'origin/main' into histogram-update
jameskerr Jun 28, 2023
197abab
Fix timeout lint problem
jameskerr Jun 28, 2023
58ac017
Make the default timeField the pool key
jameskerr Jun 29, 2023
c04987c
Fix grammar
jameskerr Jun 29, 2023
cec4c4f
Merge remote-tracking branch 'origin/main' into histogram-update
jameskerr Jun 29, 2023
f71ecc2
Update CODE.md
jameskerr Jun 29, 2023
7b3fe3d
Use the timeField when creating a new time range pin
jameskerr Jun 29, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .eslintrc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,5 @@ rules:
prettier/prettier:
- error
- endOfLine: auto
react/react-in-jsx-scope:
- off
2 changes: 1 addition & 1 deletion .node-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v16.8.0
v16.10.0
51 changes: 51 additions & 0 deletions CODE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Coding Patterns

This is here to document the design patterns chosen by the developers. It documents structures, abstractions, and philosophy in this repo.

## Folders

Documentation for where code should go.

### src/domain

Domains are concepts specific to Zui. Things like pools, queries, sessions, history. A domain can contain operations, handlers, a plugin-api, models, types.

### src/core

Core objects and functions can be used across multiple domains. Like menus and commands.

### src/util

This is super generic JavaScript code that knows nothing about Zui, Electron, or Web. It can be copied into another project and work right out of the gate. Code must contain no dependencies in this folder.

## src/plugins

This is a directory of plugins that use the plugin api to add functionality to the app. Come of the plugins are prefixed with the word core to indicate they provide core functionality, but only require the plugin api to achieve this. To add a plugin today, create a new directory in the plugins directory, create and index.ts file within it, then export a named function called activate. It will accept the PluginContext object as its only argument. The go to the `run-plugins.ts` file and call the activate function within the body of runPlugins. This could be make automatic one day, but it's hardcoded for now.
jameskerr marked this conversation as resolved.
Show resolved Hide resolved

## Glossary of Terms

In no particular order.

_Domain_

A named concept relative to the Zui app. Examples of domains are pools, queries, panes, configurations, plugins. First-class things that people can point to when they describe the app.

_Main Object_

The main object contains methods for managing the application in the main process. It can manage windows, session data, plugins, and app lifecycle.

_Operation_

A function that runs in the main process and has the main object in scope. When the app boots, operations begin listening for IPC invocations from a renderer process. They can also be called directly from the main process using their .run() method. When invoking an operation from the main process, use the invoke function found in src/core/invoke.ts.

_Handler_

A handler is an event listener in the renderer process. It waits for a message from the main process before running its callback function. Messages can be sent to handlers from the main process using sendToFocusedWindow(message, ...arguments).

_Message_

A message is a TypeScript type defining a name and a set of arguments. Messages are necessary to provide types for Operations and Handlers.

_Plugin Api_

The plugin api runs in the Node main process and is given to plugin authors to extend the app. It should not have privileged access to everything. Only what is useful for plugin authors. This means it should not contain references to the full store or the main process. Instead, it exposes methods that in turn call operations. Operations are not exposed to plugins and therefore have the main object and store in scope. The pattern should be: plugin-api exposes a simple method which then runs an operation.
17 changes: 10 additions & 7 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,8 @@
},
"devDependencies": {
"@babel/core": "^7.17.9",
"@brimdata/zed-js": "0.0.16",
"@brimdata/zed-node": "0.0.16",
"@brimdata/zed-js": "0.0.17",
"@brimdata/zed-node": "0.0.17",
"@codemirror/autocomplete": "^0.19.15",
"@codemirror/closebrackets": "^0.19.1",
"@codemirror/commands": "^0.19.8",
Expand All @@ -67,7 +67,7 @@
"@testing-library/user-event": "^14.1.1",
"@types/animejs": "^3.1.2",
"@types/classnames": "^2.2.10",
"@types/d3": "^5.7.2",
"@types/d3": "^6.7.0",
"@types/electron-devtools-installer": "^2.2.0",
"@types/fs-extra": "^9.0.1",
"@types/lodash": "^4.14.161",
Expand All @@ -83,8 +83,8 @@
"@types/sprintf-js": "^1.1.2",
"@types/styled-components": "^5.1.3",
"@types/tmp": "^0.2.0",
"@typescript-eslint/eslint-plugin": "^5.16.0",
"@typescript-eslint/parser": "^5.16.0",
"@typescript-eslint/eslint-plugin": "5.60.1",
"@typescript-eslint/parser": "5.60.1",
"abort-controller": "^3.0.0",
"acorn": "^7.4.1",
"ajv": "^6.9.1",
Expand All @@ -96,7 +96,7 @@
"classnames": "^2.2.6",
"commander": "^2.20.3",
"cross-fetch": "^3.1.6",
"d3": "^5.16.0",
"d3": "^6.7.0",
"date-fns": "^2.16.1",
"decompress": "^4.2.1",
"electron": "^22.0.0",
Expand Down Expand Up @@ -146,11 +146,14 @@
"react-dnd": "^14.0.5",
"react-dnd-html5-backend": "^14.0.2",
"react-dom": "^18.0.0",
"react-error-boundary": "^4.0.10",
"react-hook-form": "^7.44.3",
"react-hot-toast": "^1.0.1",
"react-input-autosize": "^3.0.0",
"react-is": "^17.0.2",
"react-markdown": "^6.0.2",
"react-redux": "^8.0.5",
"react-resizable-panels": "^0.0.45",
"react-router": "5.3.1",
"react-spring": "^8.0.27",
"react-tooltip": "^4.2.7",
Expand All @@ -166,7 +169,7 @@
"styled-components": "^5.3.5",
"tmp": "^0.1.0",
"tree-model": "^1.0.7",
"typescript": "^4.6.2",
"typescript": "5.1.5",
"use-resize-observer": "^8.0.0",
"web-file-polyfill": "^1.0.4",
"web-streams-polyfill": "^3.2.0",
Expand Down
8 changes: 6 additions & 2 deletions packages/e2e-tests/tests/queries.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,12 @@ test.describe("Query tests", () => {
const entries = await history.evaluateAll<string[], HTMLElement>((nodes) =>
nodes.map((n) => n.innerText.trim().replaceAll(/\s+/g, " "))
)

expect(entries).toEqual(["3 now", "2 now", "1 now"])
const expected = [
"from 'sample.tsv' | 3 now",
"from 'sample.tsv' | 2 now",
"from 'sample.tsv' | 1 now",
]
expect(entries).toEqual(expected)
})

test("named queries' creation, modification, update/save, proper outdated status display", async () => {
Expand Down
2 changes: 1 addition & 1 deletion src/app/features/right-pane/history/history-item.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ function getValue(active: ActiveQuery) {
if (active.isDeleted()) {
return "(Deleted)"
} else if (active.isAnonymous() || active.isModified()) {
return active.value() || "(Empty)"
return active.toZed() || "(Empty)"
} else {
return active.name()
}
Expand Down
4 changes: 2 additions & 2 deletions src/app/query-home/flows/submit-search.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@ import Current from "src/js/state/Current"
import Editor from "src/js/state/Editor"
import Results from "src/js/state/Results"
import QueryVersions from "../../../js/state/QueryVersions"
import {MAIN_RESULTS} from "src/js/state/Results/types"
import {Thunk} from "src/js/state/types"
import {QueryModel} from "../../../js/models/query-model"
import {RESULTS_QUERY} from "src/panes/results-pane/run-results-query"

const submitSearch =
(): Thunk =>
Expand All @@ -16,7 +16,7 @@ const submitSearch =
// An error with the syntax
if (error) {
const tabId = Current.getTabId(getState())
dispatch(Results.error({id: MAIN_RESULTS, error, tabId}))
dispatch(Results.error({id: RESULTS_QUERY, error, tabId}))
return
}

Expand Down
39 changes: 0 additions & 39 deletions src/app/query-home/histogram/ChartSVG.tsx

This file was deleted.

44 changes: 0 additions & 44 deletions src/app/query-home/histogram/MainHistogram/Chart.tsx

This file was deleted.

45 changes: 0 additions & 45 deletions src/app/query-home/histogram/MainHistogram/format.ts

This file was deleted.

Loading