-
Notifications
You must be signed in to change notification settings - Fork 53
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Add materialized views (#3000)
## Relevant issue(s) Resolves #2951 ## Description Adds materialized views. Also makes materialized views the default (see discord discussion). The caching behaviour of views in tests is now selected via an environment variable, meaning (with the exception of a few specific examples) a test with a view will test both cacheless and materialized variants - in the CI this adds a new dimension to the matrix, although materialized views are only executed using the simple settings (in-mem store, go client, etc) for now.
- Loading branch information
1 parent
ea3a74f
commit 4989901
Showing
63 changed files
with
1,673 additions
and
119 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| // Copyright 2024 Democratized Data Foundation | ||
| // | ||
| // Use of this software is governed by the Business Source License | ||
| // included in the file licenses/BSL.txt. | ||
| // | ||
| // As of the Change Date specified in that file, in accordance with | ||
| // the Business Source License, use of this software will be governed | ||
| // by the Apache License, Version 2.0, included in the file | ||
| // licenses/APL.txt. | ||
|
|
||
| package cli | ||
|
|
||
| import ( | ||
| "github.com/sourcenetwork/immutable" | ||
| "github.com/spf13/cobra" | ||
|
|
||
| "github.com/sourcenetwork/defradb/client" | ||
| ) | ||
|
|
||
| func MakeViewRefreshCommand() *cobra.Command { | ||
| var name string | ||
| var schemaRoot string | ||
| var versionID string | ||
| var getInactive bool | ||
| var cmd = &cobra.Command{ | ||
| Use: "refresh", | ||
| Short: "Refresh views.", | ||
| Long: `Refresh views, executing the underlying query and LensVm transforms and | ||
| persisting the results. | ||
| View is refreshed as the current user, meaning the cached items will reflect that user's | ||
| permissions. Subsequent query requests to the view, regardless of user, will receive | ||
| items from that cache. | ||
| Example: refresh all views | ||
| defradb client view refresh | ||
| Example: refresh views by name | ||
| defradb client view refresh --name UserView | ||
| Example: refresh views by schema root id | ||
| defradb client view refresh --schema bae123 | ||
| Example: refresh views by version id. This will also return inactive views | ||
| defradb client view refresh --version bae123 | ||
| `, | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| store := mustGetContextStore(cmd) | ||
|
|
||
| options := client.CollectionFetchOptions{} | ||
| if versionID != "" { | ||
| options.SchemaVersionID = immutable.Some(versionID) | ||
| } | ||
| if schemaRoot != "" { | ||
| options.SchemaRoot = immutable.Some(schemaRoot) | ||
| } | ||
| if name != "" { | ||
| options.Name = immutable.Some(name) | ||
| } | ||
| if getInactive { | ||
| options.IncludeInactive = immutable.Some(getInactive) | ||
| } | ||
|
|
||
| return store.RefreshViews( | ||
| cmd.Context(), | ||
| options, | ||
| ) | ||
| }, | ||
| } | ||
| cmd.Flags().StringVar(&name, "name", "", "View name") | ||
| cmd.Flags().StringVar(&schemaRoot, "schema", "", "View schema Root") | ||
| cmd.Flags().StringVar(&versionID, "version", "", "View version ID") | ||
| cmd.Flags().BoolVar(&getInactive, "get-inactive", false, "Get inactive views as well as active") | ||
| return cmd | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| # Materialized views | ||
|
|
||
| Views have been made materialized by default, this caused the tests to change slightly. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
65 changes: 65 additions & 0 deletions
65
docs/website/references/cli/defradb_client_view_refresh.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| ## defradb client view refresh | ||
|
|
||
| Refresh views. | ||
|
|
||
| ### Synopsis | ||
|
|
||
| Refresh views, executing the underlying query and LensVm transforms and | ||
| persisting the results. | ||
|
|
||
| View is refreshed as the current user, meaning the cached items will reflect that user's | ||
| permissions. Subsequent query requests to the view, regardless of user, will receive | ||
| items from that cache. | ||
|
|
||
| Example: refresh all views | ||
| defradb client view refresh | ||
|
|
||
| Example: refresh views by name | ||
| defradb client view refresh --name UserView | ||
|
|
||
| Example: refresh views by schema root id | ||
| defradb client view refresh --schema bae123 | ||
|
|
||
| Example: refresh views by version id. This will also return inactive views | ||
| defradb client view refresh --version bae123 | ||
|
|
||
|
|
||
| ``` | ||
| defradb client view refresh [flags] | ||
| ``` | ||
|
|
||
| ### Options | ||
|
|
||
| ``` | ||
| --get-inactive Get inactive views as well as active | ||
| -h, --help help for refresh | ||
| --name string View name | ||
| --schema string View schema Root | ||
| --version string View version ID | ||
| ``` | ||
|
|
||
| ### Options inherited from parent commands | ||
|
|
||
| ``` | ||
| -i, --identity string Hex formatted private key used to authenticate with ACP | ||
| --keyring-backend string Keyring backend to use. Options are file or system (default "file") | ||
| --keyring-namespace string Service name to use when using the system backend (default "defradb") | ||
| --keyring-path string Path to store encrypted keys when using the file backend (default "keys") | ||
| --log-format string Log format to use. Options are text or json (default "text") | ||
| --log-level string Log level to use. Options are debug, info, error, fatal (default "info") | ||
| --log-output string Log output path. Options are stderr or stdout. (default "stderr") | ||
| --log-overrides string Logger config overrides. Format <name>,<key>=<val>,...;<name>,... | ||
| --log-source Include source location in logs | ||
| --log-stacktrace Include stacktrace in error and fatal logs | ||
| --no-keyring Disable the keyring and generate ephemeral keys | ||
| --no-log-color Disable colored log output | ||
| --rootdir string Directory for persistent data (default: $HOME/.defradb) | ||
| --source-hub-address string The SourceHub address authorized by the client to make SourceHub transactions on behalf of the actor | ||
| --tx uint Transaction ID | ||
| --url string URL of HTTP endpoint to listen on or connect to (default "127.0.0.1:9181") | ||
| ``` | ||
|
|
||
| ### SEE ALSO | ||
|
|
||
| * [defradb client view](defradb_client_view.md) - Manage views within a running DefraDB instance | ||
|
|
Oops, something went wrong.