diff --git a/CHANGELOG.md b/CHANGELOG.md index de0ea9843f..a89974064b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,8 +2,14 @@ ## HEAD +### Storage + * Add PostgreSQL quota manager and storage backend by @robstradling in https://github.com/google/trillian/pull/3644 +### Misc + +* Control included quota and storage providers via build tags by @robstradling in https://github.com/google/trillian/pull/3664 + ## v1.6.1 * Recommended go version for development: 1.22 diff --git a/README.md b/README.md index b9f3b84267..1ad5f708a0 100644 --- a/README.md +++ b/README.md @@ -93,6 +93,10 @@ cd trillian go build ./... ``` +To build slimmer Trillian binaries that only include the storage and quota +implementations that you need, consider specifying +[build tags](/storage/README.md#build-tags). + To build and run tests, use: ```bash @@ -152,6 +156,9 @@ additional dependencies and tools, described in the following sections. The also a useful reference for the required tools and scripts, as it may be more up-to-date than this document. +Anyone wanting to add a new storage and/or quota implementation should +understand how Trillian uses [build tags](/storage/README.md#build-tags). + ### Rebuilding Generated Code Some of the Trillian Go code is autogenerated from other files: diff --git a/cmd/internal/provider/cloudspanner.go b/cmd/internal/provider/cloudspanner.go new file mode 100644 index 0000000000..3c83f1f1a3 --- /dev/null +++ b/cmd/internal/provider/cloudspanner.go @@ -0,0 +1,7 @@ +//go:build cloudspanner || !(crdb || mysql || postgresql) + +package provider + +import ( + _ "github.com/google/trillian/storage/cloudspanner" +) diff --git a/cmd/internal/provider/crdb.go b/cmd/internal/provider/crdb.go new file mode 100644 index 0000000000..4e1ffb1fc9 --- /dev/null +++ b/cmd/internal/provider/crdb.go @@ -0,0 +1,9 @@ +//go:build crdb || !(cloudspanner || mysql || postgresql) + +package provider + +import ( + _ "github.com/google/trillian/storage/crdb" + + _ "github.com/google/trillian/quota/crdbqm" +) diff --git a/cmd/internal/provider/mysql.go b/cmd/internal/provider/mysql.go new file mode 100644 index 0000000000..b04a9ff0ef --- /dev/null +++ b/cmd/internal/provider/mysql.go @@ -0,0 +1,9 @@ +//go:build mysql || !(cloudspanner || crdb || postgresql) + +package provider + +import ( + _ "github.com/google/trillian/storage/mysql" + + _ "github.com/google/trillian/quota/mysqlqm" +) diff --git a/cmd/internal/provider/postgresql.go b/cmd/internal/provider/postgresql.go new file mode 100644 index 0000000000..a3db2ab755 --- /dev/null +++ b/cmd/internal/provider/postgresql.go @@ -0,0 +1,9 @@ +//go:build postgresql || !(cloudspanner || crdb || mysql) + +package provider + +import ( + _ "github.com/google/trillian/storage/postgresql" + + _ "github.com/google/trillian/quota/postgresqlqm" +) diff --git a/cmd/trillian_log_server/main.go b/cmd/trillian_log_server/main.go index 7733d0302f..3f57122bf6 100644 --- a/cmd/trillian_log_server/main.go +++ b/cmd/trillian_log_server/main.go @@ -45,16 +45,8 @@ import ( "google.golang.org/grpc" "k8s.io/klog/v2" - // Register supported storage providers. - _ "github.com/google/trillian/storage/cloudspanner" - _ "github.com/google/trillian/storage/crdb" - _ "github.com/google/trillian/storage/mysql" - _ "github.com/google/trillian/storage/postgresql" - - // Load quota providers - _ "github.com/google/trillian/quota/crdbqm" - _ "github.com/google/trillian/quota/mysqlqm" - _ "github.com/google/trillian/quota/postgresqlqm" + // Register supported storage and quota providers. + _ "github.com/google/trillian/cmd/internal/provider" ) var ( diff --git a/cmd/trillian_log_signer/main.go b/cmd/trillian_log_signer/main.go index 16e081c9ad..eaa56b568b 100644 --- a/cmd/trillian_log_signer/main.go +++ b/cmd/trillian_log_signer/main.go @@ -51,16 +51,8 @@ import ( "google.golang.org/grpc" "k8s.io/klog/v2" - // Register supported storage providers. - _ "github.com/google/trillian/storage/cloudspanner" - _ "github.com/google/trillian/storage/crdb" - _ "github.com/google/trillian/storage/mysql" - _ "github.com/google/trillian/storage/postgresql" - - // Load quota providers - _ "github.com/google/trillian/quota/crdbqm" - _ "github.com/google/trillian/quota/mysqlqm" - _ "github.com/google/trillian/quota/postgresqlqm" + // Register supported storage and quota providers. + _ "github.com/google/trillian/cmd/internal/provider" ) var ( diff --git a/docs/Feature_Implementation_Matrix.md b/docs/Feature_Implementation_Matrix.md index 36b709e9b8..07eae75874 100644 --- a/docs/Feature_Implementation_Matrix.md +++ b/docs/Feature_Implementation_Matrix.md @@ -45,6 +45,9 @@ It currently exists as internal prototype. This section lists the status of implementations for the _pluggable_ subsystems which Trillian supports. +[Build tags](/storage/README.md#build-tags) can be used to control which [storage](#storage) and +[quota](#quota) implementations are compiled in to Trillian binaries. + ### Storage Trillian supports "pluggable" storage implementations for durable storage of the merkle tree data. diff --git a/storage/README.md b/storage/README.md index 78dd7d1cbd..045cae4544 100644 --- a/storage/README.md +++ b/storage/README.md @@ -13,10 +13,59 @@ The MySQL / MariaDB implementation includes support for Maps. This has not yet been implemented by Cloud Spanner. There may be other storage implementations available from third parties. +These implementations are in alpha mode and are not yet ready to be used by +real applications: + * PostgreSQL in the [postgresql/](postgresql) package. + * CockroachDB in the [crdb/](crdb) package. + These implementations are for test purposes only and should not be used by real applications: * In-memory Storage, in the [memory](memory) package. +## Build tags + +By default all of the storage and quota implementations are compiled in to the +log server and signer binaries. These binaries can be slimmed down +significantly by specifying one or more of the following build tags: + + * cloudspanner + * crdb + * mysql + * postgresql + +### Adding a new storage implementation + +To add a new storage and/or quota implementation requires: + + * each of the `go:build` directives in the existing files in the +[cmd/internal/provider](/cmd/internal/provider) directory to be made aware of +the build tag for the new implementation. + * a new file to be created for the new implementation in that same +directory, whose contents follow the pattern established by the existing files. + +### Examples + +Include all storage and quota implementations (default): + +```bash +> cd cmd/trillian_log_server && go build && ls -sh trillian_log_server +62M trillian_log_server* +``` + +Include just one storage and associated quota implementation: + +```bash +> cd cmd/trillian_log_server && go build -tags=crdb && ls -sh trillian_log_server +37M trillian_log_server* +``` + +Include multiple storage and associated quota implementations: + +```bash +> cd cmd/trillian_log_server && go build -tags=mysql,postgresql && ls -sh trillian_log_server +40M trillian_log_server* +``` + ## Notes and Caveats The design is such that both `LogStorage` and `MapStorage` models reuse a