Control included quota and storage providers via build tags (#3664)

* Control included quota and storage providers via build tags

* Update CHANGELOG.md

* Mention the PostgreSQL and CockroachDB storage implementations in the Package Layout section of storage/README.md

* Describe the build tags

* Highlight the build tags in the README.md sections on Using / Working on the code

* Mention build tags in the feature implementation matrix

* 'storage and quota' is never singular

CHANGELOG.md

@@ -2,8 +2,14 @@
 
 ## HEAD
 
+### Storage
+
 * Add PostgreSQL quota manager and storage backend by @robstradling in https://github.com/google/trillian/pull/3644
 
+### Misc
+
+* Control included quota and storage providers via build tags by @robstradling in https://github.com/google/trillian/pull/3664
+
 ## v1.6.1
 
 * Recommended go version for development: 1.22

README.md

@@ -93,6 +93,10 @@ cd trillian
 go build ./...
 ```
 
+To build slimmer Trillian binaries that only include the storage and quota
+implementations that you need, consider specifying
+[build tags](/storage/README.md#build-tags).
+
 To build and run tests, use:
 
 ```bash
@@ -152,6 +156,9 @@ additional dependencies and tools, described in the following sections. The
 also a useful reference for the required tools and scripts, as it may be more
 up-to-date than this document.
 
+Anyone wanting to add a new storage and/or quota implementation should
+understand how Trillian uses [build tags](/storage/README.md#build-tags).
+
 ### Rebuilding Generated Code
 
 Some of the Trillian Go code is autogenerated from other files:

cmd/internal/provider/cloudspanner.go

@@ -0,0 +1,7 @@
+//go:build cloudspanner || !(crdb || mysql || postgresql)
+
+package provider
+
+import (
+	_ "github.com/google/trillian/storage/cloudspanner"
+)

cmd/internal/provider/crdb.go

@@ -0,0 +1,9 @@
+//go:build crdb || !(cloudspanner || mysql || postgresql)
+
+package provider
+
+import (
+	_ "github.com/google/trillian/storage/crdb"
+
+	_ "github.com/google/trillian/quota/crdbqm"
+)

cmd/internal/provider/mysql.go

@@ -0,0 +1,9 @@
+//go:build mysql || !(cloudspanner || crdb || postgresql)
+
+package provider
+
+import (
+	_ "github.com/google/trillian/storage/mysql"
+
+	_ "github.com/google/trillian/quota/mysqlqm"
+)

cmd/internal/provider/postgresql.go

@@ -0,0 +1,9 @@
+//go:build postgresql || !(cloudspanner || crdb || mysql)
+
+package provider
+
+import (
+	_ "github.com/google/trillian/storage/postgresql"
+
+	_ "github.com/google/trillian/quota/postgresqlqm"
+)

cmd/trillian_log_server/main.go

@@ -45,16 +45,8 @@ import (
 	"google.golang.org/grpc"
 	"k8s.io/klog/v2"
 
-	// Register supported storage providers.
-	_ "github.com/google/trillian/storage/cloudspanner"
-	_ "github.com/google/trillian/storage/crdb"
-	_ "github.com/google/trillian/storage/mysql"
-	_ "github.com/google/trillian/storage/postgresql"
-
-	// Load quota providers
-	_ "github.com/google/trillian/quota/crdbqm"
-	_ "github.com/google/trillian/quota/mysqlqm"
-	_ "github.com/google/trillian/quota/postgresqlqm"
+	// Register supported storage and quota providers.
+	_ "github.com/google/trillian/cmd/internal/provider"
 )
 
 var (

cmd/trillian_log_signer/main.go

@@ -51,16 +51,8 @@ import (
 	"google.golang.org/grpc"
 	"k8s.io/klog/v2"
 
-	// Register supported storage providers.
-	_ "github.com/google/trillian/storage/cloudspanner"
-	_ "github.com/google/trillian/storage/crdb"
-	_ "github.com/google/trillian/storage/mysql"
-	_ "github.com/google/trillian/storage/postgresql"
-
-	// Load quota providers
-	_ "github.com/google/trillian/quota/crdbqm"
-	_ "github.com/google/trillian/quota/mysqlqm"
-	_ "github.com/google/trillian/quota/postgresqlqm"
+	// Register supported storage and quota providers.
+	_ "github.com/google/trillian/cmd/internal/provider"
 )
 
 var (

docs/Feature_Implementation_Matrix.md

@@ -45,6 +45,9 @@ It currently exists as internal prototype.
 
 This section lists the status of implementations for the _pluggable_ subsystems which Trillian supports.
 
+[Build tags](/storage/README.md#build-tags) can be used to control which [storage](#storage) and
+[quota](#quota) implementations are compiled in to Trillian binaries.
+
 ### Storage
 
 Trillian supports "pluggable" storage implementations for durable storage of the merkle tree data.

storage/README.md

@@ -13,10 +13,59 @@ The MySQL / MariaDB implementation includes support for Maps. This has not yet
 been implemented by Cloud Spanner. There may be other storage implementations
 available from third parties.
 
+These implementations are in alpha mode and are not yet ready to be used by
+real applications:
+   * PostgreSQL in the [postgresql/](postgresql) package.
+   * CockroachDB in the [crdb/](crdb) package.
+
 These implementations are for test purposes only and should not be used by real
 applications:
    * In-memory Storage, in the [memory](memory) package.
 
+## Build tags
+
+By default all of the storage and quota implementations are compiled in to the
+log server and signer binaries. These binaries can be slimmed down
+significantly by specifying one or more of the following build tags:
+
+   * cloudspanner
+   * crdb
+   * mysql
+   * postgresql
+
+### Adding a new storage implementation
+
+To add a new storage and/or quota implementation requires:
+
+   * each of the `go:build` directives in the existing files in the
+[cmd/internal/provider](/cmd/internal/provider) directory to be made aware of
+the build tag for the new implementation.
+   * a new file to be created for the new implementation in that same
+directory, whose contents follow the pattern established by the existing files.
+
+### Examples
+
+Include all storage and quota implementations (default):
+
+```bash
+> cd cmd/trillian_log_server && go build && ls -sh trillian_log_server
+62M trillian_log_server*
+```
+
+Include just one storage and associated quota implementation:
+
+```bash
+> cd cmd/trillian_log_server && go build -tags=crdb && ls -sh trillian_log_server
+37M trillian_log_server*
+```
+
+Include multiple storage and associated quota implementations:
+
+```bash
+> cd cmd/trillian_log_server && go build -tags=mysql,postgresql && ls -sh trillian_log_server
+40M trillian_log_server*
+```
+
 ## Notes and Caveats
 
 The design is such that both `LogStorage` and `MapStorage` models reuse a