feat: add vault transit kms engine

[RichiCoder1]

Adds a KMS option based off Vault Transit. Also slightly reorganized how providers are registered, inspired in part by go-cloud.

Related: sigstore/cosign#131

[dlorenc]

This is great!

[RichiCoder1]

Verified.

Edit: I did slightly modify registration some more. go-cloud assumes you'll reference the KMS providers you want (e.g.: _ github.com/google/go-cloud/secrets/hashivault) but I wasn't sure that was a reasonable expectation for sigstore. So it flips registration back to pkg/kms referencing all it's sub packages instead of the other way around.

E2E Tests

package main

import (
	"context"
	"crypto/ecdsa"
	"crypto/sha256"
	"fmt"
	"os"
	"testing"

	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
	"github.com/stretchr/testify/suite"

	"github.com/sigstore/sigstore/pkg/kms"

	vault "github.com/hashicorp/vault/api"
)

type VaultSuite struct {
	suite.Suite
}

func (suite *VaultSuite) GetProvider(key string) kms.KMS {
	provider, err := kms.Get(context.Background(), fmt.Sprintf("hashivault://%s", key))
	require.NoError(suite.T(), err)
	require.NotNil(suite.T(), provider)
	return provider
}

func (suite *VaultSuite) SetupSuite() {
	client, err := vault.NewClient(&vault.Config{
		Address: os.Getenv("VAULT_ADDR"),
	})
	require.Nil(suite.T(), err)
	require.NotNil(suite.T(), client)

	err = client.Sys().Mount("transit", &vault.MountInput{
		Type: "transit",
	})
	require.Nil(suite.T(), err)
}

func (suite *VaultSuite) TearDownSuite() {
	client, err := vault.NewClient(&vault.Config{
		Address: os.Getenv("VAULT_ADDR"),
	})
	require.Nil(suite.T(), err)
	require.NotNil(suite.T(), client)

	err = client.Sys().Unmount("transit")
	require.Nil(suite.T(), err)
}

func (suite *VaultSuite) TestProviders() {
	providers := kms.ProvidersMux().Providers()
	assert.Len(suite.T(), providers, 2)
}

func (suite *VaultSuite) TestProvider() {
	suite.GetProvider("provider")
}

func (suite *VaultSuite) TestCreateKey() {
	provider := suite.GetProvider("createkey")

	key, err := provider.CreateKey(context.Background())
	assert.Nil(suite.T(), err)
	assert.NotNil(suite.T(), key)
}

func (suite *VaultSuite) TestSign() {
	provider := suite.GetProvider("testsign")

	key, err := provider.CreateKey(context.Background())
	assert.Nil(suite.T(), err)
	assert.NotNil(suite.T(), key)

	data := []byte("mydata")
	sig, signed, err := provider.Sign(context.Background(), data)
	assert.Nil(suite.T(), err)
	assert.NotNil(suite.T(), sig)
	assert.NotNil(suite.T(), signed)

	hash := sha256.Sum256(signed)
	ok := ecdsa.VerifyASN1(key, hash[:], sig)
	assert.True(suite.T(), ok)
}

func (suite *VaultSuite) TestVerify() {
	provider := suite.GetProvider("testverify")

	key, err := provider.CreateKey(context.Background())
	assert.Nil(suite.T(), err)
	assert.NotNil(suite.T(), key)

	data := []byte("mydata")
	sig, signed, err := provider.Sign(context.Background(), data)
	assert.Nil(suite.T(), err)
	assert.NotNil(suite.T(), sig)
	assert.NotNil(suite.T(), signed)

	err = provider.Verify(context.Background(), data, sig)
	assert.Nil(suite.T(), err)
}

func (suite *VaultSuite) TestNoProvider() {
	provider, err := kms.Get(context.Background(), "hashi://nonsense")
	require.Error(suite.T(), err)
	require.Nil(suite.T(), provider)
}

func TestVault(t *testing.T) {
	suite.Run(t, new(VaultSuite))
}

[dlorenc]

Do you want to add the e2e tests here as well? We might even be able to get the vault docker container running automatically in github actions, but we can handle that as a follow on.

[RichiCoder1]

Do you want to add the e2e tests here as well? We might even be able to get the vault docker container running automatically in github actions, but we can handle that as a follow on.

I'd def be game to contribute them, I just wasn't sure how ya'll wanted to handle E2E (especially since vault might be a one off case where E2E is "easy"). I'm actually using the vanilla Vault docker image locally, so that'd def work.

[RichiCoder1]

I've done e2e before with vault (https://github.com/hashicorp/vault-action/blob/master/.github/workflows/build.yml#L62) so luckily I already know all the component parts we need :D.

Edit: AWS KMS e2e might be possible when I add that via https://hub.docker.com/r/localstack/localstack so I could be game to go ahead and setup the infra for it here

[dlorenc]

This is great! A few nits that I wouldn't normally block on, but since you have to cleanup the golangci-lint issues anyway, might as well :)

[dlorenc]

nit: in other places here we use a build tag for this so you can still run "go test ./..." at the root: https://github.com/sigstore/cosign/blob/main/test/e2e_test.go#L16

[RichiCoder1]

Ohhh, noted! I'm still a bit of a golang novice, I was wondering how I could do that.

[RichiCoder1]

Random addendum: This still scopes the e2e test run to this subpackage. I figured it was somewhat wasteful to run the whole test suite again just to run e2e

[dlorenc]

nit: inline the if statement?

[dlorenc]

nit: inline?

[dlorenc]

nit: const and put at the top of the file?

[RichiCoder1]

Addressed feedback (thanks!) and when ahead and added a local run script for e2e like cosign.

[dlorenc]

Thanks for the awesome PR!