sigstore test env #1630

sallyom · 2022-04-26T21:37:43Z

This PR introduces 2 make targets:

make sigstore-testenv-up
make sigstore-testenv-down

This will enable testing for addition of sigstore image signing/verification

/cc @lukehinds @mtrmac @vrothberg :)

sallyom · 2022-04-28T14:36:56Z

@vrothberg @mtrmac I've tested this setup by running subset of sigstore integration tests locally like so:

$ cd path/to/containers/skopeo 
$ make sigstore-testenv-up
$ cd  path/to/sigstore
$ curl -o e2e-test.sh https://gist.githubusercontent.com/sallyom/9f828f46a6ade7a98eaefecd56190fa2/raw/6501803c4c3170aa5d41c8acfc0ad643d4a5a912/e2e-test.sh
$ chmod +x e2e-test.sh && ./e2e-test.sh
$ cd back/to/containers/skopeo
$ make sigstore-testenv-down

output:

$ ./e2e-test.sh 
+ echo 'running tests'
running tests
+ export VAULT_TOKEN=testtoken
---env-vars-set
+ go test -tags e2e -count=1 ./...
ok  	github.com/sigstore/sigstore/pkg/cryptoutils	6.619s
?   	github.com/sigstore/sigstore/pkg/oauth	[no test files]
ok  	github.com/sigstore/sigstore/pkg/oauth/internal	0.007s
ok  	github.com/sigstore/sigstore/pkg/oauth/oidc	1.160s
ok  	github.com/sigstore/sigstore/pkg/oauthflow	1.581s
ok  	github.com/sigstore/sigstore/pkg/signature	0.356s
ok  	github.com/sigstore/sigstore/pkg/signature/dsse	0.013s
ok  	github.com/sigstore/sigstore/pkg/signature/kms	0.016s
ok  	github.com/sigstore/sigstore/pkg/signature/kms/aws	0.851s
ok  	github.com/sigstore/sigstore/pkg/signature/kms/azure	0.030s
ok  	github.com/sigstore/sigstore/pkg/signature/kms/fake	0.007s
ok  	github.com/sigstore/sigstore/pkg/signature/kms/gcp	0.009s
ok  	github.com/sigstore/sigstore/pkg/signature/kms/hashivault	0.064s
?   	github.com/sigstore/sigstore/pkg/signature/options	[no test files]
ok  	github.com/sigstore/sigstore/pkg/signature/payload	0.004s
ok  	github.com/sigstore/sigstore/pkg/signature/ssh	0.059s

sallyom · 2022-04-28T14:43:20Z

To avoid using docker.io images (rate limits) EDIT: ~~I've pushed necessary images~~ the following images should be pushed somewhere like quay.io/libpod/..

docker.io/library/vault:1.9.6
docker.io/library/localstack/localstack:0.12.16
docker.io/dexidp/dex:v2.31.1

latest localstack 0.14.2 but 0.12.6 in use by sigstore atm

The above docker.io images are referenced in this PR

vrothberg

Thanks!

Before merging, we should probably push the images to quay.io/libpod/sigstore-testing-*.

vrothberg · 2022-05-02T11:32:43Z

contrib/cirrus/runner.sh

@@ -119,7 +119,12 @@ _run_integration() {
    # Ensure we start with a clean-slate
    podman system reset --force

+    # uncomment when sigstore integration tests merge
+    # make sigstore-testenv-up


I think this should be uncommented already to make sure that it's working and that it will continue to work in the future.

uncommented, thanks

vrothberg · 2022-05-02T11:33:55Z

hack/sigstore-setup/docker-compose.yml

+version: "3.8"
+services:
+  vault:
+    image: quay.io/sallyom/vault:latest


Let's avoid using the latest tag. The image may be updated in the future and we need to make sure that old/stable branches continue to work.

can we move these images to a common quay.io directory/repo? I know we have one for libpod, do we have one for skopeo?

Using the libpod one should be OK given we need these tests in Podman CI as well.

updated from localstack:latest to localstack:0.12.16 (currently used in sigstore)

vrothberg · 2022-05-02T11:34:36Z

hack/sigstore-setup/docker-compose.yml

+    ports:
+      - "5556:5556"
+    volumes:
+      - ./dex-config.yml:/etc/dex/dex-config.yml:z


Do we need this dex-config.yml?

EDIT: keeping the dex-config.yml & the dex volume in docker-compose.yml ~~nope, I've removed this and also updated the gist test script (not included in this PR but to prove this setup works until skopeo adds sigstore e2es)~~

vrothberg · 2022-05-02T11:35:37Z

hack/sigstore-setup/sigstore-setup.sh

+  export AWS_REGION=us-east-1
+  export AWS_ENDPOINT=localhost:4566
+  export AWS_TLS_INSECURE_SKIP_VERIFY=1
+  export OIDC_ISSUER=http://127.0.0.1:5556/auth


Should we export the IPs in the Makefile to make them accessible to all tests?

right, yup - updated/added to Makefile

sallyom · 2022-05-06T14:36:32Z

@TomSweeneyRedHat @vrothberg thanks for the reviews! I've updated this PR, ptal

vrothberg

Tests are failing at the moment as integration/copy_test.go wants to use port 5556 which is now blocked by dex.

sallyom · 2022-05-13T12:26:36Z

Tests are failing at the moment as integration/copy_test.go wants to use port 5556 which is now blocked by dex.

fixed, updated

TomSweeneyRedHat · 2022-05-16T19:11:24Z

@sallyom another rebase is needed here

sallyom · 2022-05-17T14:54:35Z

@sallyom another rebase is needed here

rebased!

sallyom · 2022-05-19T19:21:43Z

@TomSweeneyRedHat @vrothberg anything else needed? thanks!

vrothberg · 2022-05-23T11:41:34Z

@sallyom, it would be helpful to document how this can be used integration tests. For instance, which env variables should be used for the local fulcio and recor instance?

@mtrmac WDYT?

sallyom · 2022-05-23T18:40:36Z

@vrothberg @mtrmac I documented above (and in the Makefile) the env vars required. There's an example for how to use this setup against sigstore/sigstore e2e tests in this gist, too

export VAULT_TOKEN=testtoken
export VAULT_ADDR=http://localhost:8200/

export AWS_ACCESS_KEY_ID=test
export AWS_SECRET_ACCESS_KEY=test
export AWS_REGION=us-east-1
export AWS_ENDPOINT=localhost:4566
export AWS_TLS_INSECURE_SKIP_VERIFY=1
export OIDC_ISSUER=http://127.0.0.1:5556/auth
export OIDC_ID=sigstore

rhatdan · 2022-07-12T13:29:46Z

@sallyom @mtrmac @vrothberg Where are we with this one? Now most of the sigstore stuff is merged?

sallyom · 2022-07-13T17:14:31Z

@sallyom @mtrmac @vrothberg Where are we with this one? Now most of the sigstore stuff is merged?

what will help is if there is a WIP/PoC bringing in whatever is necessary from sigstore/cosign - then it will become clear what should be included in/ported to sigstore/sigstore

relevant upstream:
sigstore/signature library: https://github.com/sigstore/sigstore/tree/main/pkg/signature
ongoing work with sigstore/gitsign to move code to sigstore/sigstore
sigstore/gitsign#62
and work to move providers into separate repos:
sigstore/cosign#1867

Signed-off-by: Sally O'Malley <somalley@redhat.com>

github-actions · 2022-08-13T00:01:17Z

A friendly reminder that this PR had no activity for 30 days.

rhatdan · 2022-09-13T17:21:17Z

@sallyom @vrothberg @mtrmac do we care about this PR at this point? Should it be fixed or closed?

sallyom · 2022-10-21T18:33:14Z

closing this for now and will reopen if necessary after examining current sigstore implementation

sallyom force-pushed the ci-sigstore branch 6 times, most recently from f4582ca to 5bce179 Compare April 27, 2022 16:52

sallyom changed the title ~~WIP: Sigstore test env~~ sigstore test env Apr 28, 2022

sallyom force-pushed the ci-sigstore branch from 5bce179 to b1cdbc0 Compare April 29, 2022 23:07

vrothberg reviewed May 2, 2022

View reviewed changes

sallyom force-pushed the ci-sigstore branch from b1cdbc0 to d11a780 Compare May 6, 2022 14:25

sallyom force-pushed the ci-sigstore branch from d11a780 to e16d5a8 Compare May 9, 2022 21:23

vrothberg reviewed May 10, 2022

View reviewed changes

sallyom force-pushed the ci-sigstore branch 3 times, most recently from 8330f02 to 8a60f9b Compare May 13, 2022 00:08

sallyom force-pushed the ci-sigstore branch from 8a60f9b to 4833479 Compare May 17, 2022 14:54

sallyom force-pushed the ci-sigstore branch from 4833479 to d4b27ea Compare May 23, 2022 18:38

sallyom force-pushed the ci-sigstore branch from d4b27ea to c3cee53 Compare May 27, 2022 15:06

sallyom force-pushed the ci-sigstore branch from c3cee53 to 2d78f6f Compare June 14, 2022 14:48

sallyom force-pushed the ci-sigstore branch from 2d78f6f to 4ae3544 Compare June 24, 2022 15:38

sallyom force-pushed the ci-sigstore branch from 4ae3544 to 7ee804a Compare July 12, 2022 11:05

sallyom added 2 commits July 13, 2022 15:34

Sigstore test env

a733aea

Signed-off-by: Sally O'Malley <somalley@redhat.com>

integration: update DockerRegistryURL

8f4fbb5

Signed-off-by: Sally O'Malley <somalley@redhat.com>

sallyom force-pushed the ci-sigstore branch from 7ee804a to 8f4fbb5 Compare July 13, 2022 19:41

github-actions bot added the stale-pr label Aug 13, 2022

sallyom closed this Oct 21, 2022

github-actions bot added the locked - please file new issue/PR label Sep 18, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sigstore test env #1630

sigstore test env #1630

sallyom commented Apr 26, 2022 •

edited

Loading

sallyom commented Apr 28, 2022 •

edited

Loading

sallyom commented Apr 28, 2022 •

edited

Loading

vrothberg left a comment

vrothberg May 2, 2022

sallyom May 6, 2022

vrothberg May 2, 2022

TomSweeneyRedHat May 2, 2022

vrothberg May 2, 2022

sallyom May 6, 2022

vrothberg May 2, 2022

sallyom May 6, 2022 •

edited

Loading

vrothberg May 2, 2022

sallyom May 6, 2022 •

edited

Loading

sallyom commented May 6, 2022

vrothberg left a comment

sallyom commented May 13, 2022

TomSweeneyRedHat commented May 16, 2022

sallyom commented May 17, 2022

sallyom commented May 19, 2022

vrothberg commented May 23, 2022

sallyom commented May 23, 2022

rhatdan commented Jul 12, 2022

sallyom commented Jul 13, 2022

github-actions bot commented Aug 13, 2022

rhatdan commented Sep 13, 2022

sallyom commented Oct 21, 2022

sigstore test env #1630

sigstore test env #1630

Conversation

sallyom commented Apr 26, 2022 • edited Loading

sallyom commented Apr 28, 2022 • edited Loading

output:

sallyom commented Apr 28, 2022 • edited Loading

vrothberg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sallyom May 6, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sallyom May 6, 2022 • edited Loading

Choose a reason for hiding this comment

sallyom commented May 6, 2022

vrothberg left a comment

Choose a reason for hiding this comment

sallyom commented May 13, 2022

TomSweeneyRedHat commented May 16, 2022

sallyom commented May 17, 2022

sallyom commented May 19, 2022

vrothberg commented May 23, 2022

sallyom commented May 23, 2022

rhatdan commented Jul 12, 2022

sallyom commented Jul 13, 2022

github-actions bot commented Aug 13, 2022

rhatdan commented Sep 13, 2022

sallyom commented Oct 21, 2022

sallyom commented Apr 26, 2022 •

edited

Loading

sallyom commented Apr 28, 2022 •

edited

Loading

sallyom commented Apr 28, 2022 •

edited

Loading

sallyom May 6, 2022 •

edited

Loading

sallyom May 6, 2022 •

edited

Loading