Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensure the $ARTIFACT_DIR variable is respected #2721

Merged
merged 2 commits into from
Apr 7, 2022
Merged

Ensure the $ARTIFACT_DIR variable is respected #2721

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6deef78
.gitignore: Filter out the nested .kube cache
timflannagan Mar 29, 2022
f82e274
Makefile,test: Ensure that the $ARTIFACT_DIR is respected
timflannagan Mar 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -455,5 +455,5 @@ apiserver.key

!vendor/**
test/e2e-local.image.tar

test/e2e/.kube
dist/
3 changes: 1 addition & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,11 +139,10 @@ E2E_TEST_NS ?= operators
e2e:
$(GINKGO) $(E2E_OPTS) $(or $(run), ./test/e2e) $< -- -namespace=$(E2E_TEST_NS) -olmNamespace=$(E2E_INSTALL_NS) -dummyImage=bitnami/nginx:latest $(or $(extra_args), -kubeconfig=${KUBECONFIG})


# See workflows/e2e-tests.yml See test/e2e/README.md for details.
.PHONY: e2e-local
e2e-local: BUILD_TAGS="json1 experimental_metrics"
e2e-local: extra_args=-kind.images=../test/e2e-local.image.tar -test-data-dir=../test/e2e/testdata
e2e-local: extra_args=-kind.images=../test/e2e-local.image.tar -test-data-dir=../test/e2e/testdata -gather-artifacts-script-path=../test/e2e/collect-ci-artifacts.sh
e2e-local: run=bin/e2e-local.test
e2e-local: bin/e2e-local.test test/e2e-local.image.tar
e2e-local: e2e
Expand Down
19 changes: 15 additions & 4 deletions test/e2e/ctx/ctx.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,9 @@ type TestContext struct {
e2eClient *util.E2EKubeClient
ssaClient *controllerclient.ServerSideApplier

kubeconfigPath string
artifactsDir string
kubeconfigPath string
artifactsDir string
artifactsScriptPath string

scheme *runtime.Scheme

Expand Down Expand Up @@ -132,8 +133,7 @@ func (ctx TestContext) DumpNamespaceArtifacts(namespace string) error {
"KUBECONFIG=" + kubeconfigPath,
}

// compiled test binary running e2e tests is run from the root ./bin directory
cmd := exec.Command("../test/e2e/collect-ci-artifacts.sh")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into a sourcing issue when running the make e2e command locally as it's working directory points to ./test/e2e, so there's some skew between the working directory when using the kind or kubeconfig provisioners.

cmd := exec.Command(ctx.artifactsScriptPath)
cmd.Env = append(cmd.Env, envvars...)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
Expand All @@ -153,6 +153,17 @@ func setDerivedFields(ctx *TestContext) error {
return fmt.Errorf("nil RESTClient")
}

if ctx.artifactsDir == "" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this poorly interact with some downstream only code: https://github.com/openshift/operator-framework-olm/blob/master/staging/operator-lifecycle-manager/test/e2e/e2e_test.go#L69?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be in the clear there as we're building up the junit reporter for ginkgo vs. telling either of the provisioner implementation where to dump testing artifacts when configured.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: we could bubble up that downstream-only change to make upstream backwards-compatible to eliminate any skew between implementations (which might be relevant for any effort around ginkgo v1 -> v2)?

if artifactsDir := os.Getenv("ARTIFACT_DIR"); artifactsDir != "" {
ctx.artifactsDir = artifactsDir
}
}
if ctx.artifactsScriptPath == "" {
if scriptPath := os.Getenv("E2E_ARTIFACT_SCRIPT"); scriptPath != "" {
ctx.artifactsScriptPath = scriptPath
}
}

kubeClient, err := operatorclient.NewClientFromRestConfig(ctx.restConfig)
if err != nil {
return err
Expand Down
5 changes: 1 addition & 4 deletions test/e2e/ctx/provisioner_kind.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
//go:build kind
// +build kind

package ctx
Expand Down Expand Up @@ -135,10 +136,6 @@ func Provision(ctx *TestContext) (func(), error) {
return nil, fmt.Errorf("error loading kubeconfig: %s", err.Error())
}
ctx.restConfig = restConfig

if artifactsDir := os.Getenv("ARTIFACTS_DIR"); artifactsDir != "" {
ctx.artifactsDir = artifactsDir
}
ctx.kubeconfigPath = kubeconfigPath

var once sync.Once
Expand Down
10 changes: 10 additions & 0 deletions test/e2e/e2e_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,12 @@ var (
"dummy image to treat as an operator in tests",
)

collectArtifactsScriptPath = flag.String(
"gather-artifacts-script-path",
"./collect-ci-artifacts.sh",
"configures the relative/absolute path to the script resposible for collecting CI artifacts",
)

testdataPath = flag.String(
"test-data-dir",
"./testdata",
Expand Down Expand Up @@ -79,6 +85,9 @@ var _ = BeforeSuite(func() {
// This flag can be deprecated in favor of the kubeconfig provisioner:
os.Setenv("KUBECONFIG", *kubeConfigPath)
}
if collectArtifactsScriptPath != nil && *collectArtifactsScriptPath != "" {
os.Setenv("E2E_ARTIFACT_SCRIPT", *collectArtifactsScriptPath)
}

Comment on lines +88 to 91
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: just a hack to inject state so the ctx package can derive where to find the artifacts bash script path location.

testNamespace = *namespace
operatorNamespace = *olmNamespace
Expand Down Expand Up @@ -114,6 +123,7 @@ var _ = BeforeSuite(func() {
HaveLen(1),
ContainElement(Not(BeZero())),
))

_, err := fetchCatalogSourceOnStatus(ctx.Ctx().OperatorClient(), "operatorhubio-catalog", operatorNamespace, catalogSourceRegistryPodSynced)
Expect(err).NotTo(HaveOccurred())

Expand Down