Skip to content

🌱 OPRUN-3956: Add experimental e2e tests #2064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 29, 2025
Merged

🌱 OPRUN-3956: Add experimental e2e tests #2064

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
07a1b51
Add experimental e2e tests
tmshort Jun 27, 2025
8eff28d
fixup! Add experimental e2e tests
tmshort Jun 29, 2025
1a42acf
fixup! Add experimental e2e tests
tmshort Jun 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .github/workflows/e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,33 @@ jobs:
flags: e2e
token: ${{ secrets.CODECOV_TOKEN }}

experimental-e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: actions/setup-go@v5
with:
go-version-file: go.mod

- name: Run e2e tests
run: ARTIFACT_PATH=/tmp/artifacts make test-experimental-e2e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For curiosity, what was the issue that was not finding the path?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CollectTestArtifacts() function was not being called; one of the things it does is create the /tmp/artifacts directory.


- uses: actions/upload-artifact@v4
if: failure()
with:
name: experimental-e2e-artifacts
path: /tmp/artifacts/

- uses: codecov/codecov-action@v5.4.3
with:
disable_search: true
files: coverage/e2e.out
flags: experimental-e2e
token: ${{ secrets.CODECOV_TOKEN }}

upgrade-e2e:
runs-on: ubuntu-latest
steps:
Expand Down
15 changes: 14 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ endif
KUSTOMIZE_STANDARD_OVERLAY := config/overlays/standard
KUSTOMIZE_STANDARD_E2E_OVERLAY := config/overlays/standard-e2e
KUSTOMIZE_EXPERIMENTAL_OVERLAY := config/overlays/experimental
KUSTOMIZE_EXPERIMENTAL_E2E_OVERLAY := config/overlays/experimental-e2e

export RELEASE_MANIFEST := operator-controller.yaml
export RELEASE_INSTALL := install.sh
Expand All @@ -84,6 +85,7 @@ MANIFEST_HOME := ./manifests
STANDARD_MANIFEST := ./manifests/standard.yaml
STANDARD_E2E_MANIFEST := ./manifests/standard-e2e.yaml
EXPERIMENTAL_MANIFEST := ./manifests/experimental.yaml
EXPERIMENTAL_E2E_MANIFEST := ./manifests/experimental-e2e.yaml
CATALOGS_MANIFEST := ./manifests/default-catalogs.yaml

# Manifest used by kind-deploy, which may be overridden by other targets
Expand All @@ -110,7 +112,7 @@ SOURCE_MANIFEST := $(STANDARD_MANIFEST)

.PHONY: help
help: #HELP Display essential help.
@awk 'BEGIN {FS = ":[^#]*#HELP"; printf "\nUsage:\n make \033[36m<target>\033[0m\n\n"} /^[a-zA-Z_0-9-]+:.*#HELP / { printf " \033[36m%-17s\033[0m %s\n", $$1, $$2 } ' $(MAKEFILE_LIST)
@awk 'BEGIN {FS = ":[^#]*#HELP"; printf "\nUsage:\n make \033[36m<target>\033[0m\n\n"} /^[a-zA-Z_0-9-]+:.*#HELP / { printf " \033[36m%-21s\033[0m %s\n", $$1, $$2 } ' $(MAKEFILE_LIST)

.PHONY: help-extended
help-extended: #HELP Display extended help.
Expand Down Expand Up @@ -157,6 +159,7 @@ manifests: $(CONTROLLER_GEN) $(KUSTOMIZE) #EXHELP Generate WebhookConfiguration,
$(KUSTOMIZE) build $(KUSTOMIZE_STANDARD_OVERLAY) > $(STANDARD_MANIFEST)
$(KUSTOMIZE) build $(KUSTOMIZE_STANDARD_E2E_OVERLAY) > $(STANDARD_E2E_MANIFEST)
$(KUSTOMIZE) build $(KUSTOMIZE_EXPERIMENTAL_OVERLAY) > $(EXPERIMENTAL_MANIFEST)
$(KUSTOMIZE) build $(KUSTOMIZE_EXPERIMENTAL_E2E_OVERLAY) > $(EXPERIMENTAL_E2E_MANIFEST)

.PHONY: generate
generate: $(CONTROLLER_GEN) #EXHELP Generate code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.
Expand Down Expand Up @@ -206,6 +209,10 @@ test: manifests generate fmt lint test-unit test-e2e #HELP Run all tests.
e2e: #EXHELP Run the e2e tests.
go test -count=1 -v ./test/e2e/...

.PHONY: experimental-e2e
experimental-e2e: #EXHELP Run the experimental e2e tests.
go test -count=1 -v ./test/experimental-e2e/...

E2E_REGISTRY_NAME := docker-registry
E2E_REGISTRY_NAMESPACE := operator-controller-e2e

Expand Down Expand Up @@ -261,6 +268,12 @@ test-e2e: KIND_CLUSTER_NAME := operator-controller-e2e
test-e2e: GO_BUILD_EXTRA_FLAGS := -cover
test-e2e: run image-registry prometheus e2e e2e-metrics e2e-coverage kind-clean #HELP Run e2e test suite on local kind cluster

.PHONY: test-experimental-e2e
test-experimental-e2e: SOURCE_MANIFEST := $(EXPERIMENTAL_E2E_MANIFEST)
test-experimental-e2e: KIND_CLUSTER_NAME := operator-controller-e2e
test-experimental-e2e: GO_BUILD_EXTRA_FLAGS := -cover
test-experimental-e2e: run image-registry prometheus experimental-e2e e2e-metrics e2e-coverage kind-clean #HELP Run experimental e2e test suite on local kind cluster

.PHONY: prometheus
prometheus: PROMETHEUS_NAMESPACE := olmv1-system
prometheus: PROMETHEUS_VERSION := v0.83.0
Expand Down
2 changes: 1 addition & 1 deletion codecov.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
codecov:
notify:
after_n_builds: 2
after_n_builds: 3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change related ?

Copy link
Contributor Author

@tmshort tmshort Jun 29, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as the experimental-e2e pushes code coverage. There are three builds that do code-coverage:

  • e2e
  • unit
  • experimental-e2e

This is supposed to make codecov wait until it gets three build pushes (the above).
See: https://app.codecov.io/gh/operator-framework/operator-controller/pull/2064/flags


# Configure the paths to include in coverage reports.
# Exclude documentation, YAML configurations, and test files.
Expand Down
9 changes: 9 additions & 0 deletions config/overlays/experimental-e2e/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# kustomization file for all the experimental e2e's
# DO NOT ADD A NAMESPACE HERE
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
components:
- ../../components/base/experimental
- ../../components/e2e
# This must be last due to namespace overwrite issues of the ca
- ../../components/cert-manager
Loading
Loading