Establish end-to-end testing procedure #38
Conversation
Introduce a consistent method for testing changes against a 4.0 cluster. 1. Provide an uninstaller. 2. Provide a way to publish and install a locally built operator. 3. Document the procedure for executing upstream e2e tests against the cluster. 4. Remove the existing integration tests, which are already covered comprehensively by the e2e tests and which were misleading as to whether the installed operator setup was functional.
openshift-ci-robot
added
size/XL
|
Awaiting merge of #37 since this depends on it. /hold |
openshift-ci-robot
added
the
do-not-merge/hold
|
/retest |
| To run all of them, try: | ||
|
|
||
| ``` | ||
| $ FOCUS='DNS' SKIP='\[Disabled:.+\]|\[Disruptive\]|\[Skipped\]|\[Slow\]|\[Flaky\]|\[local\]|\[Local\]' TEST_ONLY=1 test/extended/conformance.sh |
There was a problem hiding this comment.
test/extended/conformance.sh file from which repo? cluster-dns-operator repo doesn't contain this file path.
There was a problem hiding this comment.
From OpenShift... updated docs.
| To run the bare minimum smoke test, try: | ||
|
|
||
| ``` | ||
| $ FOCUS='should provide DNS for the cluster' SKIP='\[Disabled:.+\]|\[Disruptive\]|\[Skipped\]|\[Slow\]|\[Flaky\]|\[local\]|\[Local\]' TEST_ONLY=1 test/extended/conformance.sh |
There was a problem hiding this comment.
Same as above, test/extended/conformance.sh from which repo?
There was a problem hiding this comment.
From OpenShift... updated docs.
| # Required for the operator-sdk. | ||
| export KUBERNETES_CONFIG="${KUBECONFIG}" | ||
|
|
||
| go test -v -tags integration ./test/integration |
There was a problem hiding this comment.
What are we testing? integration_test.go is empty
Are we not removing integration tests in favor of comprehensive origin/upstream e2e dns tests?
There was a problem hiding this comment.
I can imagine adding some other more focused operator-centric tests here later, didn't want to tear down all the machinery yet if it's benign.
| @@ -1,28 +1,9 @@ | |||
| package manifests | |||
There was a problem hiding this comment.
Do we still need to keep this file?
There was a problem hiding this comment.
I can imagine adding some other more focused operator-centric tests here later, didn't want to tear down all the machinery yet if it's benign.
|
#37 merged. /hold cancel |
openshift-ci-robot
removed
the
do-not-merge/hold
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ironcladlou, pravisankar The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest |
|
/retest |
1 similar comment
|
/retest |
Introduce a consistent method for testing changes against a 4.0 cluster.
comprehensively by the e2e tests and which were misleading as to whether the
installed operator setup was functional.