step-registry: add Origin E2E test step #6965
Conversation
openshift-ci-robot
added
size/M
| @@ -202,3 +202,5 @@ tests: | |||
| steps: | |||
| cluster_profile: aws | |||
| workflow: ipi | |||
| test: | |||
| - ref: origin-e2e | |||
There was a problem hiding this comment.
we likely want a workflow for this specific thing since everyone will have it
|
/retest Please review the full test history for this PR and help us cut down flakes. |
21 similar comments
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
|
/retest Please review the full test history for this PR and help us cut down flakes. |
| fi | ||
|
|
||
| # set up cloud-provider-specific env vars | ||
| KUBE_SSH_BASTION="$( oc --insecure-skip-tls-verify get node -l node-role.kubernetes.io/master -o 'jsonpath={.items[0].status.addresses[?(@.type=="ExternalIP")].address}' ):22" |
There was a problem hiding this comment.
Adding notes here so they're captured (as I'm reviewing the step registry)
This is wrong, bastion is an optional requirement for some steps, not required for all.
There was a problem hiding this comment.
In fact, we should probably move all "SSH requiring" things out to their own step. It might even be appropriate for that always to be expecting the bastion. Will have to think about that.
| export KUBE_SSH_USER=core | ||
| ;; | ||
| azure4) export TEST_PROVIDER=azure;; | ||
| *) echo >&2 "Unsupported cluster type '${CLUSTER_TYPE}'"; exit 1;; |
There was a problem hiding this comment.
This is wrong, the test suite correctly infers platform starting from 4.3. Remove this line.
There was a problem hiding this comment.
These tests are also executed in 4.1 and 4.2.
| popd | ||
| fi | ||
|
|
||
| test_suite=openshift/conformance/parallel |
There was a problem hiding this comment.
This should always be injected from a variable, all "standard" suites should be parameterized by the test (possibly by the workflow, but either way, this is wrong).
There was a problem hiding this comment.
We're working on adding test/step parameters.
| fi | ||
|
|
||
| openshift-tests run "${test_suite}" \ | ||
| --provider "${TEST_PROVIDER}" \ |
There was a problem hiding this comment.
This must be conditional based on whether TEST_PROVIDER is set. If TEST_PROVIDER was not set, this flag should not be set.
| @@ -0,0 +1,12 @@ | |||
| ref: | |||
| as: origin-e2e-test | |||
There was a problem hiding this comment.
We'll need to rename all these steps, this is not origin. Probably this should be openshift-test-e2e and successive variants will be openshift-test-e2e-serial openshift-test-e2e-disruptive, etc.
The name of our platform is openshift, okd and ocp are variants that are expected to run and pass the same test suites.
|
@smarterclayton: can you open a PR with the changes you propose? It will be much faster than me, without any knowledge of the test suite, trying to understand your comments. Also keep in mind this is almost a direct copy of the templates, so some of these issues exist there as well. |
…ST_SUITE Taking advantage of openhsift/ci-tools@62e7498d66 (ci-operator multi-stage: add step parameters, 2020-05-27, openshift/ci-tools#854). This commit doesn't override the defaults yet, but a useful override would be: TEST_COMMAND=run-upgrade TEST_SUITE=all This replaces test-suite.txt, which landed in a2fd8c3 (step-registry: add Origin E2E test step, 2020-01-31, openshift#6965), and also had no consumers.
…step These have been part of the e2e test step since that step landed in a2fd8c3 (step-registry: add Origin E2E test step, 2020-01-31, openshift#6965). But pushing manifests is more of an install thing than a test thing. For example, if a user swaps in their own tests, we still want the cluster to push insights if their cluster profile includes the token). The logic I'm replacing was just handling the insights operator. To allow us to recycle the same logic for additional manifests, I've carved out generic handling for ${CLUSTER_PROFILE_DIR} and ${SHARED_DIR}. The day-2 naming avoids conflicts with the ipi/install step that is already consuming ${SHARED_DIR}/manifest_*.yml Glob expansion should have stable ordering; from POSIX [1]: If the pattern matches any existing filenames or pathnames, the pattern shall be replaced with those filenames and pathnames, sorted according to the collating sequence in effect in the current locale. Ideally we'd update the cluster profiles themselves to store the config in day-2-manifests-insights-live.yaml (via a core-services/ci-secret-bootstrap/_config.yaml edit?). But we can't do that until we've removed or ported the old-style templates under ci-operator/templates, so I'm leaving that off for now and keeping a HACK check to copy the current profile's file into the newly-expected target location. The previous logic also used an error-ignoring '|| true' since at least b887d20 (Deploy the insights secret for the support-operator in all templates, 2019-06-04, openshift#3951). That commit didn't motivate its use, so I'm dropping it here. If the step fails, figuring out the culprit should be pretty straightforward. [1]: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13_03
… openshift-tests The end-to-end step originated without the background/wait pattern in a2fd8c3 (step-registry: add Origin E2E test step, 2020-01-31, openshift#6965). But as described in 4472ace (ci-operator/templates/openshift/installer: Restore backgrounded 'create cluster', 2019-01-23, openshift#2680), we want the background-and-wait pattern to make openshift-tests a child process that will receive TERMs via this step's existing 'trap' handler. openshift-tests can take hours, and when the step gets a TERM, we want to quickly pass that through to the slow children, so they exit, so the step can exit, so we have plenty of time for asset uploading and subsequent gather and teardown steps. That gives us the resources we need to figure out why the test was abnormally slow.
No description provided.