Document the various event files produced by CI #1066
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # Openshift CI produced files related to events | ||
|
|
||
| To aid in debugging, the Openshift CI (prow) produces various files related to events. The files | ||
| reside in the gather-must-gather and gather-extra subdirectories in the artifacts storage area. | ||
| You will see the "Artifacts" link in the upper right of a prow job (for examples, click on any | ||
| prow job from the [main prow page](https://prow.ci.openshift.org/). | ||
|
|
||
| This document provides details on where the files are located and what they contain. | ||
|
|
||
| Throughout this document, we provide links to actual files as examples but realize that after some | ||
| period of time, prow jobs are purged and the files will no longer be present. To get a recent/valid | ||
| link goto prow, change the search pattern to match a prow job you're interested in | ||
| (e.g., [link for e2e-aws jobs](https://prow.ci.openshift.org/?job=periodic*e2e-aws&state=success)), | ||
| and follow a similar path shown in the example links. | ||
|
|
||
| Artifacts will be in a path with a URL of the form: `.../{job-name}/{run-id}/artifacts/{job-slug}/{step-name}/artifacts/`. | ||
| For must-gather output, it will be in `.../{job-name}/{run-id}/artifacts/{job-slug}/{step-name}/artifacts/gather-must-gather`. | ||
| For gather-extra output, it will be in `.../{job-name}/{run-id}/artifacts/{job-slug}/{step-name}/artifacts/gather-extra`. | ||
|
|
||
| ## Events files | ||
|
|
||
| * event-filter.html | ||
| * included in the output of a `oc adm must-gather` | ||
| * contains events from namespaces gathered by `oc adm must-gather` plus events that exist in the kube-apiserver at the time `oc adm must-gather` was run. | ||
| Namespaces not listed as relatedResources for clusteroperators will be missing. | ||
| * located in the prow job stored artifacts at `.../gather-must-gather/artifacts/` and contained in the must-gather.tar | ||
| file in that same location | ||
| * [Example](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-gcp-ovn-upgrade/1505747388038385664/artifacts/e2e-gcp-ovn-upgrade/gather-must-gather/artifacts/event-filter.html) | ||
|
There was a problem hiding this comment. before we get down to individual step content like this, it's probably worth talking about the |
||
| * Tip: wait several seconds for the html page to finish rendering so you can use the text entry boxes to search | ||
|
|
||
| * events.json (same as `oc_cmds/events` mentioned below except as json) | ||
| * generated via `oc get events --all-namespaces -o json` (see [source code](https://github.com/openshift/release/blob/f5017d5136a740a4186477b02bed70047ade200b/ci-operator/step-registry/gather/extra/gather-extra-commands.sh#L61)) | ||
| * located in the prow job stored artifacts at `.../gather-extra/aftifacts/` | ||
| * [Example](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-gcp-ovn-upgrade/1505747388038385664/artifacts/e2e-gcp-ovn-upgrade/gather-extra/artifacts/events.json) | ||
|
|
||
| * events (same as `gather-extra/artifacts/events.json` except not json) | ||
| * generated via `oc get events --all-namespaces` (see [source code](https://github.com/openshift/release/blob/f5017d5136a740a4186477b02bed70047ade200b/ci-operator/step-registry/gather/extra/gather-extra-commands.sh#L62)). This file has all the events present in the apiserver at the time it was generated. | ||
| * located in the prow job stored artifacts at `.../gahter-extra/artifacts/oc_cmds/events` | ||
| * [Example](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-gcp-ovn-upgrade/1505747388038385664/artifacts/e2e-gcp-ovn-upgrade/gather-extra/artifacts/oc_cmds/events) | ||
|
|
||
| * resource-events_(date)-(number).zip | ||
|
There was a problem hiding this comment. these contain kube-apiserver events. |
||
| * located in the prow job stored artifacts at `.../<test_name>/openshift-e2e-test/artifacts/junit` | ||
| * There will be one file per invocation of `openshift-tests` | ||
| * Contains a watch of all events including "delete" related events | ||
| * These contain kube-apiserver events | ||
| * [Example1](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-gcp-ovn-upgrade/1505747388038385664/artifacts/e2e-gcp-ovn-upgrade/openshift-e2e-test/artifacts/junit/resource-events_20220321-040307.zip), [Example2](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-gcp-ovn-upgrade/1505747388038385664/artifacts/e2e-gcp-ovn-upgrade/openshift-e2e-test/artifacts/junit/resource-events_20220321-052002.zip) | ||
| * Here's a sample of the contents: | ||
|
|
||
| ```bash | ||
| ./default: | ||
| total 332 | ||
| -rw-rw-r-- 1 dperique dperique 339013 Dec 31 1979 events.json | ||
|
|
||
| ./e2e-check-for-dns-availability-1546: | ||
| total 68 | ||
| -rw-rw-r-- 1 dperique dperique 67899 Dec 31 1979 events.json | ||
|
|
||
| ./e2e-k8s-service-load-balancer-with-pdb-new-485: | ||
| total 88 | ||
| -rw-rw-r-- 1 dperique dperique 86468 Dec 31 1979 events.json | ||
| ``` | ||
|
|
||
| * e2e-events_(date)-(number).json | ||
|
There was a problem hiding this comment. there are monitor events, not API events. the monitor is part of the openshift-tests invocation. They do not have the same info as the API events. There was a problem hiding this comment. Would it be worth the bother of renaming these? I believe this is fairly confusing for anyone who encounters them. There was a problem hiding this comment.
I'd merge it. I'd also merge a PR to write the raw events separate from the computed ones. |
||
| * located in the prow job stored artifacts at `.../artifcts/<test_name>/openshift-e2e-test/artifacts/junit`; there are two of them | ||
| * these are monitor events, not API events, produced by `monitor` which is part of the openshift-tests invocation. They do not have the same info as the API events. | ||
| * The first one is from the initial installation | ||
| * The second one is from the upgrade (if initial installation fails, this one will not be present) | ||
| * [Example1](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-gcp-ovn-upgrade/1505747388038385664/artifacts/e2e-gcp-ovn-upgrade/openshift-e2e-test/artifacts/junit/e2e-events_20220321-040307.json), [Example2](https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-gcp-ovn-upgrade/1505747388038385664/artifacts/e2e-gcp-ovn-upgrade/openshift-e2e-test/artifacts/junit/e2e-events_20220321-052002.json) | ||
|
|
||
| * core/events.yaml | ||
| * located in the must-gather.tar at `.../namespaces/(aNamespace)/core/events.yaml`; there is one per namespace. | ||
| * produced by [must-gather](https://github.com/openshift/must-gather/) using `oc adm inspect`. | ||
|
|
||
| * We talked about events related to [Devan's PR](https://github.com/openshift/origin/pull/26862) which we can document | ||
| more later | ||
| * These events are Monitor events (received by a Watcher) | ||
|
There was a problem hiding this comment. I thought these were the e2e-events above but now I'm not so sure. :) |
||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
how do "events that exist in the kube-apiserver" differ from the ones must-gather collected? Ah, because must-gather is only walking
relatedResources.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that true? I'd have figured events falls into a category of things must-gather would grab globally without need of related resources.