Kops - Don't publish the latest-ci marker from periodics

[rifelpet]

Context

The sequence of Kops version markers is:

1. The kops-postsubmit-push-to-staging postsubmit job writes to latest-ci.txt (make target, example job log)

Step #1 - "artifacts": gsutil -h "Cache-Control:private, max-age=0, no-transform" cp /workspace/.build/upload/latest.txt gs://k8s-staging-kops/kops/releases/markers/master/latest-ci.txt

2. The e2e-kops-pipeline-updown-kopsmaster periodic job reads from latest-ci.txt and if creating and tearing down a cluster is successful, writes the same marker to latest-ci-updown-green.txt

test-infra/config/jobs/kubernetes/kops/kops-periodics-pipeline.yaml

Lines 36 to 37 in 5bc25c3

	--kops-version-marker=https://storage.googleapis.com/k8s-staging-kops/kops/releases/markers/master/latest-ci.txt \
	--publish-version-marker=gs://k8s-staging-kops/kops/releases/markers/master/latest-ci-updown-green.txt \

3. Almost all other periodic jobs then read from latest-ci-updown-green.txt:

test-infra/config/jobs/kubernetes/kops/build_jobs.py

Lines 81 to 82 in 5bc25c3

	if kops_version is None:
	kops_deploy_url = marker_updown_green(None)

---

Problem

The e2e-kops-aws-k8s-latest periodic job has been reading from latest-ci-updown-green.txt and writing to latest-ci.txt:

test-infra/config/jobs/kubernetes/kops/kops-periodics-versions.yaml

Lines 36 to 38 in 5749033

	--kops-version-marker=https://storage.googleapis.com/k8s-staging-kops/kops/releases/markers/master/latest-ci-updown-green.txt \
	--publish-version-marker=gs://k8s-staging-kops/kops/releases/markers/master/latest-ci.txt \
	--kubernetes-version=https://storage.googleapis.com/k8s-release-dev/ci/latest.txt \

This means there is a race condition with the updown-kopsmaster periodic job, reading latest-ci.txt from either the postsubmit-push-to-staging job or the kops-aws-k8s-latest periodic job. It also means that all of our other e2e jobs can be using outdated artifacts from kops' master branch until a kops PR merge wins the race.

---

Solution

This PR updates the e2e-kops-aws-k8s-latest job to stop writing to the latest-ci.txt version marker, breaking the race condition. The This version marker is only used by the k/k presubmit job comment is not true because we see the marker is used by the pipeline job that write to latest-ci-updown-green.txt.

---

I'm also adding a storage class setting to build_jobs.py that was added directly to yaml in #33704

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman, rifelpet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• config/jobs/kubernetes/kops/OWNERS [hakman,rifelpet]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

@rifelpet: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

• key kops-periodics-distros.yaml using file config/jobs/kubernetes/kops/kops-periodics-distros.yaml
• key kops-periodics-grid.yaml using file config/jobs/kubernetes/kops/kops-periodics-grid.yaml
• key kops-periodics-misc2.yaml using file config/jobs/kubernetes/kops/kops-periodics-misc2.yaml
• key kops-periodics-versions.yaml using file config/jobs/kubernetes/kops/kops-periodics-versions.yaml
• key kops-presubmits-distros.yaml using file config/jobs/kubernetes/kops/kops-presubmits-distros.yaml

In response to this:

Context

The sequence of Kops version markers is:

1. The kops-postsubmit-push-to-staging postsubmit job writes to latest-ci.txt (make target, example job log)

Step #1 - "artifacts": gsutil -h "Cache-Control:private, max-age=0, no-transform" cp /workspace/.build/upload/latest.txt gs://k8s-staging-kops/kops/releases/markers/master/latest-ci.txt

2. The e2e-kops-pipeline-updown-kopsmaster periodic job reads from latest-ci.txt and if creating and tearing down a cluster is successful, writes the same marker to latest-ci-updown-green.txt

test-infra/config/jobs/kubernetes/kops/kops-periodics-pipeline.yaml

Lines 36 to 37 in 5bc25c3

	--kops-version-marker=https://storage.googleapis.com/k8s-staging-kops/kops/releases/markers/master/latest-ci.txt \
	--publish-version-marker=gs://k8s-staging-kops/kops/releases/markers/master/latest-ci-updown-green.txt \

3. Almost all other periodic jobs then read from latest-ci-updown-green.txt:

test-infra/config/jobs/kubernetes/kops/build_jobs.py

Lines 81 to 82 in 5bc25c3

	if kops_version is None:
	kops_deploy_url = marker_updown_green(None)

---

Problem

The e2e-kops-aws-k8s-latest periodic job has been reading from latest-ci-updown-green.txt and writing to latest-ci.txt:

test-infra/config/jobs/kubernetes/kops/kops-periodics-versions.yaml

Lines 36 to 38 in 5749033

	--kops-version-marker=https://storage.googleapis.com/k8s-staging-kops/kops/releases/markers/master/latest-ci-updown-green.txt \
	--publish-version-marker=gs://k8s-staging-kops/kops/releases/markers/master/latest-ci.txt \
	--kubernetes-version=https://storage.googleapis.com/k8s-release-dev/ci/latest.txt \

This means there is a race condition with the updown-kopsmaster periodic job, reading latest-ci.txt from either the postsubmit-push-to-staging job or the kops-aws-k8s-latest periodic job. It also means that all of our other e2e jobs can be using outdated artifacts from kops' master branch until a kops PR merge wins the race.

---

Solution

This PR updates the e2e-kops-aws-k8s-latest job to stop writing to the latest-ci.txt version marker, breaking the race condition. The This version marker is only used by the k/k presubmit job comment is not true because we see the marker is used by the pipeline job that write to latest-ci-updown-green.txt.

---

I'm also adding a storage class setting to build_jobs.py that was added directly to yaml in #33704

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.