Expand AWS e2e cluster across 3 availability zones

[davidvossel]

The ocs-operator uses failure domains to help spread out placement of ceph mon and osd daemons across multiple availability zones.

In order to test in an environment as close to production as possible, ideally we'd like to our e2e functional test environment to supply us with 3 availability zones. At the moment the standard aws e2e cluster is only using us-east-1a and us-east-1b. Our request is to expand that list to include us-east-1c as well.

If we can't make this change directly to the standard aws e2e cluster setup template, is this something that we can parameterize?

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: davidvossel
To complete the pull request process, please assign wking
You can assign the PR to them by writing /assign @wking in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• ci-operator/templates/openshift/installer/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[davidvossel]

any thoughts here?

[stevekuznetsov]

/assign @wking @abhinavdahiya

[wking]

#3615 explains why we dropped us-east-1c. I'm still concerned about the throttling impact of restoring it. Maybe after we shard over AWS regions or otherwise grow some AWS-throttling headroom?

[davidvossel]

#3615 explains why we dropped us-east-1c. I'm still concerned about the throttling impact of restoring it. Maybe after we shard over AWS regions or otherwise grow some AWS-throttling headroom?

I see. If nothing has changed, then the throttling situation will likely be worse now that there are more e2e tests being run.

What's holding us back from being able to randomly assign these tests to different AWS regions?

[wking]

What's holding us back from being able to randomly assign these tests to different AWS regions?

Dev Productivity bumping resource limits (instance counts, etc.) for additional regions. Hopefully happens in the next few weeks.

[openshift-ci-robot]

@davidvossel: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@davidvossel: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@davidvossel: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/step-registry-shellcheck	e899f4a	link	/test step-registry-shellcheck
ci/build-farm/build01-dry	e899f4a	link	/test build01-dry
ci/prow/generated-cluster	e899f4a	link	/test generated-cluster
ci/prow/ci-operator-config	e899f4a	link	/test ci-operator-config
ci/prow/ci-operator-config-metadata	e899f4a	link	/test ci-operator-config-metadata

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.