NE-585: Make ROUTER_BACKEND_CHECK_INTERVAL configurable #712

candita · 2022-03-02T19:25:40Z

NE-585 Expose and make configurable ROUTER_BACKEND_CHECK_INTERVAL to customize the length of time between subsequent liveness checks on backends. Details in the enhancement proposal

Refactor some unit tests and add e2e test.

candita · 2022-03-02T21:55:58Z

failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: failed to apply Terraform: exit status 1

/retest

candita · 2022-03-04T19:26:19Z

/retest

candita · 2022-03-07T21:19:28Z

/retest

candita · 2022-03-08T18:16:44Z

/test e2e-aws

candita · 2022-03-09T22:26:01Z

/retest

candita · 2022-03-10T15:44:07Z

The e2e test for componentRoutes could use tweaking if this comes up frequently:
=== RUN TestConfigurableRouteNoConsumingUserNoRBAC
configurable_route_test.go:301: failed to restore cluster ingress resource to original state: Operation cannot be fulfilled on ingresses.config.openshift.io "cluster": the object has been modified; please apply your changes to the latest version and try again
configurable_route_test.go:305: failed to restore cluster ingress resource to original state: Operation cannot be fulfilled on ingresses.config.openshift.io "cluster": the object has been modified; please apply your changes to the latest version and try again
--- FAIL: TestConfigurableRouteNoConsumingUserNoRBAC (1.28s)

/retest

candita · 2022-03-16T22:58:54Z

/retest

candita · 2022-03-17T14:00:34Z

Interesting failure:

Mar 17 03:18:32.343: FAIL: ns/openshift-etcd pod etcd-quorum-guard-6cddbc8997-mrxld and pod etcd-quorum-guard-6cddbc8997-8mkx4 are running on the same node: ci-op-5zwyg8np-6da5c-9tmhw-master-2

/retest

candita · 2022-03-21T13:40:36Z

/retest

candita · 2022-03-21T19:46:41Z

/retest

pkg/operator/controller/ingress/deployment_test.go

candita · 2022-03-30T18:36:57Z

could not run steps: step e2e-aws failed: failed to acquire lease for "aws-quota-slice": resources not found
/retest

candita · 2022-04-19T03:18:40Z

/retest

candita · 2022-04-19T14:36:54Z

=== RUN TestHostNetworkEndpointPublishingStrategy
operator_test.go:756: failed to observe expected conditions: timed out waiting for the condition
operator_test.go:758: deleted ingresscontroller host
--- FAIL: TestHostNetworkEndpointPublishingStrategy (300.04s)
/retest

candita · 2022-04-19T18:10:50Z

evel=error msg=Error: Error waiting for instance (i-0c49e1870a72f8c20) to become ready: timeout while waiting for state to become 'running' (last state: 'pending', timeout: 10m0s)
level=error
level=error msg= with module.masters.aws_instance.master[2],
level=error msg= on master/main.tf line 129, in resource "aws_instance" "master":
level=error msg= 129: resource "aws_instance" "master" {

/retest

candita · 2022-04-19T20:30:49Z

Again:

=== RUN TestHostNetworkEndpointPublishingStrategy
operator_test.go:756: failed to observe expected conditions: timed out waiting for the condition
operator_test.go:758: deleted ingresscontroller host
--- FAIL: TestHostNetworkEndpointPublishingStrategy (300.44s)

/retest

Miciah · 2022-04-19T22:04:18Z

Again:

=== RUN TestHostNetworkEndpointPublishingStrategy
operator_test.go:756: failed to observe expected conditions: timed out waiting for the condition
operator_test.go:758: deleted ingresscontroller host
--- FAIL: TestHostNetworkEndpointPublishingStrategy (300.44s)

/retest

Try bumping openshift/api to openshift/api@716dc56 or newer.

Miciah · 2022-04-19T22:07:37Z

test/e2e/checkinterval_test.go

+	// when set to an unacceptable value, it should not be accepted
+	if err := waitForDeploymentEnvVar(t, kclient, routerDeployment, deploymentTimeout, ingresscontroller.RouterBackendCheckInterval, (0 * time.Second).String()); err == nil {
+		t.Fatalf("unexpected router deployment %s set at %v: %v", ingresscontroller.RouterBackendCheckInterval, 2*defaultHealthCheckInterval, err)
+	}
+
+	// when set to an unacceptable value, the env variable should not exist
+	if err := waitForDeploymentEnvVar(t, kclient, routerDeployment, deploymentTimeout, ingresscontroller.RouterBackendCheckInterval, ""); err != nil {
+		t.Fatalf("expected router deployment to remove %s: %v", ingresscontroller.RouterBackendCheckInterval, err)
+	}


Are both waitForDeploymentEnvVar calls necessary? It seems like the second one (make sure the environment variable is absent) would suffice, and the first one (make sure the environment variable isn't set to an unexpected value) is superfluous.

candita · 2022-04-21T13:55:10Z

=== RUN TestConfigurableRouteNoConsumingUserNoRBAC
configurable_route_test.go:301: failed to restore cluster ingress resource to original state: Operation cannot be fulfilled on ingresses.config.openshift.io "cluster": the object has been modified; please apply your changes to the latest version and try again
configurable_route_test.go:305: failed to restore cluster ingress resource to original state: Operation cannot be fulfilled on ingresses.config.openshift.io "cluster": the object has been modified; please apply your changes to the latest version and try again
--- FAIL: TestConfigurableRouteNoConsumingUserNoRBAC (0.97s)

/test e2e-aws-operator

Miciah · 2022-04-21T21:20:39Z

/lgtm

openshift-ci · 2022-04-21T21:21:07Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: candita, Miciah

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Miciah,candita]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ShudiLi · 2022-04-22T03:49:07Z

/label qe-approved

openshift-bot · 2022-04-22T08:14:15Z

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2022-04-22T10:38:16Z

/retest-required

Please review the full test history for this PR and help us cut down flakes.

openshift-bot · 2022-04-22T12:17:14Z

/retest-required

Please review the full test history for this PR and help us cut down flakes.

Miciah · 2022-04-22T14:29:52Z

 === RUN   TestRouterCompressionParsing
    router_compression_test.go:101: failed to update ingress controller, retrying...
    router_compression_test.go:40: deleted ingresscontroller http-compression-1
    router_compression_test.go:101: failed to update ingress controller, retrying...
    router_compression_test.go:41: deleted ingresscontroller http-compression-2
    router_compression_test.go:101: failed to update ingress controller, retrying...
    router_compression_test.go:42: error testing compression policy that needs quotes: failed to update ingress controller: Put "https://api.ci-op-p2cm6vv3-43abb.origin-ci-int-aws.dev.rhcloud.com:6443/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/http-compression-3": read tcp 10.131.191.98:56832->54.241.76.165:6443: read: connection reset by peer
    router_compression_test.go:42: deleted ingresscontroller http-compression-3
    router_compression_test.go:101: failed to update ingress controller, retrying...
    router_compression_test.go:53: deleted ingresscontroller http-compression-4
--- FAIL: TestRouterCompressionParsing (94.68s)

We might need to add a retry around the update in that test.
/test e2e-aws-operator

openshift-ci · 2022-04-22T16:00:52Z

@candita: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 2, 2022

candita force-pushed the NE-585-HealthCheckInterval branch from eeb2ed5 to 7111096 Compare March 2, 2022 19:28

openshift-ci bot requested review from lmzuccarelli and miheer March 2, 2022 19:28

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 2, 2022

candita force-pushed the NE-585-HealthCheckInterval branch 2 times, most recently from da7698d to 8928044 Compare March 4, 2022 16:56

candita mentioned this pull request Mar 8, 2022

NE-585 Update enhancement after implementation openshift/enhancements#1054

Merged

candita force-pushed the NE-585-HealthCheckInterval branch from 8928044 to 40e3112 Compare March 16, 2022 21:10

openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 16, 2022

candita force-pushed the NE-585-HealthCheckInterval branch from dd2bb4c to df7d2b8 Compare March 17, 2022 02:39

openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 17, 2022

candita force-pushed the NE-585-HealthCheckInterval branch from df7d2b8 to ea5f804 Compare March 18, 2022 16:41

gcs278 reviewed Mar 22, 2022

View reviewed changes

pkg/operator/controller/ingress/deployment_test.go Outdated Show resolved Hide resolved

pkg/operator/controller/ingress/deployment_test.go Show resolved Hide resolved

pkg/operator/controller/ingress/deployment_test.go Show resolved Hide resolved

candita force-pushed the NE-585-HealthCheckInterval branch 2 times, most recently from c306d56 to b35f5c4 Compare March 25, 2022 23:14

candita changed the title ~~[WIP] NE-585 Make ROUTER_BACKEND_CHECK_INTERVAL configurable~~ NE-585 Make ROUTER_BACKEND_CHECK_INTERVAL configurable Mar 25, 2022

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 25, 2022

openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 2, 2022

candita force-pushed the NE-585-HealthCheckInterval branch from 8267ef0 to 1d6bfc2 Compare April 19, 2022 00:01

Miciah reviewed Apr 19, 2022

View reviewed changes

candita force-pushed the NE-585-HealthCheckInterval branch 2 times, most recently from c2d53f2 to 6c51354 Compare April 20, 2022 21:58

candita force-pushed the NE-585-HealthCheckInterval branch 3 times, most recently from bf72dde to 24de4a9 Compare April 21, 2022 21:10

NE-585: Make ROUTER_BACKEND_CHECK_INTERVAL configurable

2849fcf

candita force-pushed the NE-585-HealthCheckInterval branch from 24de4a9 to 2849fcf Compare April 21, 2022 21:18

openshift-ci bot assigned Miciah Apr 21, 2022

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 21, 2022

Miciah mentioned this pull request Apr 21, 2022

NE-882: Set ROUTER_DOMAIN to enable route subdomain field #674

Merged

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Apr 22, 2022

openshift-merge-robot merged commit b04c25e into openshift:master Apr 22, 2022

Miciah mentioned this pull request Apr 22, 2022

Bug 2076193: Remove special-casing for default ingresscontroller probe parameters #742

Merged

ahardin-rh mentioned this pull request May 25, 2022

OSDOCS-3419: healthCheckInterval Parameter documentation openshift/openshift-docs#44848

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NE-585: Make ROUTER_BACKEND_CHECK_INTERVAL configurable #712

NE-585: Make ROUTER_BACKEND_CHECK_INTERVAL configurable #712

candita commented Mar 2, 2022 •

edited

Loading

candita commented Mar 2, 2022

candita commented Mar 4, 2022

candita commented Mar 7, 2022

candita commented Mar 8, 2022

candita commented Mar 9, 2022

candita commented Mar 10, 2022

candita commented Mar 16, 2022

candita commented Mar 17, 2022

candita commented Mar 21, 2022

candita commented Mar 21, 2022

candita commented Mar 30, 2022

candita commented Apr 19, 2022

candita commented Apr 19, 2022

candita commented Apr 19, 2022

candita commented Apr 19, 2022

Miciah commented Apr 19, 2022

Miciah Apr 19, 2022

candita Apr 20, 2022

candita commented Apr 21, 2022

Miciah commented Apr 21, 2022

openshift-ci bot commented Apr 21, 2022

ShudiLi commented Apr 22, 2022

openshift-bot commented Apr 22, 2022

openshift-bot commented Apr 22, 2022

openshift-bot commented Apr 22, 2022

Miciah commented Apr 22, 2022

openshift-ci bot commented Apr 22, 2022

NE-585: Make ROUTER_BACKEND_CHECK_INTERVAL configurable #712

NE-585: Make ROUTER_BACKEND_CHECK_INTERVAL configurable #712

Conversation

candita commented Mar 2, 2022 • edited Loading

candita commented Mar 2, 2022

candita commented Mar 4, 2022

candita commented Mar 7, 2022

candita commented Mar 8, 2022

candita commented Mar 9, 2022

candita commented Mar 10, 2022

candita commented Mar 16, 2022

candita commented Mar 17, 2022

candita commented Mar 21, 2022

candita commented Mar 21, 2022

candita commented Mar 30, 2022

candita commented Apr 19, 2022

candita commented Apr 19, 2022

candita commented Apr 19, 2022

candita commented Apr 19, 2022

Miciah commented Apr 19, 2022

Miciah Apr 19, 2022

Choose a reason for hiding this comment

candita Apr 20, 2022

Choose a reason for hiding this comment

candita commented Apr 21, 2022

Miciah commented Apr 21, 2022

openshift-ci bot commented Apr 21, 2022

ShudiLi commented Apr 22, 2022

openshift-bot commented Apr 22, 2022

openshift-bot commented Apr 22, 2022

openshift-bot commented Apr 22, 2022

Miciah commented Apr 22, 2022

openshift-ci bot commented Apr 22, 2022

candita commented Mar 2, 2022 •

edited

Loading