Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase timeout for namespace e2e test polling #348

Merged
merged 3 commits into from
Jul 17, 2024
Merged

Increase timeout for namespace e2e test polling #348

merged 3 commits into from
Jul 17, 2024

Conversation

SgtCoDFish
Copy link
Member

@SgtCoDFish SgtCoDFish commented Jul 17, 2024 •
edited
Loading

The CA Root Controller all namespaces should have valid configs in test regularly flakes, causing CI to fail.

As a first step, try to reduce flakes by increasing this timeout, on the assumption that the flakes are caused by a heavy load in CI or some other thing which causes them to time out.

Also:

  • Changes to use variables when calling the Poll function, which makes the arguments clearer to reason about.
  • Adds no-color to ginkgo when running in CI to make output more readable

@SgtCoDFish
use variables for polling to make arguments clearer
9ac135a 
Signed-off-by: Ashley Davis <ashley.davis@venafi.com>
@SgtCoDFish
increase timeout for namespace e2e test polling
6466444 
this test regularly flakes causing CI to fail. As a first step,
try to reduce flakes by increasing this timeout on the assumption
that the flakes are caused by a heavy load in CI

Signed-off-by: Ashley Davis <ashley.davis@venafi.com>
@cert-manager-prow cert-manager-prow bot added dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 17, 2024
@SgtCoDFish
Copy link
Member Author

SgtCoDFish commented Jul 17, 2024 •
edited
Loading

Interesting result from the test runs on this PR regarding the usually-flaky CA Root Controller all namespaces should have valid configs in test.

This job for v1.18 had that test take 44s and would have flaked before this PR.

All other instances of the flaky test passed in a few seconds!

@cert-manager-prow cert-manager-prow bot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jul 17, 2024
@SgtCoDFish
Copy link
Member Author

SgtCoDFish commented Jul 17, 2024 •
edited
Loading

On the second run when I pushed more commits, the ecc test flaked because I added a commit to change the default istio version. I've removed that now for a separate PR.

As for e2e tests:

  • v1.20 would have flaked before this change. It took 44s.
  • v1.21 would have flaked too. Also took 44s.

Other e2e tests all had the flaky test pass in a couple of seconds.

The second run included the change to set --no-color in Ginkgo for e2e tests on prow, and that worked as expected.

@SgtCoDFish
don't output color in ginkgo tests in prow
fa16cb8 
this makes raw output much easier to read

Signed-off-by: Ashley Davis <ashley.davis@venafi.com>
@SgtCoDFish
Copy link
Member Author

On the third run (after I removed the commit which broke the ecc test) I saw the same story:

  • v1.18 took 44s and would have flaked before this PR
  • v1.19 would've flaked too, also 44s
  • All other versions took seconds.

inteon
inteon reviewed Jul 17, 2024
View reviewed changes
Copy link
Member

@inteon inteon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully this fixes the flake. However, let's stay alert for any weirdness we find in the testing logic that might explain these kind of flakes.
/approve
/lgtm

@cert-manager-prow cert-manager-prow bot added the lgtm Indicates that a PR is ready to be merged. label Jul 17, 2024
@cert-manager-prow
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: inteon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cert-manager-prow cert-manager-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 17, 2024
@cert-manager-prow cert-manager-prow bot merged commit ba51b04 into cert-manager:main Jul 17, 2024
12 checks passed
@SgtCoDFish SgtCoDFish deleted the citweaks branch July 17, 2024 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@inteon inteon inteon left review comments

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@SgtCoDFish @inteon