Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve reliability of playwright tests #1018

Closed
willdavsmith opened this issue Mar 19, 2024 · 3 comments
Closed

Improve reliability of playwright tests #1018

willdavsmith opened this issue Mar 19, 2024 · 3 comments
Assignees
Labels
important This item is a high priority Issue we intend to address as soon as possible triaged This item has been triaged by project maintainers and is in the backlog

Comments

@willdavsmith
Copy link
Contributor

willdavsmith commented Mar 19, 2024

We have seen some issues with the playwright tests with network issues.

#885
#883
#907

We should consider improving the reliability of these tests through retries, better logging, etc.

AB#11555

@youngbupark
Copy link
Contributor

The major failure happens only when eshop is being deployed to k3d cluster. testing it on EKS cluster is always stable. My hypothesis is that it might be related to resource constraints of single vm cluster. I would check whether the root cause of this problem stems from resource limitation or not by switching k3d cluster to aks cluster and running it for at least a week.

@ytimocin
Copy link
Contributor

ytimocin commented Mar 20, 2024

I will schedule a meeting with you, @youngbupark and @willdavsmith. We can discuss about these flaky tests. I am also working on a task right now which is also related to this.

@sylvainsf sylvainsf added triaged This item has been triaged by project maintainers and is in the backlog important This item is a high priority Issue we intend to address as soon as possible labels Mar 21, 2024
@ytimocin
Copy link
Contributor

ytimocin commented May 6, 2024

I have added retries, better logging, and improved the workflows. I added the video recordings of the failure and changed the readiness checks to liveness checks. I am closing this for now and, going forward, we can focus on the issues that pop up in the tests.

@ytimocin ytimocin closed this as completed May 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
important This item is a high priority Issue we intend to address as soon as possible triaged This item has been triaged by project maintainers and is in the backlog
Projects
None yet
Development

No branches or pull requests

4 participants