chore: comment out skipped tests where tests always hangs + add logs #28632
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
14 flaky tests on run #53465 ↗︎
Details:
|
|||||||||||||||||||||||||||||||||||||||||||||
| Test | Artifacts | |
|---|---|---|
| ... > can timeout when retrieving upstream response |
Test Replay
|
|
commands/querying/querying.cy.js • 1 flaky test • 5x-driver-electron
| Test | Artifacts | |
|---|---|---|
| ... > intercept aliases > returns null if no xhr is found |
Test Replay
|
|
e2e/origin/origin.cy.ts • 1 flaky test • 5x-driver-electron
| Test | Artifacts | |
|---|---|---|
| cy.origin > withBeforeEach > passes runnable state to the secondary origin on retry |
Test Replay
|
|
cypress/cypress.cy.js • 3 flaky tests • 5x-driver-electron
| Test | Artifacts | |
|---|---|---|
| ... > correctly returns currentRetry |
Test Replay
|
|
| ... > correctly returns currentRetry |
Test Replay
|
|
| ... > correctly returns currentRetry |
Test Replay
|
|
project-setup.cy.ts • 2 flaky tests • launchpad-e2e
The first 5 flaky specs are shown, see all 7 specs in Cypress Cloud.
Review all test suite changes for PR #28632 ↗︎
chrisbreiding
approved these changes
Jan 11, 2024
AtofStryker
approved these changes
Jan 12, 2024
| @@ -369,7 +369,9 @@ function visitLaunchpad () { | |||
| return logInternal(`visitLaunchpad ${Cypress.env('e2e_launchpadPort')}`, () => { | |||
| return cy.visit(`/__launchpad/index.html`, { log: false }).then((val) => { | |||
| return cy.get('[data-e2e]', { timeout: 10000, log: false }).then(() => { | |||
| return val | |||
| return cy.get('.spinner', { timeout: 10000, log: false }).should('not.exist').then(() => { | |||
There was a problem hiding this comment.
do we think these timeout increases are performance related to the app? Wonder if its worth creating an issue but I also think this might be pretty difficult to track down the why.
There was a problem hiding this comment.
I have no idea what's causing this to hang and whether it's an App issue or just an issue in our own tests and how we're setting things up. I've never seen this behavior in the App outside of these tests or gotten a report of an issue of this page hanging.
|
Released in This comment thread has been locked. If you are still experiencing this issue after upgrading to |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Additional details
This PR attempts to address some of the flake in our repo by doing the following:
visitLaunchpadutil to wait for thespinnerto no longer exist in the DOM. This is generally the accurate failure for these flaky frontend tests and their failures, but the errors get muddied on the cause because you just end up seeing errors like 'Welcome Cypress' wasn't visible or 'x wasn't visible', but if you review the Test Replay - the problem is that the page is just displaying the loading page. In the future, it will be more obvious that the test was stuck in a loading state.Steps to test
How has the user experience changed?
PR Tasks
cypress-documentation?type definitions?