-
Notifications
You must be signed in to change notification settings - Fork 5.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Test Snap Cronjob can trigger a cronjob to open a di...
flaky tests
#28363
Conversation
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
Builds ready [5f92210]
Page Load Metrics (2244 ± 108 ms)
Bundle size diffs
|
@@ -74,6 +75,8 @@ describe('Test Snap Cronjob', function () { | |||
text: 'Reconnect to Cronjobs Snap', | |||
}); | |||
|
|||
await driver.delay(largeDelayMs); | |||
|
|||
// switch to dialog popup, wait for a maximum of 65 seconds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should remove the comment wait for a maximum of 65 seconds
It used to do that in a previous version, but it no longer does, and that's actually the problem here. It used to have
windowHandles = await driver.waitUntilXWindowHandles(3, 1000, 65000);
I researched it, and there's no easy way to increase the timeout in my new switchToWindowWithTitle
(oops)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, I will remove the comment. Thanks for checking @HowardBraham.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Builds ready [7a33bab]
Page Load Metrics (2731 ± 682 ms)
Bundle size diffs
|
Description
This flaky test is due to the dialog taking time to appear and disappear, which is the expected behavior. I have attached a video demonstrating the current behavior.
Cronjob-snap.mov
To address this issue, I added retry logic and validation. However, I noticed that when the dialog does not appear,
switchToWindowWithTitle
method throws an exception and fails the test hence causing the retry logic to be ineffective. Implementation is in the first commit in this PR b97dbbe.Fixing this issue within the framework would require significant effort and could impact other tests. Therefore, I have implemented a wait period before attempting to switch to the window. This approach ensures that the dialog has enough time to appear, thereby reducing the likelihood of test failures.
Related issues
Fixes:
#28154
Manual testing steps
Run the test locally or in codespaces using below command
yarn
yarn build:test:webpack
ENABLE_MV3=false yarn test:e2e:single test/e2e/snaps/test-snap-cronjob.spec.js --browser=chrome --leave-running
Pre-merge author checklist
Pre-merge reviewer checklist