Fix the flaky site editor list view tests #51598
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mamaduka
added
[Type] Bug
|
Flaky tests detected in 52519f8. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/5296361738
|
alexstine
approved these changes
Jun 16, 2023
There was a problem hiding this comment.
@Mamaduka Very sorry about that, I introduced the new list view tests. What a weird bug for sure. Had no idea we already had list view tests. Sometimes I am not a fan of our naming conventions.
If you can get the checks passing, LGTM. 👍
Mamaduka
force-pushed
the
update/block-list-panel-preferences-add-cleanup
branch
from
24a2fce
to
52519f8
Compare
|
No worries, @alexstine! Similar bugs are hard to catch locally, but at least we caught them early via CI. @ramonjd, what do you think about consolidating site editor list view tests in a single file? Happy to follow up in a separate PR. |
|
Size Change: 0 B Total Size: 1.4 MB ℹ️ View Unchanged
|
Great idea! That makes much more sense – I guess I expected the preferences pane to have more E2E tests 😄 Also thanks for offering to pick it up. |
sethrubenstein
pushed a commit
to pewresearch/gutenberg
that referenced
this pull request
Jul 13, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Fixes #51533.
The issue reported that the following assertion
await expect( openNavigationButton ).toBeFocused();was failing on line 86 in the new site editor list view e2e tests -test/e2e/specs/site-editor/list-view.spec.js.Why?
The editor preferences persist across the tests, and the "Always open list view" setting was "leaking" from the block list panel preferences tests.
How?
I added the cleanup code to the block list panel preferences test to turn off persisted list view after the tests are finished.
Testing Instructions
It's hard to reproduce the exact scenario when the test fails on CI, but you can add this code snippet at the beginning of the "ensures List View global shortcut works properly" test to confirm my theory.
Then run the tests.