Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draggable block e2e tests: re-enable tests and make them pass #43737

Closed
ciampo opened this issue Aug 31, 2022 · 2 comments
Closed

Draggable block e2e tests: re-enable tests and make them pass #43737

ciampo opened this issue Aug 31, 2022 · 2 comments
Labels
[Package] E2E Tests /packages/e2e-tests [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Regression Related to a regression in the latest release

Comments

@ciampo
Copy link
Contributor

ciampo commented Aug 31, 2022

Draggable block e2e tests are currently being skipped (#43736) because they were failing ofter, blocking other PRs from being merged.

A previous tentative fix was also merged, but it didn't seem to affect the tests.


From my research, these tests started erroring when #43617 got merged, which is not totally unrelated to these tests — so they may be some truth there.

What makes debugging these failures particularly hard is that, so far, I haven't been able to reproduce those failures on my local machine — the tests always pass without issues.

Error details
● Draggable block › can drag and drop to the top of a block list

    expect(received).toMatchSnapshot()

    Snapshot name: `Draggable block can drag and drop to the top of a block list 1`

    - Snapshot  - 7
    + Received  + 0

    - <!-- wp:paragraph -->
    - <p>1</p>
    - <!-- /wp:paragraph -->
    -
    - <!-- wp:paragraph -->
    - <p>2</p>
    - <!-- /wp:paragraph -->

      46 |
      47 | 		// Confirm correct setup.
    > 48 | 		expect( await getEditedPostContent() ).toMatchSnapshot();
         | 		                                       ^
      49 |
      50 | 		await showBlockToolbar();
      51 | 		const dragHandle = await page.waitForSelector(

      at Object.<anonymous> (specs/editor/various/draggable-block.test.js:48:42)
          at runMicrotasks (<anonymous>)

  ● Draggable block › can drag and drop to the top of a block list

    TimeoutError: waiting for selector `tr[data-slug="gutenberg-test-plugin-disables-the-css-animations"] .deactivate a` failed: timeout 30000ms exceeded

      at new WaitTask (../../node_modules/puppeteer-core/src/common/DOMWorld.ts:813:28)
      at DOMWorld.waitForSelectorInPage (../../node_modules/puppeteer-core/src/common/DOMWorld.ts:656:22)
      at Object.internalHandler.waitFor (../../node_modules/puppeteer-core/src/common/QueryHandler.ts:78:19)
      at DOMWorld.waitForSelector (../../node_modules/puppeteer-core/src/common/DOMWorld.ts:511:25)
      at Frame.waitForSelector (../../node_modules/puppeteer-core/src/common/FrameManager.ts:1273:47)
      at Page.page [as waitForSelector] (../../node_modules/puppeteer-core/src/common/Page.ts:3[21](https://github.com/WordPress/gutenberg/runs/8110781726?check_suite_focus=true#step:6:22)0:29)
      at activatePlugin (../e2e-test-utils/build/@wordpress/e2e-test-utils/src/activate-plugin.js:24:8)
          at runMicrotasks (<anonymous>)
      at Object.<anonymous> (specs/editor/various/draggable-block.test.js:36:3)

  ● Draggable block › can drag and drop to the bottom of a block list

    expect(received).toMatchSnapshot()

    Snapshot name: `Draggable block can drag and drop to the bottom of a block list 1`

    - Snapshot  - 7
    + Received  + 0

    - <!-- wp:paragraph -->
    - <p>1</p>
    - <!-- /wp:paragraph -->
    -
    - <!-- wp:paragraph -->
    - <p>2</p>
    - <!-- /wp:paragraph -->

      89 |
      90 | 		// Confirm correct setup.
    > 91 | 		expect( await getEditedPostContent() ).toMatchSnapshot();
         | 		                                       ^
      92 |
      93 | 		const [ , secondParagraph ] = await page.$$(
      94 | 			'[data-type="core/paragraph"]'

      at Object.<anonymous> (specs/editor/various/draggable-block.test.js:91:42)
          at runMicrotasks (<anonymous>)

  ● Draggable block › can drag and drop to the bottom of a block list

    TimeoutError: waiting for selector `tr[data-slug="gutenberg-test-plugin-disables-the-css-animations"] .deactivate a` failed: timeout 30000ms exceeded

      at new WaitTask (../../node_modules/puppeteer-core/src/common/DOMWorld.ts:813:28)
      at DOMWorld.waitForSelectorInPage (../../node_modules/puppeteer-core/src/common/DOMWorld.ts:656:[22](https://github.com/WordPress/gutenberg/runs/8110781726?check_suite_focus=true#step:6:23))
      at Object.internalHandler.waitFor (../../node_modules/puppeteer-core/src/common/QueryHandler.ts:78:19)
      at DOMWorld.waitForSelector (../../node_modules/puppeteer-core/src/common/DOMWorld.ts:511:25)
      at Frame.waitForSelector (../../node_modules/puppeteer-core/src/common/FrameManager.ts:1273:47)
      at Page.page [as waitForSelector] (../../node_modules/puppeteer-core/src/common/Page.ts:3210:29)
      at activatePlugin (../e2e-test-utils/build/@wordpress/e2e-test-utils/src/activate-plugin.js:[24](https://github.com/WordPress/gutenberg/runs/8110781726?check_suite_focus=true#step:6:25):8)
          at runMicrotasks (<anonymous>)
      at Object.<anonymous> (specs/editor/various/draggable-block.test.js:[36](https://github.com/WordPress/gutenberg/runs/8110781726?check_suite_focus=true#step:6:37):3)

 › 1 snapshot failed.

Interestingly, from the error message it looks like the received snapshots are completely empty, meaning that:

  • either there is something preventing blocks from being created as a result of the keyboard events; and/or
  • there's something specific to our e2e testing CI environment that causes this failure (e.g concurrency, a previous test "leaking" some changes into this test...)
@ciampo ciampo added [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Regression Related to a regression in the latest release [Package] E2E Tests /packages/e2e-tests labels Aug 31, 2022
@ciampo
Copy link
Contributor Author

ciampo commented Aug 31, 2022

As @WunderBart pointed out, from the test artifacts it looks like the "new" post that is being created during the test, seems to have a local backup

can drag and drop to the top of a block list 2022-08-31T06-08-41

One hypothesis is that the appearance of this banner may move the focus away from the editor, causing the test to fail

@talldan
Copy link
Contributor

talldan commented Sep 7, 2022

Fixed by #43798

@talldan talldan closed this as completed Sep 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Package] E2E Tests /packages/e2e-tests [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Regression Related to a regression in the latest release
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants