Skip to content
This repository has been archived by the owner on Sep 11, 2024. It is now read-only.

End to End Tests

End to End Tests #7890

Triggered via workflow run March 6, 2024 23:57
Status Success
Total duration 17m 58s
Artifacts 9

end-to-end-tests.yaml

on: workflow_run
Prepare
3s
Prepare
Matrix: tests
Report results
55s
Report results
Fit to window
Zoom out
Zoom in

Annotations

10 errors, 7 warnings, and 1 notice
Report results
Pull Request not found!
Report results
The template is not valid. matrix-org/pr-details-action/v1.3/action.yml (Line: 15, Col: 12): Error reading JToken from JsonReader. Path '', line 0, position 0.,matrix-org/pr-details-action/v1.3/action.yml (Line: 18, Col: 12): Error reading JToken from JsonReader. Path '', line 0, position 0.,matrix-org/pr-details-action/v1.3/action.yml (Line: 21, Col: 12): Error reading JToken from JsonReader. Path '', line 0, position 0.
[Legacy Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L103
1) [Legacy Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme Error: Screenshot comparison failed: 154 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/audio-player/audio-player.spec.ts/Selected-EventTile-of-audio-player--high-contrast--irc-layout-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-be-correctly-rendered---high-contrast-theme-Legacy-Crypto/Selected-EventTile-of-audio-player--high-contrast--irc-layout-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-be-correctly-rendered---high-contrast-theme-Legacy-Crypto/Selected-EventTile-of-audio-player--high-contrast--irc-layout-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 154 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 154 pixels (ratio 0.01 of all image pixels) are different. 101 | 102 | // Take a snapshot of mx_EventTile_last on IRC layout > 103 | await expect(page.locator(".mx_EventTile_last")).toMatchScreenshot( | ^ 104 | `${detail.replaceAll(" ", "-")}-irc-layout.png`, 105 | screenshotOptions, 106 | ); at takeSnapshots (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:103:58) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:165:9
[Legacy Crypto] › audio-player/audio-player.spec.ts:214:9 › Audio player › should support replying to audio file with another audio file: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L103
2) [Legacy Crypto] › audio-player/audio-player.spec.ts:214:9 › Audio player › should support replying to audio file with another audio file Error: Screenshot comparison failed: 13 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/audio-player/audio-player.spec.ts/Selected-EventTile-of-audio-player-with-a-reply-irc-layout-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-support-replying-to-audio-file-with-another-audio-file-Legacy-Crypto/Selected-EventTile-of-audio-player-with-a-reply-irc-layout-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-support-replying-to-audio-file-with-another-audio-file-Legacy-Crypto/Selected-EventTile-of-audio-player-with-a-reply-irc-layout-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 13 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 13 pixels (ratio 0.01 of all image pixels) are different. 101 | 102 | // Take a snapshot of mx_EventTile_last on IRC layout > 103 | await expect(page.locator(".mx_EventTile_last")).toMatchScreenshot( | ^ 104 | `${detail.replaceAll(" ", "-")}-irc-layout.png`, 105 | screenshotOptions, 106 | ); at takeSnapshots (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:103:58) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:236:9
[Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A: matrix-react-sdk/playwright/e2e/room/room.spec.ts#L100
3) [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A Error: Timed out 5000ms waiting for expect(locator).toBeInViewport() Locator: locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') Expected: in viewport Received: outside viewport Call log: - expect.toBeInViewport with timeout 5000ms - waiting for locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') 98 | // The timeline should display the first message 99 | // The previous position before switching to Room B should be remembered > 100 | await expect(page.locator(".mx_MTextBody .mx_EventTile_body").getByText("test0")).toBeInViewport(); | ^ 101 | }); 102 | }); 103 | at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/room/room.spec.ts:100:91
[Legacy Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly: matrix-react-sdk/playwright/e2e/sliding-sync/sliding-sync.spec.ts#L26
4) [Legacy Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly Test timeout of 30000ms exceeded while running "beforeEach" hook. 24 | let roomId: string; 25 | > 26 | test.beforeEach(async ({ slidingSyncProxy, page, user, app }) => { | ^ 27 | roomId = await app.client.createRoom({ name: "Test Room" }); 28 | }); 29 | at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/sliding-sync/sliding-sync.spec.ts:26:10
[Legacy Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly: matrix-react-sdk/playwright/element-web-test.ts#L258
4) [Legacy Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly Error: page.goto: Test timeout of 30000ms exceeded. Call log: - navigating to "http://localhost:8080/", waiting until "load" at ../element-web-test.ts:258 256 | window.localStorage.setItem("mx_labs_feature_feature_sliding_sync", "true"); 257 | }, proxyAddress); > 258 | await page.goto("/"); | ^ 259 | await page.waitForSelector(".mx_MatrixChat", { timeout: 30000 }); 260 | await use(proxyInstance); 261 | await proxy.stop(); at Object.slidingSyncProxy (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/element-web-test.ts:258:20)
[Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog: matrix-react-sdk/playwright/e2e/user-onboarding/user-onboarding-new.spec.ts#L48
5) [Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog Error: Screenshot comparison failed: 8 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/user-onboarding/user-onboarding-new.spec.ts/User-Onboarding-new-user-app-download-dialog-1-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/user-onboarding-user-onboarding-new-User-Onboarding-new-user-app-download-dialog-Legacy-Crypto/User-Onboarding-new-user-app-download-dialog-1-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/user-onboarding-user-onboarding-new-User-Onboarding-new-user-app-download-dialog-Legacy-Crypto/User-Onboarding-new-user-app-download-dialog-1-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_Dialog') - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 8 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_Dialog') - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 8 pixels (ratio 0.01 of all image pixels) are different. 46 | page.getByRole("dialog").getByRole("heading", { level: 1, name: "Download Element" }), 47 | ).toBeVisible(); > 48 | await expect(page.locator(".mx_Dialog")).toMatchScreenshot(); | ^ 49 | }); 50 | 51 | test("using find friends action should increase progress", async ({ page, homeserver }) => { at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/user-onboarding/user-onboarding-new.spec.ts:48:50
[Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L81
6) [Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font Error: expect(received).toBeGreaterThan(expected) Expected: > 0 Received: 0 79 | 80 | // Check the status of the seek bar > 81 | expect(await page.locator(".mx_AudioPlayer_seek input[type='range']").count()).toBeGreaterThan(0); | ^ 82 | 83 | // Enable IRC layout 84 | await app.settings.setValue("layout", null, SettingLevel.DEVICE, Layout.IRC); at takeSnapshots (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:81:88) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:149:9
[Rust Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly: matrix-react-sdk/playwright/e2e/sliding-sync/sliding-sync.spec.ts#L1
7) [Rust Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly Test finished within timeout of 30000ms, but tearing down "slidingSyncProxy" ran out of time. Please allow more time for the test, since teardown is attributed towards the test timeout budget.
Prepare
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Report results
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e, bobheadxi/deployments@88ce5600046c82542f8246ac287d0a53c461bca3, actions/github-script@v6, nwtgck/actions-netlify@7a92f00dde8c92a5a9e8385ec2919775f7647352. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Slow Test: [Rust Crypto] › read-receipts/redactions.spec.ts#L1
[Rust Crypto] › read-receipts/redactions.spec.ts took 5.9m
Slow Test: [Legacy Crypto] › read-receipts/redactions.spec.ts#L1
[Legacy Crypto] › read-receipts/redactions.spec.ts took 5.3m
Slow Test: [Rust Crypto] › read-receipts/new-messages.spec.ts#L1
[Rust Crypto] › read-receipts/new-messages.spec.ts took 3.9m
Slow Test: [Rust Crypto] › read-receipts/high-level.spec.ts#L1
[Rust Crypto] › read-receipts/high-level.spec.ts took 3.7m
Slow Test: [Legacy Crypto] › read-receipts/new-messages.spec.ts#L1
[Legacy Crypto] › read-receipts/new-messages.spec.ts took 3.5m
🎭 Playwright Run Summary
7 flaky [Legacy Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme [Legacy Crypto] › audio-player/audio-player.spec.ts:214:9 › Audio player › should support replying to audio file with another audio file [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A [Legacy Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly [Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog [Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font [Rust Crypto] › sliding-sync/sliding-sync.spec.ts:186:9 › Sliding Sync › should update user settings promptly 121 skipped 654 passed (16.0m)

Artifacts

Produced during runtime
Name Size
all-blob-reports-1 Expired
9.03 MB
all-blob-reports-2 Expired
619 KB
all-blob-reports-3 Expired
5.92 MB
all-blob-reports-4 Expired
9.7 MB
all-blob-reports-5 Expired
1.19 MB
all-blob-reports-6 Expired
589 KB
all-blob-reports-7 Expired
950 KB
all-blob-reports-8 Expired
4.12 MB
html-report--attempt-1 Expired
27.1 MB