Canvas elements not captured and shown during Test Replay, shows striped placeholder

[mblasco]

Current behavior

Cypress tests are being executed without issues, both locally and remotely, but when we try to review the test execution in Test Replay, after some seconds, one particular canvas in the HTML rendered completely hides the test execution, preventing to see what is happening in the screen.

Inspecting the HTML during Test Replay, we found that this component:

<canvas data-cy-replay-is-placeholder="" data-v-25b03cc4="" class="guided-tour-canvas" width="1920" height="1080" title="Test Replay: unsupported content type" style="width: 1920px; height: 1080px;"></canvas>

is the one covering the whole screen.

If I manually remove the canvas from the HTML, I can see the screen of the test execution, but as soon as I click the play to resume the execution, the canvas is added back and coverts the screen again.

This started happening some days after upgrading to Cypress 13.4.0. Reverting to Cypress 13.3.0 or upgrading to Cypress 13.5.0 is not fixing the issue. I don't think it's related to Cypress version itself, but to the cloud service that takes care of replaying the test.

Desired behavior

That canvas should not hide test execution.

Test code to reproduce

https://cloud.cypress.io/projects/3pahs1/runs/14754/overview/16526418-5237-4e8b-8287-f9e8e4f9f4bf/replay?att=1&roarHideRunsWithDiffGroupsAndTags=0&ts=1699515442815.4143

Cypress Version

13.4.0

Node version

20

Operating System

Ubuntu

Debug Logs

No response

Other

No response

[jennifer-shehane]

@mblasco Capturing of canvas elements are not currently supported, but it's on our roadmap.

[mblasco]

I think the problem is that this canvas in our app is a transparent element whose size is the whole screen, but it's transparent, allowing to see all the elements in the screen. But from the Test Replay execution, the canvas is appearing as an opaque element covering the whole screen.

[izabelahotala]

@mblasco Capturing of canvas elements are not currently supported, but it's on our roadmap.

Is it possible to predict when this functionality will initially work again?

[jennifer-shehane]

@mblasco Interesting, I'll relay this feedback.

[whitespacecode]

Thank you for this ticket.
We are reverting back to v12 because of this.
Our test are failing because we need to interact with the canvas and for some reason v13 can't handle it anymore?
https://cloud.cypress.io/projects/yzkmsw/runs/410/overview/5787671b-18f8-434b-85dd-6b5af96091bb/replay?roarHideRunsWithDiffGroupsAndTags=1&ts=1705310246885.3005&att=1

[jennifer-shehane]

@whitespacecode There should be no change to the interaction of canvas elements within your test. The canvas elements are simply not represented in the Test Replay. If you record a video or take screenshots of the run with canvas - you will see that it is rendered and interacted with in Cypress, so you're likely encountering another issue.

[whitespacecode]

@whitespacecode There should be no change to the interaction of canvas elements within your test. The canvas elements are simply not represented in the Test Replay. If you record a video or take screenshots of the run with canvas - you will see that it is rendered and interacted with in Cypress, so you're likely encountering another issue.

Thank you for the reply.
Very strange that on v13 the single test where i click the canvas failed each time. Going back to v12 fixes it.

[jennifer-shehane]

@whitespacecode Could you open a new issue with a reproducible example so we can track down the issue? It could indeed be a separate v13 related issue.

[mentaman]

@jennifer-shehane Any ETA? I see this issue exists for half year. Where can we follow it?
Unfortunately we couldn't downgrade to v12, since until we noticed this issue we already became dependent on v13 ( once we downgraded to v12 we started having a lot of flakyness )

[jennifer-shehane]

@mentaman The lack of canvas being captured on Test Replay should have zero effect on how your tests run and the output of your tests. The canvas however will not be represented in the Test Replay. If you require seeing the Canvas for review after a test is run, we recommend turning video on.

This is still on our roadmap to provide.

[mentaman]

@jennifer-shehane I'm aware. Sorry I wasn't clear enough, but the issue for us is that our canvas is above other elements and most of the canvas is transparent in real life.
While in Replay it shows it this way instead of transparent

so it just hides everything behind it and we can't know what happens there. and when we try to research flakyness it makes it more challenging.

If it could be transparent at least instead of showing this white/gray pattern, this would also work for us.

[varunraorao]

I am facing the same issue @jennifer-shehane, this makes us difficult to verify the failed test cases during test replay mode.

[jennifer-shehane]

We're excited to announce that canvas elements can now be captured and displayed in Cypress Test Replay. When enabled, Cypress will take a screenshot representation of any canvas element and rehydrate those images within the existing live DOM during replay.

If you'd like canvas capture enabled for your Test Replays, let us know (comment here or email support@cypress.io) —it is currently behind a feature flag that we have to enable on our side. (We intend to have this feature flag be user facing in the future, but want to get this out as soon as possible to customers.)

To get started capturing canvas, you’ll need to be aware of a few things:

• You need to use Cypress App 13.15.0 or greater.
• Since capturing canvas elements requires additional capturing during a test run, it may affect performance, particularly if your app has many or large canvas elements or if you are on a resource constrained machine. Once enabled, we recommend monitoring for any performance issues and reaching out so we can help address them.

Note for UI Coverage and Accessibility customers: Canvas display for UI Coverage and Accessibility is not yet available, but it’s on the way!

[varunraorao]

Wow wonderful, Thank you @jennifer-shehane.

[varunraorao]

Thank you for taking this as priority and resolving the issue.

…

On Thu, Oct 3, 2024 at 7:59 PM Jennifer Shehane ***@***.***> wrote: We're excited to announce that canvas elements can now be captured and displayed in Cypress Test Replay. See a live example here. <https://cloud.cypress.io/projects/4b7344/runs/29274/test-results/e76819f0-98b6-4d1c-b0e3-fb284266f5ad/replay?actions=%5B%5D&att=1&browsers=%5B%5D&groups=%5B%5D&isFlaky=%5B%5D&modificationDateRange=%7B%22startDate%22%3A%221970-01-01%22%2C%22endDate%22%3A%222038-01-19%22%7D&orderBy=EXECUTION_ORDER&oses=%5B%5D&pc=log-http%3A%2F%2Flocalhost%3A8080-95__command-logs&specs=%5B%5D&statuses=%5B%5D&testingTypesEnum=%5B%5D&ts=1727963817544.1> When enabled, Cypress will take a screenshot representation of any canvas element and rehydrate those images within the existing live DOM during replay. *If you'd like canvas capture enabled for your Test Replays, let us know (comment here or email ***@***.*** ***@***.***>) —it is currently behind a feature flag that we have to enable on our side.* (We intend to have this feature flag be user facing in the future, but want to get this out as soon as possible to customers.) To get started capturing canvas, you’ll need to be aware of a few things: - You need to use Cypress App 13.15.0 or greater. - Since capturing canvas elements requires additional capturing during a test run, it may affect performance, particularly if your app has many or large canvas elements or if you are on a resource constrained machine. Once enabled, we recommend monitoring for any performance issues and reaching out so we can help address them. *Note for UI Coverage and Accessibility customers: Canvas display for UI Coverage and Accessibility is not yet available, but it’s on the way!* — Reply to this email directly, view it on GitHub <#28289 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AXRHHGVMQVBAVMKT6LOXT73ZZVIDRAVCNFSM6AAAAAA7EE7AU6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJRGU3TANJQGM> . You are receiving this because you commented.Message ID: ***@***.***>

[nrutman]

@jennifer-shehane any update on the ability to test contents of canvas elements? Is that coming at any point in the near future?