-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Record canvas snapshots N times per second #859
Conversation
Makes recompiling+debugging a lot faster
Replaces @rollup/plugin-typescript for rollup-plugin-typescript2 as the former is incompatible with rollup-plugin-web-worker-loader
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not familiar with webGL so there are some parts I don't understand. But I built a demo to test this PR and it does solve the problem.
@Mark-Fenng Thanks Mark! |
Hi @Juice10 Thanks for doing this amazing work! Is there a reason this hasn't been merged? |
Thanks @DeanGracey! It's currently waiting on one more rrweb core team member review and then it should be good to go. |
Amazing, thanks @Juice10. We're currently thinking of patching rr-web in our system to get this functionality in. Do you have a rough idea when this would be released? If not too long we would prefer to wait for the release rather than patching |
@@ -12,6 +12,15 @@ rrweb.record({ | |||
}); | |||
``` | |||
|
|||
或者启用每秒 15 帧的 Canvas 图像快照记录: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
😆
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm sorry if it doesn't make any sense 😅 I figured I'd use Google translate to put in something and then you, @Mark-Fenng or another community member could put in a suggestion for something that is correct.
docs/recipes/canvas.md
Outdated
```js | ||
rrweb.record({ | ||
emit(event) {}, | ||
recordCanvas: 15, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does it make sense to move the fps option to our sampling object?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh yes that totally makes sense, thanks for the suggestion @Yuyz0112
I think I have some time this Monday to move it over!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Juice10 |
@ryanZiegler done |
@Yuyz0112 Thank you very much for your quick response 👍 👍 👍 ! Hope NPM can also update this version synchronously. |
* Only record canvas when recordCanvas is true * All should be compiled first Makes recompiling+debugging a lot faster * Add support for compiling web workes Replaces @rollup/plugin-typescript for rollup-plugin-typescript2 as the former is incompatible with rollup-plugin-web-worker-loader * Update yarn.lock * Upgrade to typescript 4.5.5 * add support for replay of ImageBitmap in 2d canvas * Snapshot canvases in a web-worker on FPS basis * Fix performance of canvas recording and playback * Wait for all images to be preloaded before checking results * flatten base64 strings, as encoding isn't consistent * Cleanup * Add serializing to 2d canvases as well * Disable blob serialize test We don't have any code for it yet * Upgrade @rollup/plugin-commonjs to 21.0.2 Fixes https://linguinecode.com/post/import-export-appear-at-the-top-level * Move canvas recording options to `sampling` Based on: rrweb-io/rrweb#859 (comment)
Problem & solution summary
Sometimes recording a canvas via regular means (
recordCanvas: true
) isn't possible or is to data intensive.When regular canvas recording isn't feasible it's nice to be able to record full snapshots of the canvas on a period basis.
Configuration
Problem explained
Recording snapshots is especially useful in the event that an rrweb recording gets initialized after a canvas was added to the dom and was populated as there is no way to read the initial state of these canvas events.
Another useful case is if a canvas gets repainted every requestAnimationFrame with the same (large) sources but doesn't actually change that much.
For example if you were to call:
Every invocation of drawImage would get recorded, the
HUGE_IMAGE_SOURCE
would get base64 encoded and saved, rapidly ballooning the size of recorded events. Utilizing the current recording mechanism I was able to generate about 300mb worth of json events in just 10-20 seconds of recording.Performance
This feature uses web workers to minimize the impact of recording on the performance of the website being recorded.
If the computer experiences heavy load the web workers will slow down, the recorded frame rate will go down, but no frames will get dropped (because of the recording).
Browser support
This feature uses OffscreenCanvas and isn't supported by every browser yet. According to caniuse this OffscreenCanvas is currently supported by 77% of browsers with webgl(2) support hidden behind feature flags in Firefox and pretty far along in Safari. There is currently no polyfill for browsers that don't support OffscreenCanvas so any canvas snapshotting attempted in those browsers will fail gracefully.
Other benefits of this PR
The web worker is currently used to do the heavy operation of converting a Canvas element to a base64 encoded string.
We are currently using
canvas.toDataURL
and wrapping it in asetTimeout
in the hopes of not locking the main thread. Building on the work in this PR, we can move this to a web worker where supported.Serializes & deserializes 2d canvas arguments (fixes #851)