Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compact style mutation #464

Merged
merged 3 commits into from
Jul 11, 2021
Merged

Conversation

eoghanmurray
Copy link
Contributor

@eoghanmurray eoghanmurray commented Jan 18, 2021

I had one recording in the wild that turned into a monster at 12GB.
One of the sources was continuous DOM Mutations caused by a 3rd party 'snow animation' library that was animating individual snowflakes multiple times per second via JavaScript, causing a constant stream of very large mutations.

Each mutation was of the format
"style":"color: rgb(255, 255, 255); position: absolute; width: 8px; height: 8px; font-family: arial, verdana; overflow: hidden; font-weight: normal; z-index: 0; display: block; bottom: auto; opacity: 1; padding: 0px; margin: 0px; font-size: 10px; line-height: 10px; text-align: center; vertical-align: baseline; left: 242.807px; top: 85.7332px;"

Even though just the left or top style attributes had changed.
The new format for this part of the attribute mutation looks like:
"style": {"left": "242.807px", "top": "85.7332px"}

There's a threshold to the attribute length before this change applies, currently set at a style attribute length of 25 characters (this value could be tweaked) (the threshold would have caused errors, so I've removed it)

Diffs are generated by creating a dummy node to house the old style and then comparing each style (sub)attribute in turn using getPropertyValue and getPropertyPriority. Attributes on the old value are also iterated over to see if any of them have been removed.

At playback time, these sub-style mutations are applied via setProperty with either a single value argument, or 2 arguments in the case that the !important priority flag is present in the change.

@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch 5 times, most recently from 6031664 to b11673e Compare January 18, 2021 16:24
@eoghanmurray
Copy link
Contributor Author

(Sorry still iterating on this as tests are not passing; my dev test environment isn't working so uploading changes here to check)

@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch 6 times, most recently from 6304e0e to 69c1e30 Compare January 19, 2021 16:28
@eoghanmurray
Copy link
Contributor Author

eoghanmurray commented Jan 19, 2021

@Yuyz0112 would you be able to take a look at the failing build e.g. https://travis-ci.com/github/rrweb-io/rrweb/builds/213675796

i.e. there should be a text attribute change as follows:
"style": ""left: 8px; width: 46.2109px; top: 115.5px; bottom: auto; display: block;""

[Edit] #468 has the fix for why the style changes weren't showing up [End Edit]

@eoghanmurray
Copy link
Contributor Author

Okay, I've got all the tests sorted now and have added one extra test for !important as this is treated separately.

Aside: during replay, it won't break if a previous string value for the style attribute is present from a prior recording.

@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch from 6f686c9 to 94eb6d6 Compare January 20, 2021 16:34
@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch 2 times, most recently from d0c4f0c to b92e85a Compare February 3, 2021 17:53
Comment on lines 401 to 402
const new_value = target.style.getPropertyValue(pname);
const new_priority = target.style.getPropertyPriority(pname);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For code hygiene it's probably better to camelCase these variable names.

Suggested change
const new_value = target.style.getPropertyValue(pname);
const new_priority = target.style.getPropertyPriority(pname);
const newValue = target.style.getPropertyValue(pname);
const newPriority = target.style.getPropertyPriority(pname);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks; I take it I cannot just commit this suggestion as-is?
I'll make a new commit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the original commit with your suggested change as was force-pushing anyhow.

@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch 2 times, most recently from fc68ba7 to 2ddd84f Compare February 22, 2021 12:30
@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch 3 times, most recently from 8855836 to 38f84e2 Compare June 15, 2021 12:22
@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch 2 times, most recently from a463f69 to b42dda8 Compare June 25, 2021 16:05
@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch from b42dda8 to ee7c23f Compare July 2, 2021 20:45
@Juice10
Copy link
Contributor

Juice10 commented Jul 3, 2021

Looks good to me, only thing I can think of would be doing some checking in the replay, if the element has a style attribute. Just in case something went wrong with the recorded ids and we are accidentally referencing a text node. I believe there is a try/catch around the get/setAttributes for that reason on the string type of style mutation.

@Yuyz0112
Copy link
Member

Yuyz0112 commented Jul 5, 2021

@eoghanmurray Looks like this approach will not trigger reflow/repaint during diff, because the dummy node is not in the DOM tree, am I right?

And is it worth a little benchmark test to show the time of diff?

@eoghanmurray
Copy link
Contributor Author

Ya not added to the DOM; I felt safe creating the dummy node as we are doing that elsewhere in the code e.g. at https://github.com/rrweb-io/rrweb-snapshot/blob/master/src/snapshot.ts#L204

I made a quick performance.now() comparison and got the following results

Firefox (over 277 events):
Old avg. 0.184ms
New avg 0.234ms
Slower 1.27x

Chrome (over 256 events):
Old avg 0.073ms
New avg 0.224ms
Slower 3.03x

Looks like the difference is due to the existing transformAttribute code path being just as slow in Firefox.

This is happening in the DOMMutation callback which is 'off the main loop', right?

@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch from ee7c23f to cbe0914 Compare July 8, 2021 09:15
…ngle style properties result in storage of a rewrite for the full style attribute, which may be very large.

Had an example of a website using http://schillmania.com/projects/snowstorm/ where many direct style changes were happening every second across many 'snowflake' elements, with each attribute change looking like:
"style":"color: rgb(255, 255, 255); position: absolute; width: 8px; height: 8px; font-family: arial, verdana; overflow: hidden; font-weight: normal; z-index: 0; display: block; bottom: auto; opacity: 1; padding: 0px; margin: 0px; font-size: 10px; line-height: 10px; text-align: center; vertical-align: baseline; left: 242.807px; top: 85.7332px;"
even though maybe just the left/top position had been changed
… an `!important` flag - saves 6 chars per style attr in the json :)
@eoghanmurray eoghanmurray force-pushed the compact-style-mutation branch from cbe0914 to ea78955 Compare July 8, 2021 15:11
@Yuyz0112
Copy link
Member

LGTM

@eoghanmurray This happens off the main loop, but still executing in sync mode, so it may block following execution. I think we can:

  1. support running record logic in web worker
  2. do record logic in request idle callback

@Yuyz0112 Yuyz0112 merged commit 9875a3d into rrweb-io:master Jul 11, 2021
@eoghanmurray
Copy link
Contributor Author

eoghanmurray commented Jul 12, 2021

for 1. I thought web workers didn't have access to the DOM?
2. looks good where browser supports it

eoghanmurray added a commit to statcounter/rrweb that referenced this pull request Jul 13, 2021
YunFeng0817 pushed a commit that referenced this pull request Jul 15, 2021
* My best interpretation of what the typings should look like after merge of #464

* Apply variable name changes as per Juice10 review

Co-authored-by: Justin Halsall <Juice10@users.noreply.github.com>

* fix types

Co-authored-by: Eoghan Murray <eoghan@getthere.ie>
@razor-x
Copy link

razor-x commented Jan 27, 2022

Hello. I'm coming here as a PostHog user. It uses this lib to power session recording. My app is able to run a strict CSP, specifically without 'unsafe-inline' in the policy. However, I noticed a lot of CSP errors generated like the one below:

image

After some digging, I believe this change broke support for users who run a strict CSP. Setting the style attribute on the dummy node will be blocked without 'unsafe-inline'.

Would you prefer I start a new issue or continue discussion here on possible solutions? Best.

@Juice10
Copy link
Contributor

Juice10 commented Jan 27, 2022

@razor-x thanks for bringing that issue up!
It probably makes most sense to start a new issue for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants