Nightly browser versions, bugs and polyfills

[zloirock]

I was not able to find this case in discussions, but it's a serious problem.

For example, we download an early nightly Chromium / Safari / FF build - for example, version 15.0.0.1. At this moment, feature X is not implemented in this browser. A website foo.bar uses an abstract polyfill service which sends only required for this browser polyfills and uses the feature X. This browser implements the feature X in version 15.0.0.2. After some time, released 15.0.1.0 stable version of this browser. To compat data of our service added information that 15.0.1.0 version of our browser supports the feature X. But with the CH proposal, our site will see only that the version of the browser is 15. Our abstract polyfill service will not send a polyfill for the feature X and our site will stop work in our (15.0.0.1) browser. As an option, our polyfill service could send polyfills required for the previous significant version of the browser - but it will seriously reduce the usefulness of this polyfill service.

[zloirock]

Another problem is browser bugs. I remember many cases when the first stable major version of the browser contained a significant bug and this bug was fixed immediately in a patch release. If we will not know a patch release version, we will be forced to send polyfill which fixes this bug for all versions of this browser until the next major version - and for some browsers (like Safari) it could be a long time.

[amtunlimited]

The Sec-CH-UA-Full-Version client hint should give you what you need. Is there a problem with that?

[zloirock]

The problem that Sec-CH-UA-Full-Version is not available at the initial request, so that kills all pluses of a polyfills service.

[amtunlimited]

It was my understanding that polyfills were dynamically rendered when the script was fetched, which is a subresource request that will receive client hints, even if the page hasn't been visited before. This is how, for example, polyfill.io works

[amtunlimited]

"Initial request", in this case, means the first network request (i.e. the first navigation and load of the initial document)

If you're still worried about first-navigation issues, there's also the Client Hint reliability mechanisms

[zloirock]

This subresource most likely will be on a separate domain - for example, polyfill.io case. Even if not, the requirement of adding additional response headers on their main webserver side will reduce the number of users of such service by an order of magnitude.

[jridgewell]

But with the CH proposal, our site will see only that the version of the browser is 15. Our abstract polyfill service will not send a polyfill for the feature X and our site will stop work in our (15.0.0.1) browser. As an option, our polyfill service could send polyfills required for the previous significant version of the browser - but it will seriously reduce the usefulness of this polyfill service.

This is inherent in Lowest Common Denominator builds, and there's really nothing that Client Hints can do to solve it. Because X isn't supported in all v15 browsers, X must be polyfilled for all v15 requests. When v16 comes around, it can be removed.

If we will not know a patch release version, we will be forced to send polyfill which fixes this bug for all versions of this browser until the next major version - and for some browsers (like Safari) it could be a long time.

The Sec-CH-UA header actually uses a significant version for exactly this reason. Chrome/Firefox, which release major versions every few weeks, can define their significant version as v15 (just include the MAJOR). Safari, which releases slower (and releases major features in MINOR updates), can define its significant version as v15.1 (MAJOR.MINOR).

This will hopefully avoid the case where a polyfill has to be shipped to natively-supported browsers for an extended period of time.

[zloirock]

This is inherent in Lowest Common Denominator builds, and there's really nothing that Client Hints can do to solve it. Because X isn't supported in all v15 browsers, X must be polyfilled for all v15 requests. When v16 comes around, it can be removed.

Since our compat data contains the information that this feature is supported in 15.0.1.0 and if the browser will send a hint with the full version of the browser (like now, with UA), we could send this polyfill to 15.0.0.1, but not to 15.0.1.0.

The Sec-CH-UA header actually uses a significant version for exactly this reason. Chrome/Firefox, which release major versions every few weeks, can define their significant version as v15 (just include the MAJOR). Safari, which releases slower (and releases major features in MINOR updates), can define its significant version as v15.1 (MAJOR.MINOR).

Even in this case, it's unnecessary polyfilling.

The first case is a critical case for polyfilling, so a header with the full version of the browser should be sent on the first request.

[jridgewell]

if the browser will send a hint with the full version of the browser (like now, with UA), we could send this polyfill to 15.0.0.1, but not to 15.0.1.0.

I think it's a pretty explicit goal of this proposal to limit passive information, and the patch level version information creates a lot of entropy for fingerprinting.

The first case is a critical case for polyfilling, so a header with the full version of the browser should be sent on the first request.

I think shipping a polyfill for a few weeks while the significant version remains the same is an acceptable compromise here. If full versioning information is critical, the reliability mechanisms allow you specify a Critical-CH Response Header, or a ACCEPT_CH connection level setting to avoid a round trip.

[zloirock]

If it will break a useful case like a polyfilling - it's not something that should be limited unlike the other parts of UA. For some browsers it could be a few weeks, for Safari - a half year, for some unupdated mobile browsers - many years (my mom updated iOS previous time in 2017). I already wrote above why response headers can't be applied in this case.

[amtunlimited]

Genuine question: I'm trying to find a real world example of a polyfill service using a higher fidelity than significant version (i.e. major version, or major and minor for Safari/WebKit) and I'm not finding anything.

Stepping back a bit, let's examine polyfill.io as a case study. First, polyfill's ua library drops PATCH version, so it really only uses MAJOR and MINOR. We've already discussed that major and minor will both be used for WebKit, and neither Chromium or Firefox actually use the minor version (except for Firefox ESR, which pulls in security patches from other major releases, no bug fixes or features). In almost all cases, there's nothing to use except for significant version.

[zloirock]

The simple answer - I'm the author of core-js, even naive detection shows that at this moment it's used on ~59% of top 1000 websites. I'm working on a prototype of such service which adds this functionality to core-js and I can't recommend the usage of polyfills.io because of cases like you mentioned.

[xpahos]

Genuine question: I'm trying to find a real world example of a polyfill service using a higher fidelity than significant version (i.e. major version, or major and minor for Safari/WebKit) and I'm not finding anything.

@amtunlimited it's not just polyfill problem. The previous build of Safari(14.1 15611.1.21.161.5, 15611) has a problem with binaries linkage. Safari linked with WebRTC library where some methods don't exist. The current build of Safari(Version 14.1 15611.1.21.161.7, 15611) has no problems with WebRTC. Potentially the problem could be with any part of Safari or any other browser. This year I was on vacation in a small village where our host asked me to help him with Chrome. He said that he can't update it. OS X was pre-installed on the factory and never updated for 6 years. HTTP/2, BBR, TLS 0-RTT, TLS certificate compression and a lot more optimisations will be ruined because for some users with outdated software we will be forced to use additional request with info about the build version.

[Sora2455]

@xpahos The WebRTC thing is valid, but if your browser hasn't been updated in 6 years I don't think that the minor version matters anymore...

[xpahos]

@xpahos The WebRTC thing is valid, but if your browser hasn't been updated in 6 years I don't think that the minor version matters anymore...

Problem not with browser updating. Chrome has a built-in updating mechanism. It's possible to update the browser to a buggy version before the end of support the current platform. E.g. auto-updater installs version 11.11.1 on OS X Z. Because OS X is outdated X.1.1 will be the latest available release for OS X Z. Version X.1.1 of the browser could be used by the user for years. For users like those, we have a hack. It's a lightweight page without some functionality.

[miketaylr]

Thanks for the discussion, but I don't think we plan to change anything here at the moment.

[zloirock]

A very constructive resolution. Maybe, in this case, it makes sense to make client hints and freezing of user agent incompatible with the web? I have all the required resources for that.

[miketaylr]

Maybe, in this case, it makes sense to make client hints and freezing of user agent incompatible with the web? I have all the required resources for that.

Can you clarify what you mean?

[zloirock]

I mean that in case of nothing will be done, I'm thinking about throwing an error in my libraries in case of detection of the frozen user agent and missing all required data in client hints. And, as you can see above, they are present on most internet websites. The current solution proposed ua-client-hints is destructive and I can't allow it for the web.

[miketaylr]

Again to clarify, I think all the solutions exist for the problem that's described:

1. Critical-CH and ACCEPT_CH exist to get hints on "first-request" (or nearly first request)
2. Markup-based delegation of a hint (such as full-version) to a 3rd party domain is possible via https://chromestatus.com/feature/5684289032159232

@zloirock is there something I'm missing?

[zloirock]

At least, it requires a set of additional headers - and it's not always possible.

Detection of engine version required for polyfills not only for services on the server-side - it's also client code. In too many cases, feature detection is impossible or too expensive - for example, many cases of feature detection in Chrome invalidate protectors which causes the whole code to slow down by tens and hundreds of times - so we are forced to check the version of the engine. No one will send all required headers for that.

I agree that the current user agent contains too much extra data for fingerprinting - but the full version of the engine is not something that should be hidden by default.

[miketaylr]

At least, it requires a set of additional headers - and it's not always possible.

Yeah, agree. That's the primary reason we added support for a way to request and delegate the CH via HTML. See https://groups.google.com/a/chromium.org/g/blink-dev/c/cGOjmfOamsE/m/ORGaw959BAAJ

For purely JS needs (that isn't a polyfill service that wants to Vary on a version), it's possible to use something like await navigator.userAgentData.getHighEntropyValues(["uaFullVersion"]) or await navigator.userAgentData.getHighEntropyValues(["fullVersionList"]) in a 1st or 3rd party context - the JS API doesn't require HTTP delegation of hints at all.

[zloirock]

It's an async code that is not acceptable for most polyfills.

[zloirock]

Why it can't be provided as a sync API? That solves a significant part of this issue.

[zloirock]

A month has gone and I still do not see an answer or a reopening of the issue.

[romainmenke]

Stepping back a bit, let's examine polyfill.io as a case study. First, polyfill's ua library drops PATCH version, so it really only uses MAJOR and MINOR.

This might be technically correct but should not be used as a reference to argue against patch versions in client hints. There are many aspects were polyfill.io needs to make a compromise.

We've already discussed that major and minor will both be used for WebKit, and neither Chromium or Firefox actually use the minor version (except for Firefox ESR, which pulls in security patches from other major releases, no bug fixes or features). In almost all cases, there's nothing to use except for significant version.

This is not fully correct. Chromium and Firefox definitely do use minor version even when they don't expose it implicitly as part of the user agent string.
Just a few weeks ago Chrome shipped a completely broken implementation of @layer in CSS. That mid cycle fix was released as a minor version bump.

Using client-side javascript to determine this minor version based on this bug was trivial. Adding this part to a user agent string or client hints does not increase the available finger printing surface.

The lack of a good API will just cause worse solutions to surface :/
This will harm users and the web.

[zloirock]

One more month of ignoring. So, any news? -)

[miketaylr]

Apologies, not ignoring. Just swamped with other things :( - hopefully can find some time in the next few days to respond.

[zloirock]

And three months more :-D