Detect "versioned" service workers and manifests

[ithinkihaveacat]

Even though they're often static files, having a "versioned" service worker or manifest causes issues, especially for systems that attempt to uniquely identify PWAs. (See GoogleChromeLabs/gulliver#323 (comment), "Avoid changing the URL of your service worker script".)

Lighthouse should attempt to detect PWAs that employ versioned service workers or manifests. One way of doing this might be to identify resources that have a long max-age. (Longer than a week?)

[ebidel]

...and create an audit that warns if the site is versioning those urls?

Is there justification why GoogleChromeLabs/gulliver#323 (comment) is necessarily a bad thing? Perhaps a technical reason, other than being able to uniquely identify a PWA?

[ithinkihaveacat]

I can't think of any strong reasons for not versioning service workers and manifests apart from making it easier to uniquely identify PWAs, though @slightlyoff may have some thoughts.

(I guess one argument is similar to the reason we don't version regular HTML pages. HTML pages are at the root of the dependency graph and know about all mutual dependencies, and so they're able to include the exact resources they require. Manifests and (especially) service workers are similarly sort of at the root of the "PWA" dependency graph.)

[paulirish]

@ithinkihaveacat what is the change you're suggesting here? I'm not really grokking what you're asking for. :)

[ithinkihaveacat]

Ah, sorry!

Expected

Lighthouse issues an error/warning when a manifest or service worker is "versioned." e.g. https://weather.com/weather/assets/manifest.507fcb498f4e29acfeed7596fe002857.json.

Actual

Lighthouse does not issue an error/warning.

Rationale

It's useful/necessary to uniquely identify PWAs by the tuple [manifest_url, start_url]. However, this is not effective if the manifest URL is versioned, since it changes on every deployment. (See @slightlyoff's comment.)

Implementation Idea

Although the URLs of versioned assets necessarily contain a random-looking component, it's probably not easy to reliably identify such components. However, it may be possible to determine whether these problematic URLs are cached for a long time, which is a good indication that they will change on each deployment. (For example, the weather.com manifest linked above has a cache-control: max-age=31536000.)

[paulirish]

Thanks that explanation is helpful. Basically: your filenames for manifest/sw are always the same.

In order to move this forward we'll need to follow the new audit process here. https://github.com/GoogleChrome/lighthouse/blob/master/docs/new-audits.md

[mgiuca]

You could perhaps do a more basic heuristic (without having to depend upon past history) which is checking whether the manifest filename has something that looks like a version in it. Either a version number (/v[0-9]+/) or a hash (/[0-9a-fA-F]{8,40}/), and issue a warning.

[ithinkihaveacat]

https://pwa-directory.appspot.com/ has a collection of over 1000 manifest that can be accessed via its API that can be used to test heuristics.

@mgiuca's suggestion looks pretty good. I would suggest reducing the requirement to a run of 7 hex characters, and also looking for v=:

curl -sSL 'https://pwa-directory.appspot.com/api/pwa/?limit=4000' | jq -r
'.[] | .manifestUrl' > manifestUrl.txt
perl -ne 'print if /[0-9a-fA-F]{7}/ || /v[0-9]+/ || /v=/' < manifestUrl.txt

Using this criteria, 71/1366 (~5%) of manifests are "versioned."

This still misses a few manifests e.g.

https://pwa.fishki.net/manifest.json?5257
https://meshok.net/manifest.json?3
https://ssl.tzoo-img.com/res/favicon/manifest.json?v=2kq2msw2

Is this enough info to propose a new audit? (It would go in the PWA section.)

[paulirish]

Is this enough info to propose a new audit?

Yeah this works for me.

[connorjclark]

Are there tools that tack on version info to generated manifest files? if so - may be better to just fix at the source, rather than making an audit for this

[aradalvand]

In the case of manifests, doesn't the new id member solve this problem?

[adrianaixba]

As per Chrome’s updated Installability Criteria, Lighthouse will be deprecating the PWA category in the next upcoming release. For future PWA testing, users will be directed to use the updated PWA documentation. Marking this as closed!