Distinguish autopublished /TR/ URLs from the rest

[foolip]

Some W3C specs are published to /TR/ for every commit, while most don't and require github.io URLs to lead the latest.

In mdn/browser-compat-data#22694 I learned of one such case – WebNN – and it looks like https://github.com/webmachinelearning/webnn/blob/main/.github/workflows/auto-publish.yml is how that works.

I would be great if browser-specs could provide the information needed for other projects like BCD or web-features to know when a /TR/ is autopublished and not. This would avoid the need to keep track of which WGs prefer these URLs, and can lint for most up-to-date and preferred spec URLs more easily.

One possible implementation is to use /TR/ as nightly URL if /TR/ is autopublished, and put the github.io URLs in as alternatives, not the canonical.

[tidoust]

I agree that it would be valuable information to have.

Of course, the question becomes: to avoid having to maintain the information manually, can we compute it automatically? Some ways to do that, on top of my head:

• Look for an auto-publish.yml file. But then that's just a naming convention. Some groups use other job names (e.g. WebGPU/WGSL). Or use their own logic (e.g., WebAssembly). Also, there are cases where the spec-prod action is used to deploy the spec automatically to GitHub Pages, but not to /TR (as done for WebTransport, WebAuthn, or MathML Core), which could create false positives.
• Compare the revision that appears in a <meta> tag in the /TR version with the latest repo commit. Now, that's not enough in itself to claim that the spec is published automatically to /TR. Also, commits may have touched other parts of the repo and not lead to a new published version of the spec. Or publication may still be pending when we check specs. Or publication may have failed temporarily, as happens from time to time.
• See whether the W3C API could be extended somehow to report on what triggered last call to Echidna. I'm not sure the system sees a difference and colelcts that information at all, though.
• Make the spec-prod action inject a <meta> tag to the spec published on /TR to make it explicit that the spec was published automatically. That would not capture cases where a group decided to stop automatic publications, but that should remain fairly exceptional.

Now, it's not clear to me that "most don't" is still a valid statement: a growing number of W3C working groups have switched to automatic publication in practice (including Device and Sensors, Immersive Web, Media, Pointer Events, Second Screen, WebApps, WebAppSec, WebAssembly, Web Editing). The main exception is going to be the CSS WG.

Perhaps this could be a good opportunity to assess whether linking to the Editor's Draft is still needed, and to take an opposite approach: instead of identifying which specs can be referenced using the /TR version, identify those that should not.

I'll try to create a first draft split of /TR specs (auto-published, non auto-published, don't know) to inform the discussion.

[foolip]

My claim of "most don't" wasn't researched, just based on my hunch and past experience. It's great if most groups are autopublishing now, I didn't know it was such a success 😄

If Echidna is the system that has to be involved for autopublishing, then that seems like the place to get this information.

I have to say that flipping the default makes me a bit nervous. What happens when a group starts working a new version or level of spec, do they always do it in-place or does it sometimes move to a github.io URL while /TR/ goes stale?

[dontcallmedom]

there remains at least a clear situation where TR can't keep up with the editors draft: once a spec reaches PR/Rec; even if the Process now allows for easier updates to Rec once published, that isn't a seamless or automated process yet.

[foolip]

Oh, that's very good to know. In that case I'm quite hesitant to use autopublished /TR/ URLs in places like MDN, if they will one day become snapshots that don't reflect the latest work.

[tidoust]

Discussing with @dontcallmedom, it seems clear that "is the spec autopublished?" is not a very useful question to answer to determine which URL to use. That status changes over time depending on where the spec is on the Recommendation track.

It may be better to know whether the /TR version of a spec is up-to-date or outdated. This information is not available in browser-specs, but it's available in Webref. See below for an analysis. As with an hypothetical autopublished flag, that information is not going to be useful to take a decision: it may change over time. In some cases, that's just temporary (WebGPU is a couple of days late due to some publication hiccup last time the ED was updated). In other cases, because the group changes its workflow, or because the spec reaches a status where updates are rarer but not automatable.

Anyway, looking at Webref data, I see 313 specs published to /TR and tracked in browser-specs (112 from the CSS WG, 201 from other WGs):

• 141 are up-to-date (14 from the CSS WG, 127 from other WGs)
• 172 are older than the ED (98 from the CSS WG, 74 from other WGs)

This confirms that, CSS excluded, a majority of the /TR specs tracked in browser-specs are now up-to-date with their ED. It also confirms that /TR versions of some CSS specs are very old:

Outdated /TR specs

From most to least outdated:

• /TR/css-namespaces-3/ is ~9 years (3597 days) late
• /TR/css-line-grid-1/ is ~9 years (3417 days) late
• /TR/css-scoping-1/ is ~9 years (3413 days) late
• /TR/css-regions-1/ is ~9 years (3394 days) late
• /TR/css3-exclusions/ is ~9 years (3337 days) late
• /TR/svg-integration/ is ~8 years (3247 days) late
• /TR/touch-events/ is ~8 years (3184 days) late
• /TR/webmidi/ is ~8 years (3158 days) late
• /TR/css-page-floats-3/ is ~8 years (3053 days) late
• /TR/cssom-view-1/ is ~8 years (2920 days) late
• /TR/svg-strokes/ is ~7 years (2890 days) late
• /TR/CSS22/ is ~7 years (2806 days) late
• /TR/mse-byte-stream-format-mp2t/ is ~7 years (2716 days) late
• /TR/mse-byte-stream-format-isobmff/ is ~7 years (2715 days) late
• /TR/mse-byte-stream-format-mpeg-audio/ is ~7 years (2715 days) late
• /TR/mse-byte-stream-format-registry/ is ~7 years (2715 days) late
• /TR/mse-byte-stream-format-webm/ is ~7 years (2715 days) late
• /TR/vibration/ is ~7 years (2635 days) late
• /TR/upgrade-insecure-requests/ is ~7 years (2562 days) late
• /TR/referrer-policy/ is ~6 years (2507 days) late
• /TR/WebCryptoAPI/ is ~6 years (2398 days) late
• /TR/encrypted-media/ is ~6 years (2375 days) late
• /TR/css-rhythm-1/ is ~6 years (2369 days) late
• /TR/SRI/ is ~6 years (2351 days) late
• /TR/css-round-display-1/ is ~6 years (2258 days) late
• /TR/clear-site-data/ is ~5 years (2171 days) late
• /TR/css-logical-1/ is ~5 years (1976 days) late
• /TR/css-shadow-parts-1/ is ~5 years (1952 days) late
• /TR/graphics-aam-1.0/ is ~5 years (1939 days) late
• /TR/css-break-4/ is ~5 years (1910 days) late
• /TR/filter-effects-1/ is ~5 years (1904 days) late
• /TR/webrtc-identity/ is ~5 years (1902 days) late
• /TR/css-transitions-1/ is ~5 years (1886 days) late
• /TR/DOM-Parsing/ is ~4 years (1811 days) late
• /TR/fingerprinting-guidance/ is ~4 years (1744 days) late
• /TR/css-flexbox-1/ is ~4 years (1742 days) late
• /TR/css-break-3/ is ~4 years (1727 days) late
• /TR/css-ui-3/ is ~4 years (1712 days) late
• /TR/motion-1/ is ~4 years (1686 days) late
• /TR/csp-embedded-enforcement/ is ~4 years (1656 days) late
• /TR/css-content-3/ is ~4 years (1648 days) late
• /TR/timing-entrytypes-registry/ is ~4 years (1645 days) late
• /TR/css-transforms-1/ is ~4 years (1634 days) late
• /TR/css-tables-3/ is ~4 years (1632 days) late
• /TR/SVG2/ is ~4 years (1616 days) late
• /TR/payment-method-manifest/ is ~4 years (1597 days) late
• /TR/css-layout-api-1/ is ~4 years (1555 days) late
• /TR/css-writing-modes-4/ is ~4 years (1490 days) late
• /TR/webvtt1/ is ~3 years (1436 days) late
• /TR/css-overscroll-1/ is ~3 years (1366 days) late
• /TR/css-writing-modes-3/ is ~3 years (1356 days) late
• /TR/json-ld11-framing/ is ~3 years (1330 days) late
• /TR/json-ld11-api/ is ~3 years (1329 days) late
• /TR/json-ld11/ is ~3 years (1329 days) late
• /TR/css-nav-1/ is ~3 years (1191 days) late
• /TR/geometry-1/ is ~3 years (1164 days) late
• /TR/css-lists-3/ is ~3 years (1163 days) late
• /TR/css-animation-worklet-1/ is ~3 years (1116 days) late
• /TR/css-cascade-3/ is ~2 years (1083 days) late
• /TR/resize-observer-1/ is ~2 years (1081 days) late
• /TR/svg-aam-1.0/ is ~2 years (1079 days) late
• /TR/css-ui-4/ is ~2 years (1077 days) late
• /TR/pub-manifest/ is ~2 years (1055 days) late
• /TR/css-grid-1/ is ~2 years (1050 days) late
• /TR/css-grid-2/ is ~2 years (1050 days) late
• /TR/graphics-aria-1.0/ is ~2 years (1023 days) late
• /TR/webaudio/ is ~2 years (1012 days) late
• /TR/css-sizing-4/ is ~2 years (988 days) late
• /TR/audiobooks/ is ~2 years (951 days) late
• /TR/css-masking-1/ is ~2 years (946 days) late
• /TR/css-scroll-snap-1/ is ~2 years (899 days) late
• /TR/css-counter-styles-3/ is ~2 years (874 days) late
• /TR/css-multicol-1/ is ~2 years (853 days) late
• /TR/cssom-1/ is ~2 years (834 days) late
• /TR/css-scrollbars-1/ is ~2 years (833 days) late
• /TR/mediaqueries-5/ is ~2 years (817 days) late
• /TR/css-sizing-3/ is ~2 years (804 days) late
• /TR/css-conditional-5/ is ~2 years (798 days) late
• /TR/edit-context/ is ~2 years (785 days) late
• /TR/css-conditional-4/ is ~2 years (761 days) late
• /TR/mediaqueries-4/ is ~2 years (760 days) late
• /TR/battery-status/ is ~23 months (701 days) late
• /TR/css-text-decor-4/ is ~22 months (674 days) late
• /TR/css-transforms-2/ is ~22 months (672 days) late
• /TR/css-syntax-3/ is ~21 months (653 days) late
• /TR/css-text-decor-3/ is ~20 months (629 days) late
• /TR/css-variables-1/ is ~20 months (621 days) late
• /TR/service-workers/ is ~20 months (610 days) late
• /TR/css-cascade-4/ is ~19 months (591 days) late
• /TR/css-cascade-5/ is ~19 months (591 days) late
• /TR/css-conditional-3/ is ~19 months (591 days) late
• /TR/css-contain-3/ is ~19 months (578 days) late
• /TR/pointerlock-2/ is ~19 months (570 days) late
• /TR/trusted-types/ is ~18 months (546 days) late
• /TR/mediacapture-viewport/ is ~17 months (539 days) late
• /TR/css-style-attr/ is ~17 months (518 days) late
• /TR/css-box-4/ is ~16 months (505 days) late
• /TR/css-color-adjust-1/ is ~16 months (499 days) late
• /TR/geolocation/ is ~16 months (494 days) late
• /TR/selectors-4/ is ~16 months (481 days) late
• /TR/css-highlight-api-1/ is ~15 months (475 days) late
• /TR/css-shapes-1/ is ~15 months (462 days) late
• /TR/capture-handle-identity/ is ~15 months (457 days) late
• /TR/mathml4/ is ~14 months (426 days) late
• /TR/css-contain-2/ is ~13 months (417 days) late
• /TR/css-align-3/ is ~13 months (402 days) late
• /TR/css-images-4/ is ~13 months (396 days) late
• /TR/css-ruby-1/ is ~12 months (389 days) late
• /TR/webrtc/ is ~12 months (381 days) late
• /TR/image-resource/ is ~12 months (375 days) late
• /TR/web-animations-2/ is ~12 months (373 days) late
• /TR/css-overflow-3/ is ~12 months (362 days) late
• /TR/css-box-3/ is ~11 months (351 days) late
• /TR/css-font-loading-3/ is ~11 months (348 days) late
• /TR/css-speech-1/ is ~11 months (344 days) late
• /TR/css-overflow-4/ is ~11 months (342 days) late
• /TR/selection-api/ is ~10 months (309 days) late
• /TR/css-position-3/ is ~10 months (305 days) late
• /TR/epub-33/ is ~9 months (299 days) late
• /TR/epub-rs-33/ is ~9 months (299 days) late
• /TR/css-inline-3/ is ~9 months (298 days) late
• /TR/css-easing-1/ is ~9 months (291 days) late
• /TR/css-contain-1/ is ~9 months (286 days) late
• /TR/web-animations-1/ is ~9 months (284 days) late
• /TR/css-animations-1/ is ~9 months (281 days) late
• /TR/wai-aria-1.2/ is ~9 months (280 days) late
• /TR/css-2022/ is ~9 months (278 days) late
• /TR/clipboard-apis/ is ~9 months (274 days) late
• /TR/css-cascade-6/ is ~8 months (265 days) late
• /TR/IFT/ is ~8 months (261 days) late
• /TR/css-pseudo-4/ is ~8 months (256 days) late
• /TR/css-nesting-1/ is ~8 months (247 days) late
• /TR/scroll-animations-1/ is ~7 months (239 days) late
• /TR/css-display-3/ is ~7 months (237 days) late
• /TR/css-view-transitions-1/ is ~6 months (200 days) late
• /TR/miniapp-manifest/ is ~6 months (181 days) late
• /TR/css-text-3/ is ~5 months (169 days) late
• /TR/webauthn-3/ is ~5 months (169 days) late
• /TR/mediacapture-region/ is ~5 months (153 days) late
• /TR/web-locks/ is ~4 months (145 days) late
• /TR/screen-wake-lock/ is ~4 months (136 days) late
• /TR/mst-content-hint/ is ~3 months (118 days) late
• /TR/mathml-core/ is ~3 months (115 days) late
• /TR/audio-output/ is ~3 months (103 days) late
• /TR/core-aam-1.2/ is ~3 months (102 days) late
• /TR/webtransport/ is ~3 months (97 days) late
• /TR/css-images-3/ is ~3 months (92 days) late
• /TR/css-page-3/ is ~2 months (82 days) late
• /TR/css-animations-2/ is ~2 months (66 days) late
• /TR/css-fonts-4/ is 49 days late
• /TR/wai-aria-1.3/ is 49 days late
• /TR/rdf12-xml/ is 44 days late
• /TR/webrtc-stats/ is 42 days late
• /TR/css-scroll-anchoring-1/ is 41 days late
• /TR/proximity/ is 39 days late
• /TR/uievents-key/ is 38 days late
• /TR/css-color-4/ is 37 days late
• /TR/sparql12-entailment/ is 35 days late
• /TR/sparql12-graph-store-protocol/ is 34 days late
• /TR/css-text-4/ is 25 days late
• /TR/css-fonts-5/ is 16 days late
• /TR/css-color-5/ is 14 days late
• /TR/webdriver2/ is 14 days late
• /TR/FileAPI/ is 11 days late
• /TR/mediacapture-streams/ is 11 days late
• /TR/orientation-event/ is 11 days late
• /TR/css-values-4/ is 10 days late
• /TR/css-2023/ is 4 days late
• /TR/webgpu/ is 3 days late
• /TR/compute-pressure/ is 2 days late
• /TR/webnn/ is 2 days late
• /TR/css-backgrounds-3/ is 1 days late

List of up-to-date /TR specs

• /TR/av1-spec/
• /TR/accelerometer/
• /TR/accname-1.2/
• /TR/ambient-light/
• /TR/appmanifest/
• /TR/autoplay-detection/
• /TR/badging/
• /TR/beacon/
• /TR/change-password-url/
• /TR/compositing-1/
• /TR/contact-picker/
• /TR/credential-management-1/
• /TR/CSP3/
• /TR/css-anchor-position-1/
• /TR/css-color-3/
• /TR/css-gcpm-3/
• /TR/css-paint-api-1/
• /TR/css-properties-values-api-1/
• /TR/css-transitions-2/
• /TR/css-typed-om-1/
• /TR/css-values-3/
• /TR/css-viewport-1/
• /TR/css-will-change-1/
• /TR/device-memory-1/
• /TR/device-posture/
• /TR/dpub-aam-1.1/
• /TR/dpub-aria-1.1/
• /TR/event-timing/
• /TR/fetch-metadata/
• /TR/fill-stroke-3/
• /TR/gamepad/
• /TR/generic-sensor/
• /TR/geolocation-sensor/
• /TR/gyroscope/
• /TR/hr-time-3/
• /TR/html-aam-1.0/
• /TR/html-aria/
• /TR/html-media-capture/
• /TR/i18n-glossary/
• /TR/image-capture/
• /TR/IndexedDB-3/
• /TR/input-events-2/
• /TR/intersection-observer/
• /TR/largest-contentful-paint/
• /TR/longtasks-1/
• /TR/magnetometer/
• /TR/manifest-app-info/
• /TR/media-capabilities/
• /TR/media-source-2/
• /TR/mediacapture-fromelement/
• /TR/mediacapture-transform/
• /TR/mediaqueries-3/
• /TR/mediasession/
• /TR/mediastream-recording/
• /TR/miniapp-lifecycle/
• /TR/miniapp-packaging/
• /TR/mixed-content/
• /TR/n-quads/
• /TR/navigation-timing-2/
• /TR/network-error-logging/
• /TR/openscreenprotocol/
• /TR/orientation-sensor/
• /TR/paint-timing/
• /TR/payment-handler/
• /TR/payment-method-id/
• /TR/payment-request-1.1/
• /TR/performance-timeline/
• /TR/permissions-policy-1/
• /TR/permissions/
• /TR/picture-in-picture/
• /TR/pointerevents3/
• /TR/presentation-api/
• /TR/push-api/
• /TR/rdf-canon/
• /TR/rdf12-concepts/
• /TR/rdf12-n-quads/
• /TR/rdf12-n-triples/
• /TR/rdf12-schema/
• /TR/rdf12-semantics/
• /TR/rdf12-trig/
• /TR/rdf12-turtle/
• /TR/remote-playback/
• /TR/reporting-1/
• /TR/requestidlecallback/
• /TR/resource-timing/
• /TR/screen-capture/
• /TR/screen-orientation/
• /TR/secure-contexts/
• /TR/secure-payment-confirmation/
• /TR/selectors-3/
• /TR/server-timing/
• /TR/sparql12-federated-query/
• /TR/sparql12-protocol/
• /TR/sparql12-query/
• /TR/sparql12-results-csv-tsv/
• /TR/sparql12-results-json/
• /TR/sparql12-results-xml/
• /TR/sparql12-service-description/
• /TR/sparql12-update/
• /TR/SVG11/
• /TR/test-methodology/
• /TR/uievents-code/
• /TR/uievents/
• /TR/user-timing/
• /TR/vc-data-integrity/
• /TR/vc-data-model-2.0/
• /TR/virtual-keyboard/
• /TR/wasm-core-2/
• /TR/wasm-js-api-2/
• /TR/wasm-web-api-2/
• /TR/web-share/
• /TR/webcodecs-aac-codec-registration/
• /TR/webcodecs-alaw-codec-registration/
• /TR/webcodecs-av1-codec-registration/
• /TR/webcodecs-avc-codec-registration/
• /TR/webcodecs-codec-registry/
• /TR/webcodecs-flac-codec-registration/
• /TR/webcodecs-hevc-codec-registration/
• /TR/webcodecs-mp3-codec-registration/
• /TR/webcodecs-opus-codec-registration/
• /TR/webcodecs-pcm-codec-registration/
• /TR/webcodecs-ulaw-codec-registration/
• /TR/webcodecs-vorbis-codec-registration/
• /TR/webcodecs-vp8-codec-registration/
• /TR/webcodecs-vp9-codec-registration/
• /TR/webcodecs/
• /TR/webrtc-encoded-transform/
• /TR/webrtc-priority/
• /TR/webrtc-svc/
• /TR/webxr-ar-module-1/
• /TR/webxr-depth-sensing-1/
• /TR/webxr-dom-overlays-1/
• /TR/webxr-gamepads-module-1/
• /TR/webxr-hand-input-1/
• /TR/webxr-hit-test-1/
• /TR/webxr-lighting-estimation-1/
• /TR/webxr/
• /TR/webxrlayers-1/
• /TR/WGSL/
• /TR/window-management/
• /TR/WOFF2/

[dontcallmedom]

thanks for collecting that data, that's already quite enlightening!

Just noted a couple of random bugs I spotted, in case they're the sign of another issue (in the data or in the analysis):

• it shows the AV1 spec as a (up-to-date) TR spec (which it isn't)
• it lists WebNN as 2 days out of date, which it doesn't seem to be (maybe linked to the different frequency in TR data collection?)

[tidoust]

I put the script I used in a gist. Note there are a couple of additional cases that I excluded (the CSS 2.1 spec and WOFF) as the dates in the crawl data are wrong (they're not even dates).

it shows the AV1 spec as a (up-to-date) TR spec (which it isn't)

Yes, the code was selecting specs based on the presence of release, which is also set for the AV1 spec. I adjusted the code to only consider W3C specs. That was the only spec with that issue.

it lists WebNN as 2 days out of date, which it doesn't seem to be (maybe linked to the different frequency in TR data collection?)

Soooo picky! ;) That's indeed due to the different frequency for data collection: the /TR version of WebNN was last crawled 12 hours ago, when it was still the 25 March 2024 CRD. The new version is still not in Webref. I guess that illustrates how easy it is to create wrong signals: there's always a slight delay between the publication of the ED and publication of the /TR version, and crawl introduces another delay. In practice, anything that is less than one week late can probably be considered up-to-date!

[dontcallmedom]

I think the conclusion is that auto-published is not sufficiently stable of a state to be usefully trackable and publishable in browser-specs, so closing this issue.