[explainer] Reorganize explainer

- Added back explainer for PendingBeacon API from WICG#69 for future reference. - Simplified the contents in `README.md`. API details are moved to fetch-with-pending-request-api.md. - Added empty fetch-later-api.md. Subsequent changes will fill in details according to the PR in fetch repository. - Added a Github Action to verify links in markdown files.
mingyc · Jun 30, 2023 · d12d4d6 · d12d4d6
1 parent d5189eb
commit d12d4d6
Show file tree

Hide file tree

Showing 8 changed files with 726 additions and 385 deletions.
diff --git a/.github/workflows/linter.yml b/.github/workflows/linter.yml
@@ -7,9 +7,9 @@ name: Super Linter
 
 on:
   push:
-    branches: [ "main" ]
+    branches: [main]
   pull_request:
-    branches: [ "main" ]
+    branches: [main]
 
 jobs:
   run-lint:

diff --git a/.github/workflows/md-lint.yml b/.github/workflows/md-lint.yml
@@ -0,0 +1,21 @@
+name: Check Markdown links
+
+#
+# Documentation:
+# https://github.com/gaurav-nelson/github-action-markdown-link-check#how-to-use
+#
+
+on:
+  push:
+    branches: [main]
+    paths: ["**.md"]
+  pull_request:
+    branches: [main]
+    paths: ["**.md"]
+
+jobs:
+  markdown-link-check:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - uses: gaurav-nelson/github-action-markdown-link-check@v1
diff --git a/README.md b/README.md
diff --git a/docs/alternative-approaches.md b/docs/alternative-approaches.md
@@ -0,0 +1,169 @@
+# Alternative Approaches
+
+*This document maintains the list of alternatives that have been considered to achieve better beaconing.*
+*Note that only [PendingBeacon API](#pendingbeacon-api) has ever been implemented.*
+
+## fetch() with PendingRequest API
+
+See the [fetch() with PendingRequest API explainer](fetch-with-pending-request-api.md).
+
+## PendingBeacon API
+
+See the [PendingBeacon API explainer](pending-beacon-api.md).
+
+## DOM-Based API
+
+A DOM-based API was considered as an alternative.
+This API would consist of a new possible `beacon` value for the `rel` attribute
+on the link tag, which developers could use to indicate a beacon,
+and then use standard DOM manipulation calls to change the data, cancel the beacon, etc.
+
+The stateful JS API was preferred to avoid beacon concerns intruding into the DOM,
+and because a ‘DOM-based’ API would still require scripting in many cases anyway
+(populating beacon data as the user interacts with the page, for example).
+
+## BFCache-supported `unload`-like event
+
+Another alternative is to introduce (yet) another page lifecycle event,
+that would be essentially the `unload` event, but supported by the BFCache -
+that is, its presence would not disable the BFCache, and the browser would execute this callback even on eviction from the BFCache.
+This was rejected because it would require allowing pages frozen in the BFCache to execute a JavaScript callback,
+and it would not be possible to restrict what that callback does
+(so, a callback could do things other than sending a beacon, which is not safe).
+It also doesn’t allow for other niceties such as resilience against crashes or batching of beacons,
+and complicates the already sufficiently complicated page lifecycle.
+
+## Extending `fetch()` API
+
+> **NOTE:** Discussions in [#52] and [#50].
+
+Another alternative is to extend the [Fetch API] to support the [requirements](../README.md#requirements).
+
+The existing Fetch with `keepalive` option, combined with `visibilitychagne` listener, can approximate part of (1):
+
+```js
+document.addEventListener('visibilitychange', () => {
+  if (document.visibilityState === 'hidden') {
+    fetch('/send_beacon', {keepalive: true});
+    // response may be dropped.
+  }
+});
+```
+
+or a new option `deferSend` may be introduced to cover the entire (1):
+
+```js
+// defer request sending on `hidden` or bfcahce eviction etc.
+fetch('/send_beacon', {deferSend: true});
+// Promise may not resolve and response may be dropped.
+```
+
+### Problem
+
+However, there are several problem with this approach:
+
+1. **The Fetch API shape is not designed for this (1) purpose.** Fundamentally, `window.fetch` returns a Promise with Response to resolve, which don't make sense for beaconing at page discard that doesn't expect to process response.
+2. **The (1) mechanism is too unrelated to be added to the Fetch API**. Even just with a new option, bundling it with a visibility event-specific behavior just seems wrong in terms of the API's scope.
+3. **The Fetch API does not support updating request URL or data.** This is simply not possible with its API shape. Users have to re-fetch if any update happens.
+
+The above problems suggest that a new API is neccessary for our purpose.
+
+## Extending `navigator.sendBeacon()` API
+
+> **NOTE:** Discussions in [WebKit's standard position](https://github.com/WebKit/standards-positions/issues/85#issuecomment-1418381239).
+
+Another alternative is to extend the [`navigator.sendBeacon`] API:
+
+```ts
+navigator.sendBeacon(url): bool
+navigator.sendBeacon(url, data): bool
+```
+
+To meet the [requirements](../README.md#requirements) and to make the new API backward compatible, we propose the following shape:
+
+```ts
+navigator.sendBeacon(url, data, fetchOptions): PendingBeacon
+```
+
+An optional dictionary argument `fetchOptions` can be passed in, which changes the return value from `bool` to `PendingBeacon` proposed in the [above section](#pendingbeacon-api). Some details to note:
+
+1. The proposal would like to support both `POST` and `GET` requests. As the existing API only support `POST` beacons, passing in `fetchOptions` with `method: GET` should enable queuing `GET` beacons.
+2. `fetchOptions` can only be a subset of the [Fetch API]'s [`RequestInit`] object:
+   1. `method`: one of `GET` or `POST`.
+   2. `headers`: supports custom headers, which unblocks [#50].
+   3. `body`: **not supported**. POST body should be in `data` argument.
+   4. `credentials`: enforcing `same-origin` to be consistent.
+   5. `cache`: not supported.
+   6. `redirect`: enforcing `follow`.
+   7. `referrer`: enforcing same-origin URL.
+   8. `referrerPolicy`: enforcing `same-origin`.
+   9. `keepalive`: enforcing `true`.
+   10. `integrity`: not supported.
+   11. `signal`: **not supported**.
+       * The reason why `signal` and `AbortController` are not desired is that we needs more than just aborting the requests. It is essential to check a beacon's pending states and to update or accumulate data. Supporting these requirements via the returned `PendingBeacon` object allows more flexibility.
+   12. `priority`: enforcing `auto`.
+3. `data`: For `GET` beacon, it must be `null` or `undefined`.
+4. The return value must supports updating request URL or data, hence `PendingBeacon` object.
+
+### Problem
+
+* The above API itself is enough for the [requirements](../README.md#requirements) (2) and (3), but cannot achieve the requirement (1), delaying the request.
+* The function name `sendBeacon` semantic doesn't make sense for the "delaying" behavior.
+* Combing the subset of `fetchOptions` along with the existing `data` parameter are error-proning.
+
+## Introducing `navigator.queueBeacon()` API
+
+To imprvoe from "Extending `navigator.sendBeacon()` API, it's better with a new function:
+
+```ts
+navigator.queueBeacon(url, fetchOptions, beaconOptions): PendingBeacon
+```
+
+This proposal gets rid of the `data` parameter, and request body should be put into `fetchOptions.body` directly.
+
+The extra `beaconOptions` is a dictionary taking `backgroundTimeout` and `timeout` to support the optional timeout after bfcache or hidden requirement.
+
+At the end, this proposal also requires an entirely new API, just under the existing `navigator` namespace. The advantage is that we might be able to merge this proposal into [w3c/beacon] and eliminate the burden to maintain a new spec.
+
+
+## Write-only API
+
+This is similar to the [proposed PendingBeacon API](#pendingbeacon-api) but there is no `pending` and no `setData()`.
+There are 2 classes of beacon with a base class that has
+
+* `url`
+* `method`
+* `sendNow()`
+* `deactivate()`
+* API for specifying timeouts
+
+With these APIs, the page cannot check whether the beacon has been sent already.
+
+It's unclear that these APIs can satisfy all use cases.
+If they can, they have the advantage of being easier to implement
+and simple to use.
+
+## High-Level APIs
+
+### AppendableBeacon
+
+Has `appendData(data)` which appends new data to the beacon's payload.
+The beacon will flush queued payload according to the timeouts and the browser state.
+
+The use-case is for continuously logging events that are accumulated on the server-side.
+
+### ReplaceableBeacon
+
+Has `replaceData(data)` which replaces the current the beacon's payload.
+The beacon will send the payload according to the timeouts and the browser state.
+If a payload has been sent already, replaceData simply stores a new payload to be sent in the future.
+
+The use case is for logging a total-so-far.
+The server would typically only pay attention to the latest value.
+
+
+[#50]: https://github.com/WICG/pending-beacon/issues/50
+[#52]: https://github.com/WICG/pending-beacon/issues/52
+[Fetch API]: https://fetch.spec.whatwg.org/#fetch-api
+[`RequestInit`]: https://fetch.spec.whatwg.org/#requestinit
+[w3c/beacon]: https://github.com/w3c/beacon
diff --git a/docs/fetch-later-api.md b/docs/fetch-later-api.md
@@ -0,0 +1,4 @@
+# fetchLater() API
+
+
+TODO
diff --git a/docs/fetch-with-pending-request-api.md b/docs/fetch-with-pending-request-api.md
@@ -0,0 +1,188 @@
+# fetch() with PendingRequest API (deprecated)
+
+**WARNING: This API is being replaced with [`fetchLater()`](fetch-later-api.md), a Fetch-based approach.**
+
+--
+
+*This document is an explainer for PendingRequest API.*
+*It is proposed in response to a series of [discussions and concerns][concerns] around the experimental [PendingBeacon API](pending-beacon-api.md).*
+*There many [open discussions](#open-discussions) within the explainers, which leads to the fetchLater() API.*
+
+*Note that this proposal is NEVER implemented.*
+
+## Design
+
+> **NOTE:** Discussions in [#70], [#52] and [#50].
+
+The basic idea is to extend the [Fetch API] by adding a new stateful option:
+Rather than a developer manually calling `fetch(url, {keepalive: true})` within a `visibilitychange` event listener, the developer registers that they would like to send a pending request, i.e. a beacon, for this page when it gets discarded.
+The developer can then call signal controller registered on this request to updates based on its state or abort.
+
+Then, at some point later after the user leaves the page, the browser will send the request.
+From the point of view of the developer the exact send time is unknown. On successful sending, the whole response will be ignored, including body and headers. Nothing at all should be processed or updated, as the page is already gone.
+
+## JavaScript API
+
+The following new fetch options are introduced into [`RequestInit`]:
+
+* `deferSend`: A `DeferSend` object. If set, the browser should defer the request sending until page discard or bfcache eviction.
+  Underlying implementation should ensure the request is kept alive until suceeds or fails.
+  Hence it cannot work with `keepalive: false`. The object may optionally set the following field:
+  * `sendAfterBeingBackgroundedTimeout`: Specifies a timeout in seconds for a timer that only starts after the page enters the next `hidden` visibility state.
+    Default to `-1`.
+* `sentSignal`: A `SentSignal` object to allow user to listen to the `sent` event of the request when it gets sent.
+
+
+## Examples
+
+### Defer a `GET` request until page discard
+
+```js
+fetch('/send_beacon', {deferSend: new DeferSend()}).then(res => {
+  // Promise may never be resolved and response may be dropped.
+})
+```
+
+### Defer a request until next `hidden` + 1 minute
+
+```js
+fetch('/send_beacon', {
+  deferSend: new DeferSend(sendAfterBeingBackgroundedTimeout: 60)
+  }).then(res => {
+  // Possibly resolved after next `hidden` + 1 minute.
+  // But this may still not be resolved if page is already in bfcache.
+})
+```
+
+### Update a pending request
+
+```js
+let abort = null;
+let pending = true;
+
+function createBeacon(data) {
+  pending = true;
+  abort = new AbortController();
+  let sentSignal = new SentSignal();
+  fetch(data, {
+    deferSend: new DeferSend(),
+    signal: abortController.signal,
+    sentSignal: sentsentSignal
+  });
+
+  sentSignal.addEventListener("sent", () => {
+    pending = false;
+  });
+}
+
+function updateBeacon(data) {
+  if (pending) {
+    abort.abort();
+  }
+  createBeacon(data);
+}
+```
+
+## Open Discussions
+
+### 1. Limiting the scope of pending requests
+
+> **NOTE:** Discussions in [#72].
+
+Even if moving toward a fetch-based design, this proposal does still not focus on supporting every type of requests as beacons.
+
+For example, it's non-goal to support most of [HTTP methods], i.e. being able to defer an `OPTION` or `TRACE`.
+We should look into [`RequestInit`] and decide whether `deferSend` should throw errors on some of their values:
+
+* `keepalive`: must be `true`. `{deferSend: new DeferSend(), keepalive: false}` conflicts with each other.
+* `url`: supported.
+* `method`: one of `GET` or `POST`.
+* `headers`: supported.
+* `body`: only supported for `POST`.
+* `signal`: supported.
+* `credentials`: enforcing `same-origin` to be consistent.
+* `cache`: not supported?
+* `redirect`: enforcing `follow`?
+* `referrer`: enforcing same-origin URL?
+* `referrerPolicy`: enforcing `same-origin`?
+* `integrity`: not supported?
+* `priority`: enforcing `auto`?
+
+As shown above, at least `keepalive: true` and `method` need to be enforced.
+If going with this route, can we also consider the [PendingRequest API] approach that proposes a subclass of `Request` to enforce the above?
+
+### 2. `sendAfterBeingBackgroundedTimeout` and `deferSend`
+
+> **NOTE:** Discussions in [#73], [#13].
+
+```js
+class DeferSend {
+  constructor(sendAfterBeingBackgroundedTimeout)
+}
+```
+
+Current proposal is to make `deferSend` a class, and `sendAfterBeingBackgroundedTimeout` its optional field.
+
+1. Should this be a standalone option in [`RequestInit`]? But it is not relevant to other existing fetch options.
+2. Should it be after `hidden` or `pagehide` (bfcached)? (Previous discussion in #13).
+3. Need user input for how desirable for this option.
+4. Need better naming suggestion.
+
+### 3. Promise
+
+> **NOTE:** Discussions in [#74].
+
+To maintain the same semantic, browser should resolve Promise when the pending request is sent. But in reality, the Promise may or may not be resolved, or resolved when the page is in bfcache and JS context is frozen. User should not rely on it.
+
+### 4. `SendSignal`
+
+> **NOTE:** Discussions in [#75].
+
+This is to observe a event to tell if a `deferSend` request is still pending.
+
+To prevent from data races, the underlying implementation should ensure that renderer is authoritative to the request's send state when it's alive. Similar to [this discussion](https://github.com/WICG/pending-beacon/issues/10#issuecomment-1189804245) for PendingBeacon.
+
+### 5. Handling Request Size Limit
+
+> **NOTE:** Discussions in [#76].
+
+As setting `deferSend` implies `keepalive` is also true, such request has to share the same size limit budget as a regular keepalive request’s [one][fetch-keepalive-quota]: "for each fetch group, the sum of contentLength and inflightKeepaliveBytes <= 64 KB".
+
+To comply with the limit, there are several options:
+
+1. `fetch()` throws `TypeError` whenever the budget has exceeded. Users will not be able to create new pending requests.
+2. The browser forces sending out other existing pending requests, in FIFO order, when the budget has exceeded. For a single request > 64KB, `fetch()` should still throws `TypeError`.
+3. Ignore the size limit if [BackgroundFetch] Permission is enabled for the page.
+
+
+### 6. Permissions Policy
+
+> **NOTE:** Discussions in [#77].
+
+Given that most reporting API providers are crossed origins, we propose to allow this feature by default for 3rd-party iframes.
+User should be able to opt out the feature with the corresponding Permissions Policy.
+
+
+## Other Documentation
+
+* [Initial PendingRequest API Proposal](https://docs.google.com/document/d/1QQFFa6fZR4LUiyNe9BJQNK7dAU36zlKmwf5gcBh_n2c/edit#heading=h.powlqxc01y5b)
+* [Presntation at WebPerf WG 2023/03/16](https://docs.google.com/presentation/d/1w_v2kn4RxDmGQ76HAHbuWpYMPj7XsHkYOILIkLs9ppY/edit#slide=id.pZ)
+
+
+[concerns]: https://github.com/WICG/pending-beacon/issues/70
+[#13]: https://github.com/WICG/pending-beacon/issues/13
+[#50]: https://github.com/WICG/pending-beacon/issues/50
+[#52]: https://github.com/WICG/pending-beacon/issues/52
+[#70]: https://github.com/WICG/pending-beacon/issues/70
+[#72]: https://github.com/WICG/pending-beacon/issues/72
+[#73]: https://github.com/WICG/pending-beacon/issues/73
+[#74]: https://github.com/WICG/pending-beacon/issues/74
+[#75]: https://github.com/WICG/pending-beacon/issues/75
+[#76]: https://github.com/WICG/pending-beacon/issues/76
+[#77]: https://github.com/WICG/pending-beacon/issues/77
+[Fetch API]: https://fetch.spec.whatwg.org/#fetch-api
+[`RequestInit`]: https://fetch.spec.whatwg.org/#requestinit
+[fetch-keepalive-quota]: https://fetch.spec.whatwg.org/#http-network-or-cache-fetch
+[BackgroundFetch]: https://developer.mozilla.org/en-US/docs/Web/API/Background_Fetch_API#browser_compatibility
+[HTTP methods]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods
+[PendingRequest API]: https://docs.google.com/document/d/1QQFFa6fZR4LUiyNe9BJQNK7dAU36zlKmwf5gcBh_n2c/edit#heading=h.xs53e9immw2r