From d6b83114a8280b036efd447c9cca7d206000fa2b Mon Sep 17 00:00:00 2001 From: Jake Archibald Date: Thu, 17 May 2018 13:46:43 +0100 Subject: [PATCH 1/2] Allow Range header to be set by APIs This is part of #144. The aim is to allow APIs to use the range header for no-cors requests, and allow them to pass through a service worker, but disallow modification of these requests, and disallow developers creating their own no-cors ranged requests. Tests: https://github.com/w3c/web-platform-tests/pull/10348. --- fetch.bs | 316 ++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 232 insertions(+), 84 deletions(-) diff --git a/fetch.bs b/fetch.bs index daafe5d45..5468362aa 100644 --- a/fetch.bs +++ b/fetch.bs @@ -197,6 +197,12 @@ of: queue a fetch task on request to process request end-of-body for request. +

To serialize an integer, represent it as a string of the shortest possible decimal +number. + +

This will be replaced by a more descriptive algorithm in Infra. See +infra/201. +

URL

@@ -443,6 +449,24 @@ documented in CORS protocol exceptions.

A CORS non-wildcard request-header name is a byte-case-insensitive match for `Authorization`. +

A privileged no-cors request-header name is a header +name that is a byte-case-insensitive match for one of + +

+ +
+

These are headers that can be set by privileged APIs, and will be preserved if their associated + request object is copied, but will be removed if the request is modified by unprivilaged APIs. + +

`Range` headers are commonly used by downloads + and media fetches, although neither of these currently specify + how. html/2914 aims to solve this. + +

A helper is provided to add a range header to a particular request. +

+

A CORS-safelisted response-header name, given a CORS-exposed header-name list list, is a header name that is a byte-case-insensitive match for one of @@ -1308,6 +1332,33 @@ run these steps: +

+ +

To add a range header to a +request request, with an integer first, and an optional integer +last, run these steps: + +

    +

  1. Let rangeValue be `bytes `. + +

  2. Serialize and UTF-8 encode first, and + append the result to rangeValue. + +

  3. Append 0x2D (-) to rangeValue. + +

  4. If last is given, then serialize and + UTF-8 encode it, and append the result to rangeValue. + +

  5. Append `Range`/rangeValue to + request's header list. +

+ +

A range header denotes an inclusive byte range. There a range header where +first is 0 and last is 500, is a range of 501 bytes. + +

Features that combine multiple responses into one logical resource are historically a +source of security bugs. Please seek security review for features that deal with partial responses. +

Responses

@@ -1403,6 +1454,14 @@ specified. [[!CSP]] `Access-Control-Expose-Headers` header. This list is used by a CORS filtered response to determine which headers to expose. +

A response has an associated +range-requested flag, which is +initially unset. + +

This is used to ensure to prevent a partial response from an earlier ranged request +being provided to an API that didn't make a range request. See the flag's usage for a detailed +description of the attack. +

A response can have an associated location URL (null, failure, or a URL). Unless specified otherwise, response has no @@ -2929,6 +2988,31 @@ with a CORS flag and recursive flag, run these steps:

  • should internalResponse to request be blocked due to nosniff +
  • +

    If response's type is "opaque", + internalResponse's status is 206, + internalResponse's range-requested flag is set, and + request's header list does not contain + `Range`, then set response and internalResponse to a + network error. + +

    +

    Traditionally, APIs accept a ranged response even if a range was not requested. This prevents + a partial response from an earlier ranged request being provided to an API that did not make a + range request. + +

    + Further details + +

    The above steps prevent the following attack attack: + +

    A media element is used to request a range of a cross-origin HTML resource. Although this is + invalid media, a reference to a clone of the response can be retained in a service worker. This + can later be used as the response to a script element's fetch. If the partial response is valid + JavaScript (even though the whole resource is not), executing it would leak private data. +

    +
    +

  • If response is not a network error and either request's method is @@ -3471,7 +3555,8 @@ Range Requests. [[HTTP-RANGE]] However, this is not widely supported by b

  • If httpRequest's body is non-null and httpRequest's body's source is non-null, then set contentLengthValue to httpRequest's body's - total bytes, UTF-8 encoded. + total bytes, serialized and + UTF-8 encoded.

  • If contentLengthValue is non-null, append @@ -3751,6 +3836,9 @@ Range Requests. [[HTTP-RANGE]] However, this is not widely supported by b +

  • If httpRequest's header list contains + `Range`, then set response's range-requested flag. +

  • If response's status is 401, CORS flag is unset, credentials flag is set, and request's window is an @@ -4521,9 +4609,6 @@ objects. "response" or "none". -

    "immutable" exists for service workers. -[[!SW]] -

    To append a name/value (name/value) pair to a @@ -4542,9 +4627,8 @@ objects. "request" and name is a forbidden header name, return. -

  • Otherwise, if guard is - "request-no-cors" and name/value is not a - CORS-safelisted request-header, return. +

  • Otherwise, if guard is "request-no-cors" and + name/value is not a CORS-safelisted request-header, then return.

  • Otherwise, if guard is "response" and name is a @@ -4553,6 +4637,9 @@ objects.

  • Append name/value to header list. + +

  • If headers's guard is "request-no-cors", then + remove privileged no-cors request headers from headers.

    To fill a @@ -4577,6 +4664,22 @@ run these steps: key/value to headers. +

    To +remove privileged no-cors request headers +from a {{Headers}} object (headers), run these steps: + +

      +

    1. For each headerName of + privileged no-cors request-header names: + +

        +

      1. Delete headerName from headers's + header list. +

      +
    + +

    This is called when headers are modified by unprivileged code. +

    The Headers(init) constructor, when invoked, must run these steps: @@ -4609,9 +4712,10 @@ method, when invoked, must run these steps: "request" and name is a forbidden header name, return. -

  • Otherwise, if guard is - "request-no-cors" and name/`invalid` is - not a CORS-safelisted request-header, then return. +

  • +

    Otherwise, if guard is "request-no-cors", + name/`invalid` is not a CORS-safelisted request-header, and + name is not a privileged no-cors request-header name, then return.

    `invalid` is used because delete() is not passed a value as argument. @@ -4620,8 +4724,14 @@ method, when invoked, must run these steps: "response" and name is a forbidden response-header name, return. +

  • If header list does not contain name, + then return. +

  • Delete name from header list. + +

  • If guard is "request-no-cors", then + remove privileged no-cors request headers from the context object.

    The get(name) method, when @@ -4666,9 +4776,8 @@ method, when invoked, must run these steps: "request" and name is a forbidden header name, return. -

  • Otherwise, if guard is - "request-no-cors" and name/value is not a - CORS-safelisted request-header, return. +

  • Otherwise, if guard is "request-no-cors" and + name/value is not a CORS-safelisted request-header, then return.

  • Otherwise, if guard is "response" and name is a @@ -4677,6 +4786,9 @@ method, when invoked, must run these steps:

  • Set name/value in header list. + +

  • If guard is "request-no-cors", then + remove privileged no-cors request headers from the context object.

    The value pairs to iterate over are the return value of running @@ -5219,42 +5331,61 @@ constructor must run these steps:

  • If init's window member is present, set window to "no-window". -

  • Set request to a new request - whose url is request's - current url, - method is request's - method, - header list is a copy of - request's header list, - unsafe-request flag is set, - client is - current settings object, - window is window, - origin is "client", - referrer is request's - referrer, - referrer policy is - request's - referrer policy, - mode is request's - mode, - credentials mode is - request's - credentials mode, - cache mode is - request's - cache mode, - redirect mode is - request's - redirect mode, - integrity metadata is - request's - integrity metadata, - keepalive flag is request's keepalive flag, - reload-navigation flag is request's - reload-navigation flag, and - history-navigation flag is request's - history-navigation flag. +

  • +

    Set request to a new request with the following properties: + +

    +
    url +
    request's current url. + +
    method +
    request's method. + +
    header list +
    A copy of request's header list. + +
    unsafe-request flag +
    Set. + +
    client +
    Current settings object. + +
    window +
    window. + +
    origin +
    "client". + +
    referrer +
    request's referrer. + +
    referrer policy +
    request's referrer policy. + +
    mode +
    request's mode. + +
    credentials mode +
    request's credentials mode. + +
    cache mode +
    request's cache mode. + +
    redirect mode +
    request's redirect mode. + +
    integrity metadata +
    request's integrity metadata. + +
    keepalive flag +
    request's keepalive flag. + +
    reload-navigation flag +
    request's reload-navigation flag. + +
    history-navigation flag +
    request's history-navigation flag. +

  • If any of init's members are present, then: @@ -5271,14 +5402,13 @@ constructor must run these steps:

  • Set request's referrer to "client" -

  • Set request's - referrer policy to the empty string. +

  • Set request's referrer policy to the empty string. -

    This is done to ensure that when a service worker "redirects" a request, e.g., - from an image in a cross-origin stylesheet, and makes modifications, it no longer appears to come - from the original source (i.e., the cross-origin stylesheet), but instead from the service worker - that "redirected" the request. This is important as the original source might not even be able to +

    This is done to ensure that when a service worker "redirects" a request, e.g., from + an image in a cross-origin style sheet, and makes modifications, it no longer appears to come from + the original source (i.e., the cross-origin style sheet), but instead from the service worker that + "redirected" the request. This is important as the original source might not even be able to generate the same kind of requests as the service worker. Services that trust the original source could therefore be exploited were this not done, although that is somewhat farfetched. @@ -5381,41 +5511,51 @@ constructor must run these steps:

  • Let r be a new {{Request}} object associated with request. +

  • If signal is not null, then make r's signal + follow signal. +

  • Set r's headers to a new {{Headers}} object, whose header list is request's header list, and guard is "request". -

  • If signal is not null, then make r's signal - follow signal. +

  • +

    If any of init's members are present, then: -

  • Let headers be a copy of r's headers and its - associated header list. - +

    The headers are sanitised as they might contain headers that are not allowed by this + mode. Otherwise, they were previously sanitised or are unmodified since creation by a privileged + API. -

  • If init's headers member is present, set - headers to init's headers member. +

      +

    1. Let headers be a copy of r's headers and its + associated header list. -

    2. Empty r's headers's header list. +

    3. If init's headers member is present, set + headers to init's headers member. -

    4. -

      If r's request's mode is - "no-cors", then: +

    5. Empty r's headers's header list. -

        -

      1. If r's request's method is not a - CORS-safelisted method, then throw a TypeError. +

      2. +

        If r's request's mode is + "no-cors", then: -

      3. Set r's headers's guard to - "request-no-cors". -

      +
        +

      1. If r's request's method is not a + CORS-safelisted method, then throw a TypeError. + +

      2. Set r's headers's guard to + "request-no-cors". +

      +
      6. -

    6. If headers is a Headers object, then for each - header in its header list, append - header's name/header's value to - r's {{Headers}} object. +

    7. If headers is a Headers object, then for each + header in its header list, append + header's name/header's value to + r's {{Headers}} object. -

    8. Otherwise, fill r's {{Headers}} object with - headers. +

    9. Otherwise, fill r's {{Headers}} object with + headers. +

    +

  • Let inputBody be input's request's body @@ -5571,10 +5711,18 @@ run these steps:

  • Set clonedRequestObject's request to clonedRequest. -

  • Set clonedRequestObject's headers to a new {{Headers}} object - whose header list is set to clonedRequest's - header list, and guard is context object's - headers' guard. +

  • +

    Set clonedRequestObject's headers to a new {{Headers}} object + with the following properties: + +

    +
    header list +
    clonedRequest's header list. + +
    guard +
    context object's headers's guard. +
    +

  • Make clonedRequestObject's signal follow context object's signal. @@ -5797,8 +5945,8 @@ run these steps:

  • Set clonedResponseObject's headers to a new {{Headers}} object whose header list is set to clonedResponse's - header list, and guard is context object's - headers' guard. + header list, and guard is the context object's + headers's guard.

  • Upon fulfillment of context object's trailer promise, resolve clonedResponseObject's trailer promise with a new {{Headers}} object From e1ad29dc58e8823c72162ab5923f0df2d1ac45fa Mon Sep 17 00:00:00 2001 From: Anne van Kesteren Date: Mon, 28 May 2018 13:57:11 +0200 Subject: [PATCH 2/2] use isomorphic encode for new/changed things --- fetch.bs | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fetch.bs b/fetch.bs index 5468362aa..95b5860f2 100644 --- a/fetch.bs +++ b/fetch.bs @@ -1341,13 +1341,13 @@ run these steps:

    1. Let rangeValue be `bytes `. -

    2. Serialize and UTF-8 encode first, and - append the result to rangeValue. +

    3. Serialize and isomorphic encode first, + and append the result to rangeValue.

    4. Append 0x2D (-) to rangeValue.

    5. If last is given, then serialize and - UTF-8 encode it, and append the result to rangeValue. + isomorphic encode it, and append the result to rangeValue.

    6. Append `Range`/rangeValue to request's header list. @@ -3556,7 +3556,7 @@ Range Requests. [[HTTP-RANGE]] However, this is not widely supported by b body's source is non-null, then set contentLengthValue to httpRequest's body's total bytes, serialized and - UTF-8 encoded. + isomorphic encoded.

    7. If contentLengthValue is non-null, append