index.bs

<pre class='metadata'>
Title: Storage Buckets API
Shortname: storage-buckets
Level: 1
Status: CG-DRAFT
Group: WICG
Repository: WICG/storage-buckets
URL: https://wicg.github.io/storage-buckets/
Editor: Evan Stade, Google https://www.google.com/, estade@google.com
Editor: Ayu Ishii, Google https://www.google.com/, ayui@google.com
Former Editor: Victor Costan
!Participate: <a href="https://github.com/WICG/storage-buckets">GitHub WICG/storage-buckets</a> (<a href="https://github.com/WICG/storage-buckets/issues/new">new issue</a>, <a href="https://github.com/WICG/storage-buckets/issues?state=open">open issues</a>)
Abstract: The Storage Buckets API provides a way for sites to organize locally stored data into groupings called "storage buckets". This allows the user agent or sites to manage and delete buckets independently rather than applying the same treatment to all the data from a single origin.
Markup Shorthands: css no, markdown yes
</pre>

<pre class=anchors>
spec: storage; urlPrefix: https://storage.spec.whatwg.org/
    type: dfn
        text: bottle map; url: bottle-map
        text: bucket map; url: bucket-map
        text: bucket mode; url: bucket-mode
        text: obtain a local storage shelf; url: obtain-a-local-storage-shelf
        text: queue a storage task; url: queue-a-storage-task
        text: storage bottle; url: storage-bottle
        text: storage bottle map; url: storage-bottle-map
        text: storage bucket; url: storage-bucket
        text: storage key; url: storage-key
        text: storage quota; url: storage-quota
        text: storage shelf; url: storage-shelf
spec: html; urlPrefix: https://html.spec.whatwg.org/multipage/
    type: dfn
        text: parse a duration string; url: common-microsyntaxes.html#parse-a-duration-string
spec: infra; urlPrefix: https://infra.spec.whatwg.org/
    type: dfn
        text: code point substring by positions; url: code-point-substring-by-positions
</pre>

<h2 id="storage-bucket-manager">The {{StorageBucketManager}} interface</h2>

<xmp class="idl">
[SecureContext]
interface mixin NavigatorStorageBuckets {
  [SameObject] readonly attribute StorageBucketManager storageBuckets;
};
Navigator includes NavigatorStorageBuckets;
WorkerNavigator includes NavigatorStorageBuckets;
</xmp>

Each [=environment settings object=] has an associated {{StorageBucketManager}} object.

The <dfn attribute for=NavigatorStorageBuckets><code>storageBuckets</code></dfn>
getter steps are to return [=this=]'s [=/relevant settings object=]'s {{StorageBucketManager}} object.

A user agent has an associated <dfn for=StorageBucketManager><code>storage bucket manager</code></dfn> which is the result of [=starting a new parallel queue=].

<xmp class="idl">
[Exposed=(Window,Worker),
 SecureContext]
interface StorageBucketManager {
    Promise<StorageBucket> open(DOMString name, optional StorageBucketOptions options = {});
    Promise<sequence<DOMString>> keys();
    Promise<undefined> delete(DOMString name);
};

dictionary StorageBucketOptions {
  boolean persisted = false;
  unsigned long long quota;
  DOMHighResTimeStamp expires;
};
</xmp>

<h3 id="storage-bucket-open">Creating a bucket</h3>

<div algorithm>

The <dfn method for="StorageBucketManager">open(|name|, |options|)</dfn> method steps are:

1. Let |environment| be [=/this=]'s [=/relevant settings object=].

1. Let |shelf| be the result of running [=obtain a local storage shelf=] given |environment|.

1. If |shelf| is failure, then return [=a promise rejected with=] a {{TypeError}}.

1. If the result of [=validate a bucket name=] with |name| is failure,  then return [=a promise rejected with=] a {{TypeError}}.

1. Let |p| be [=a new promise=].

1. [=Enqueue the following steps=] to [=StorageBucketManager/storage bucket manager=]:

    1. Let |r| be the result of running [=open a bucket=] with |shelf|, |name|, and |options|.

    1. If |r| is failure, then [=queue a storage task=] to [=/reject=] |p| with a {{TypeError}}.

    1. Otherwise, [=queue a storage task=] to [=/resolve=] |p| with |r|.

1. Return |p|.

</div>

<div algorithm>

To <dfn>open a bucket</dfn> for a |shelf| given a bucket |name| and optional |options|, run the following steps:

1. Let |expires| be undefined.

1. If |options|["{{StorageBucketOptions/expires}}"] exists, then:

    1. Set |expires| to |options|["{{StorageBucketOptions/expires}}"].

    1. If |expires| milliseconds after the [=Unix epoch=] is before the [=relevant settings object=]'s [=environment settings object/current wall time=], then return failure.

1. Let |quota| be undefined.

1. If |options|["{{StorageBucketOptions/quota}}"] exists, then:

    1. Set |quota| to |options|["{{StorageBucketOptions/quota}}"].

    1. If |quota| is less than or equal to zero, then return failure.

1. Let |persisted| be false.

1. If |options|["{{StorageBucketOptions/persisted}}"] is true, then:

    1. Let |permission| be the result of [=/requesting permission to use=] `"persistent-storage"`.

    1. If |permission| is "{{PermissionState/granted}}", then set |persisted| to true.

1. Let |bucket| be the result of running [=get or expire a bucket=] with |shelf| and |name|.

1. If |bucket| is null, then:

    1. Set |bucket| to a new [=/storage bucket=] with name |name|.

    1. Set |bucket|'s [=StorageBucket/quota value=] to |quota|.

    1. Set |shelf|'s [=bucket map=][|name|] to |bucket|.

1. If |persisted| is true, set |bucket|'s [=/bucket mode=] to `"persistent"`.

1. Set |bucket|'s [=StorageBucket/expiration time|expiration=] to |expires| milliseconds after the [=Unix epoch=].

1. Let |storageBucket| be a new {{StorageBucket}}.

1. Set |storageBucket|'s [=/storage bucket=] to |bucket|.

1. Return |storageBucket|.

</div>

<div algorithm>

To <dfn>validate a bucket name</dfn> given string |name|, run the following steps:

1. If |name| contains any [=code point=] that is not [=ASCII lower alpha=], [=ASCII digit=], U+005F (_), or U+002D(-), then return failure.

1. If |name| [=string/code point length=] is 0 or exceeds 64, then return failure.

1. If |name| begins with U+005F (_) or U+002D(-), then return failure.

1. Return.

</div>

To <dfn>get or expire a bucket</dfn> on a |shelf| given string |name|, run the following steps:

1. Let |bucket| be |shelf|'s [=bucket map=][|name|] if exists. Otherwise return null.

1. If |bucket|'s [=StorageBucket/expiration time=] is non-null and before the [=relevant settings object=]'s [=environment settings object/current wall time=], then:

    1. Set |bucket|'s [=storage bucket/removed=] to true.
    
    1. Return null.

1. Return |bucket|.

</div>

<h3 id="storage-bucket-delete">Deleting a bucket</h3>

<div algorithm>

The <dfn method for="StorageBucketManager">delete(|name|)</dfn> method steps are:

1. Let |environment| be [=/this=]'s [=/relevant settings object=].

1. Let |shelf| be the result of running [=obtain a local storage shelf=] given |environment|.

1. If |shelf| is failure, then return [=a promise rejected with=] a {{TypeError}}.

1. Let |p| be [=a new promise=].

1. If the result of [=validate a bucket name=] with |name| is failure, then [=/reject=] |p| with an {{InvalidCharacterError}}.

1. Otherwise, [=enqueue the following steps=] to [=StorageBucketManager/storage bucket manager=]:

    1. Run [=remove a bucket=] with |shelf| and |name|.
    
    1. [=Queue a storage task=] to [=/resolve=] |p|.

1. Return |p|.

</div>
    
<div algorithm>

To <dfn>remove a bucket</dfn> on a |shelf| given a bucket |name|, run the following steps:

1. Let |bucket| be |shelf|’s [=bucket map=][|name|] if exists. Otherwise return.

1. Remove [=map/key=] |name| in |shelf|'s [=bucket map=].

1. Set |bucket|'s [=storage bucket/removed=] to true.

1. Return.

<aside class="note">

Specific storage endpoints may need to run additional actions to remove data when a storage bucket is set as removed.

</aside>

Issue: [[IndexedDB-3]] needs to define how deletion occurs when data is evicted by quota.

Issue: [[FS]] needs to define how deletion occurs for Bucket File System when data is evicted by quota.

Issue: [[Service-Workers]] needs to define how deletion occurs for CacheStorage and Service Workers when data is evicted by quota.

</div>

<h3 id="storage-bucket-keys">Enumerating buckets</h3>

<div algorithm>

The <dfn method for="StorageBucketManager">keys()</dfn> method steps are:

1. Let |shelf| be the result of running [=obtain a local storage shelf=].

1. If |shelf| is failure, then return [=a promise rejected with=] a {{TypeError}}.

1. Let |p| be [=a new promise=].

1. Let |keys| be a new [=/list=].

1. [=Enqueue the following steps=] to [=StorageBucketManager/storage bucket manager=]:

    1. For each |key| in |shelf|'s [=bucket map=], run the following steps:

        1. Let |bucket| be the result of running [=get or expire a bucket=] with |shelf| and |key|. 

        1. If |bucket| is non-null, [=list/append=] |key| to |keys|.

    1. [=Queue a storage task=] to [=/resolve=] |p| with |keys|.

1. Return |p|.

</div>

<h2 id="storage-bucket">The {{StorageBucket}} interface</h2>

<xmp class="idl">
[Exposed=(Window,Worker),
 SecureContext]
interface StorageBucket {
  readonly attribute DOMString name;

  [Exposed=Window] Promise<boolean> persist();
  Promise<boolean> persisted();

  Promise<StorageEstimate> estimate();

  Promise<undefined> setExpires(DOMHighResTimeStamp expires);
  Promise<DOMHighResTimeStamp?> expires();

  [SameObject] readonly attribute IDBFactory indexedDB;

  [SameObject] readonly attribute CacheStorage caches;

  Promise<FileSystemDirectoryHandle> getDirectory();
};
</xmp>

A {{StorageBucket}} has an associated [=/storage bucket=].

A [=/storage bucket=] has an associated <dfn for="storage bucket">removed</dfn> flag, which is a boolean, initially false. Set as true when a [=/storage bucket=] is deleted.

A {{StorageBucket}} has a {{DOMString}} object <dfn attribute for=StorageBucket>name</dfn> which is the key in the [=bucket map=] that maps to the [=/storage bucket=].

<aside class="note">

Browsers will probably have IPC and disk accesses that make operations asynchronous. Promises are resolved from tasks to permit this kind of work.

</aside>

<h3 id="storage-bucket-persistence">Persistence</h3>

Issue: Merge with [[Storage#buckets]] which already defines [=bucket mode=].

<div algorithm>

The <dfn method for="StorageBucket">persist()</dfn> method steps are:

1. Let |bucket| be [=this=]'s [=/storage bucket=].

1. Let |environment| be [=/this=]'s [=/relevant settings object=].

1. Let |p| be [=a new promise=].

1. Run the following steps [=in parallel=]:

    1. If |bucket|'s [=storage bucket/removed=] flag is true, then [=queue a storage task=] to [=reject=] |p| with an {{InvalidStateError}}.

    1. Let |persisted| be true if |bucket|'s [=bucket mode=] is `"persistent"`.

    1. Otherwise,

        1. Let |permission| be the result of [=getting the current permission state=] with `"persistent-storage"` and |environment|.

        1. If |permission| is "{{PermissionState/granted}}", then set |bucket|'s [=bucket mode=] to `"persistent"` and set |persisted| to true.

        1. Otherwise, set |persisted| to false.

    1. [=Queue a storage task=] to [=resolve=] |p| with |persisted|.

1. Return |p|.

</div>

<div algorithm>

The <dfn method for="StorageBucket">persisted()</dfn> method steps are:

1. Let |p| be [=a new promise=].

1. Let |bucket| be [=this=]'s [=/storage bucket=].

1. Otherwise, run these steps [=in parallel=]:

    1. If |bucket|'s [=storage bucket/removed=] flag is true, then [=queue a storage task=] to [=reject=] |p| with an {{InvalidStateError}}.

    1. Let |persistent| be true if |bucket|'s [=bucket mode=] is `"persistent"`, otherwise false.

    1. [=Queue a storage task=] to [=resolve=] |p| with |persistent|.

1. Return |p|.

</div>

<h3 id="storage-bucket-quota">Quota</h3>

A [=/storage bucket=] has a <dfn for=StorageBucket>quota value</dfn>, a number-or-null, initially null.
Specifies the upper limit of usage in bytes which can be used by the bucket. The user agent MAY further
limit the realized storage space.

The <dfn for="storage bucket">storage usage</dfn> of a [=/storage bucket=] is an [=implementation-defined=] rough estimate
of the number of bytes used by all of its [=/storage bottle=]s.

<div algorithm>

The <dfn method for="StorageBucket">estimate()</dfn> method steps are:

1. Let |environment| be [=/this=]'s [=/relevant settings object=].

1. Let |shelf| be the result of running [=obtain a local storage shelf=] with |environment|.

1. If |shelf| is failure, then return [=a promise rejected with=] a {{TypeError}}.

1. Let |bucket| be [=this=]'s [=/storage bucket=].

1. If |bucket|'s [=storage bucket/removed=] flag is true, then return [=a promise rejected with=] an {{InvalidStateError}}.

1. Let |p| be [=a new promise=].

1. Otherwise, run the following steps [=in parallel=]:

    1. Let |quota| be [=/storage quota=] for |shelf|.

    1. Set |quota| to |bucket|'s [=StorageBucket/quota value=] if it is non-null.

    1. Let |usage| be [=storage bucket/storage usage=] for |bucket|.

    1. Let |dictionary| be a new {{StorageEstimate}} dictionary whose {{StorageEstimate/usage}} member is |usage| and {{StorageEstimate/quota}} member is |quota|.

    1. [=Queue a storage task=] to [=resolve=] |p| with |dictionary|.

1. Return |p|.

</div>

<aside class="note">

The [=StorageBucket/quota value=] will be ignored if it exceeds the total amount of space available to the [=/storage shelf=], i.e. site.
Its intended use is to keep a specific bucket from using up the entire site's storage space.

</aside>

<h3 id="storage-bucket-expiration">Expiration</h3>

A [=/storage bucket=] has an <dfn for="StorageBucket">expiration time</dfn>, which is either null or a [=moment=] on the [=wall clock=], initially null.
Specifies the upper limit of a bucket lifetime.

The [=get or expire a bucket=] algorithm removes expired buckets when {{StorageBucketManager/keys()}} or {{StorageBucketManager/open()}} is called.
User agents MAY clear buckets whose [=/bucket mode=] is `"best-effort"` before their
[=StorageBucket/expiration time=] when faced with storage pressure.
User agents MAY remove any buckets before {{StorageBucketManager/open()}} or {{StorageBucketManager/keys()}} is called when the expiration is reached regardless of the [=/bucket mode=] 

<div algorithm>

The <dfn method for="StorageBucket">setExpires(|expires|)</dfn> method steps are:

1. Let |p| be [=a new promise=].

1. Let |bucket| be [=this=]'s [=/storage bucket=].

1. Otherwise, run these steps [=in parallel=]:

    1. If |bucket|'s [=storage bucket/removed=] flag is true, then [=queue a storage task=] to [=reject=] |p| with an {{InvalidStateError}}.

    1. Otherwise, set |bucket|'s [=StorageBucket/expiration time=] to |expires| milliseconds after the [=Unix epoch=].

    1. [=Queue a storage task=] to [=resolve=] |p|.

1. Return |p|.

</div>

<div algorithm>

The <dfn method for="StorageBucket">expires()</dfn> method steps are:

1. Let |p| be [=a new promise=].

1. Let |bucket| be [=this=]'s [=/storage bucket=].

1. Otherwise, run these steps [=in parallel=]:

    1. If |bucket|'s [=storage bucket/removed=] flag is true, then [=queue a storage task=] to [=reject=] |p| with an {{InvalidStateError}}.

    1. Otherwise, let |expiration| be |bucket|'s [=StorageBucket/expiration time=].

    1. [=Queue a storage task=] to [=resolve=] |p| with |expiration|.

1. Return |p|.

</div>

<h3 id="storage-bucket-endpoints">Using storage endpoints</h3>

Storage endpoints, i.e. storage bottles, can be accessed as described below.

<h4 id="storage-bucket-indexeddb">Using Indexed Database</h4>

Issue: {{IDBFactory}} methods need to take a storage bottle map rather than a storageKey.

<div algorithm>

A {{StorageBucket}} has an {{IDBFactory}} object, initially null. The <dfn attribute for=StorageBucket>indexedDB</dfn> getter steps are:

1. If [=this=]'s {{StorageBucket/indexedDB}} is null, run the following steps:

    1. Let |bucket| be [=this=]'s [=/storage bucket=].

    1. Let |bottle map| be the result of [=obtain a local storage bottle map=] with |bucket| and `"indexedDB"`.

    1. Let |indexedDB| be an {{IDBFactory}} object.

    1. Set the [=storage bottle map=] for |indexedDB| to |bottle map|.

    1. Set [=this=]'s {{StorageBucket/indexedDB}} to |indexedDB|.

1. Return [=this=]'s {{StorageBucket/indexedDB}}.

</div>

<h4 id="storage-bucket-caches">Using CacheStorage</h4>

<div algorithm>

A {{StorageBucket}} has a {{CacheStorage}} object, initially null. The <dfn attribute for=StorageBucket>caches</dfn> getter steps are:

1. If [=this=]'s {{StorageBucket/caches}} is null, run the following steps:

    1. Let |bucket| be [=this=]'s [=/storage bucket=].

    1. Let |bottle map| be the result of [=obtain a local storage bottle map=] with |bucket| and `"cacheStorage"`.

    1. Let |cacheStorage| be a {{CacheStorage}} object.

    1. Set the <a spec="service-workers">relevant name to cache map</a> for |cacheStorage| to |bottle map|.

    1. Set [=this=]'s {{StorageBucket/caches}} to |cacheStorage|.

1. Return [=this=]'s {{StorageBucket/caches}}.

</div>

<h4 id="storage-bucket-getdirectory">Using a Bucket File System</h4>

Issue: [[Storage]] needs to define helpers to retrieve the bottle map for a given (non-default) bucket.

Issue: [[FS]] needs to define a helper to retrieve an OPFS given a bottle map.

<div algorithm>

The <dfn method for=StorageBucket>getDirectory()</dfn> steps are:

1. Let |map| be the result of [=obtain a local storage bottle map=] with [=this=]'s [=/storage bucket=] and `"fileSystem"`.

1. Return the result of {{StorageManager/getDirectory}} with |map|.

</div>

<aside class="note">

See [[FS#sandboxed-filesystem]].

</aside>

<h3 id="storage-bucket-clear-site-data">Clear Site Data integration</h3>

Issue: Update [[clear-site-data#header]].

: "<dfn grammar>`storage:bucket-name`</dfn>"

  ::  If the type string starts with "`storage:`" then the remaining characters after the
      `:` will be taken to refer to a specific [=storage bucket=] in the [=environment settings object/origin=] of a
      particular response's URL.

Issue: add the steps below to the algorithm in [[clear-site-data#parsing]].

<div algorithm>

To <dfn>parse a Clear-Site-Data header with buckets</dfn>, execute the following steps:

1. For each |type| in <var ignore>header</var>, execute the following steps:

    1. If |type| does not [=string/start with=] `"storage:"`, abort these steps.

    1. Let |bucket name| be the [=code unit substring by positions|code unit substring=] from 8 to end of |type|.

    1. If the result of [=validate a bucket name=] with |bucket name| is failure, then abort these steps.

    1. Append a [=tuple=] consisting of (`"storage-bucket"`, |bucket name|) to <var ignore>types</var>

</div>

Issue: add the steps below to the algorithm in [[clear-site-data#clear-response]].

<div algorithm>

To <dfn>clear data with buckets</dfn> given a |bucket name|, execute the following steps:

1. Let |environment| be [=/this=]'s [=/relevant settings object=].

1. Let |shelf| be the result of running [=obtain a local storage shelf=] given |environment|.

1. If |shelf| is failure, then [=exception/throw=] a {{TypeError}} and abort these steps.

1. For each |type| in <var ignore>types</var>, execute the following steps:

    1. If |type| is not a [=tuple=] or |type|[0] is not `"storage-bucket"`, abort these steps.

    1. Let |bucket| be |shelf|'s [=bucket map=][|bucket name|] if one exists. Otherwise abort these steps

    1. Remove |bucket|.

</div>

<h2 id="security-privacy">Security and privacy considerations</h2>