From bb4bf4cbad574c1eb5bd43dfd2c196d0b715cdf0 Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Fri, 15 Jun 2018 15:25:51 +0200 Subject: [PATCH 1/8] add UPGRADING.md --- CHANGELOG.md | 4 -- README.md | 2 +- UPGRADING.md | 121 +++++++++++++++++++++++++++++++++++++++++++++++++++ package.json | 1 + 4 files changed, 123 insertions(+), 5 deletions(-) create mode 100644 UPGRADING.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 8898fa9..d5d4a4f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,10 +59,6 @@ * Remove testling from `package.json` (@vweevers) * Remove `level.js` logo (@vweevers) -**Historical Note** As a result of removing `IDBWrapper`, only modern browsers with a non-prefixed `window.indexedDB` are supported in this release. The current test matrix of `level-js` includes the latest versions of Chrome, Firefox, Safari, Edge and IE. - -**Historical Note** Though `level-js` now passes the full `abstract-leveldown` test suite, fulfulling the snapshot guarantee (reads not being affected by simultaneous writes) means a loss of backpressure as iterators have to keep reading from their IndexedDB cursors. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees. - **Historical Note** This release introduced the boolean `binaryKeys` and `arrayKeys` properties on the constructor, indicating whether the environment supports binary and array keys respectively. These properties may become private. **Historical Note** The vendored `IndexedDBShim` is still included, but likely to be removed. diff --git a/README.md b/README.md index c7734a5..bff5e83 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ Being `abstract-leveldown` compliant means you can use many of the [Level module ## Example -**This assumes use of version `3.0.0-rc1`. The next release will have an upgrade guide. Until then, please see the [changelog](CHANGELOG.md).** +**If you are upgrading:** please see [UPGRADING.md](UPGRADING.md). ```js var levelup = require('levelup') diff --git a/UPGRADING.md b/UPGRADING.md new file mode 100644 index 0000000..d9e4ff8 --- /dev/null +++ b/UPGRADING.md @@ -0,0 +1,121 @@ +# Upgrade Guide + +This document describes breaking changes and how to upgrade. For a complete list of changes including minor and patch releases, please refer to the [changelog](CHANGELOG.md). + +## V3 + +This release brings `level-js` up to par with latest [`levelup`] (v2), [`abstract-leveldown`] (v5) and IndexedDB Second Edition. It targets modern `browserify`, preferring [`Buffer`] (which is the primary binary type in the Level ecosystem) over `ArrayBuffer`. Lastly, [`IDBWrapper`] has been replaced with straight IndexedDB code. + +### Usage with [`levelup`] + +Usage has changed to: + +```js +const levelup = require('levelup') +const leveljs = require('leveljs') + +const db = levelup(leveljs('mydb')) +``` + +From the old: + +```js +const db = levelup('mydb', { db: leveljs }) +``` + +Friendly reminder: encodings have moved from `levelup` to `encoding-down`. To get identical functionality to `levelup` < 2, please use the `level-browserify` convenience package or wrap `level-js` with `encoding-down`: + +```js +const encoding = require('encoding-down') +const db = levelup(encoding(leveljs('mydb'))) +``` + +### New database prefix + +The default prefix of the [`IDBDatabase`](https://developer.mozilla.org/en-US/docs/Web/API/IDBDatabase) name has changed from `IDBWrapper-` to `level-js-`. To access databases created using `level-js` < 3, pass a custom prefix to the `level-js` constructor: + +```js +const db = levelup(leveljs('mydb', { prefix: 'IDBWrapper-' })) +``` + +### Browser support + +As a result of removing [`IDBWrapper`], only modern browsers with a non-prefixed `window.indexedDB` are supported in this release. The current test matrix of `level-js` includes the latest versions of Chrome, Firefox, Safari, Edge and IE. + +:fire: Internet Explorer 10 is no longer supported. + +### Support all value types of the [structured clone algorithm](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm) + +This means you can store almost any JS type without the need for [`encoding-down`]. Depending on the environment, this includes: + +- Number, including `NaN`, `Infinity` and `-Infinity` +- String, Boolean, Date, RegExp, Array, Object +- ArrayBuffer or a view thereof (typed arrays); +- Map, Set, Blob, File, FileList, ImageData (limited support). + +In addition `level-js` now stores `Buffer` values without transformation. This works in all target environments because modern `Buffer` is a subclass of `Uint8Array`, meaning such values can be passed to `IndexedDB` as-is. + +If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `put` or `batch`. For example, IE does not support typed array values. At the time of writing, Chrome is the only browser that supports all types listed above. + +Due to the special meaning that `null` and `undefined` have in `abstract-leveldown` iterators and Node.js streams, values of this type are converted to empty strings prior to storage. + +:point_up: The structured clone algorithm can be slower than `JSON.stringify`, so don't to quick to eschew `encoding-down`. + +### Support all key types of IndexedDB Second Edition + +Depending on the environment, this includes: + +- Number, including `Infinity` and `-Infinity`, but not `NaN` +- Date, except invalid (`NaN`) +- String +- ArrayBuffer or a view thereof (typed arrays); +- Array, except cyclical, empty and sparse arrays. Elements must be valid types themselves. + +In addition you can use `Buffer` keys, giving `level-js` the same power as implementations like `leveldown` and `memdown`. If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `get`, `put`, `del`, `batch` or an iterator. Exceptions are: + +- `null` and `undefined`: rejected early by `abstract-leveldown` +- Boolean and `NaN`: though invalid per the IndexedDB specification, they are converted to strings for `abstract-leveldown` compatibility; +- Binary and array keys: if not supported by the environment, `level-js` falls back to `String(key)`. + +### No backpressure + +In `level-js`, iterators are powered by IndexedDB cursors. To fulfill `abstract-leveldown` snapshot guarantees (reads not being affected by simultaneous writes) cursors are started immediately and continuously read from, filling an in-memory cache. + +Though `level-js` now passes the full [`abstract-leveldown`] test suite, fulfulling the snapshot guarantee means a loss of backpressure. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees. + +### Remove `raw` option + +Because `level-js` no longer stringifies values, the `raw` option (which bypassed conversion) became unnecessary and has been removed. If you use `level-browserify` or `levelup` with `encoding-down`, you can store and/or retrieve raw values (as returned by IndexedDB) using the `id` encoding: + +```js +const db = levelup(encoding(leveljs('mydb'), { valueEncoding: 'id' })) +``` + +Or per operation: + +```js +const db = levelup(encoding(leveljs('mydb'))) +const value = await db.get('key', { valueEncoding: 'id' }) +``` + +Note that if you stored binary data as a Buffer or typed array, with the `id` encoding you will get the data back as an `ArrayBuffer`. Conversely, with the `binary` encoding or no encoding at all, any binary data you put in will come back as a `Buffer`. This is true for both keys and values, in an environment that supports all types. + +### New `destroy()` function signature + +Previously, a `level-js` instance could be passed to `destroy()`: + +```js +leveljs.destroy(db, callback) +``` + +This was useful to destroy a database that used a custom prefix. The new signature is `destroy(location[, prefix], callback)`. + +### Strict `.batch(array)` + +The upgrade to [`abstract-leveldown`] comes with a [breaking change](https://github.com/Level/abstract-leveldown/commit/a2621ad70571f6ade9d2be42632ece042e068805) for the array version of `.batch()`. This change ensures all elements in the batch array are objects. If you previously passed arrays to `.batch()` that contained `undefined` or `null`, they would be silently ignored. Now this will produce an error. + +[`Buffer`]: https://nodejs.org/api/buffer.html +[`IDBWrapper`]: https://www.npmjs.com/package/idb-wrapper +[`abstract-leveldown`]: https://github.com/Level/abstract-leveldown +[`levelup`]: https://github.com/Level/levelup +[`encoding-down`]: https://github.com/Level/encoding-down diff --git a/package.json b/package.json index e63b753..83a39c6 100644 --- a/package.json +++ b/package.json @@ -16,6 +16,7 @@ "util", "CONTRIBUTORS.md", "CHANGELOG.md", + "UPGRADING.md", "sauce-labs.svg" ], "browser": { From 268df3d565301aee7551fee9a6197e794f91cdfc Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Fri, 15 Jun 2018 22:48:52 +0200 Subject: [PATCH 2/8] don't to => don't be too --- UPGRADING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/UPGRADING.md b/UPGRADING.md index d9e4ff8..e2346e5 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -59,7 +59,7 @@ If the environment does not support a type, it will throw an error which `level- Due to the special meaning that `null` and `undefined` have in `abstract-leveldown` iterators and Node.js streams, values of this type are converted to empty strings prior to storage. -:point_up: The structured clone algorithm can be slower than `JSON.stringify`, so don't to quick to eschew `encoding-down`. +:point_up: The structured clone algorithm can be slower than `JSON.stringify`, so don't be too quick to eschew `encoding-down`. ### Support all key types of IndexedDB Second Edition From 67dbb977fe0abad6dbadc9b00ec2b410f69c605a Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Sat, 16 Jun 2018 13:33:12 +0200 Subject: [PATCH 3/8] add remark-toc and remark-collapse --- README.md | 15 +++++++++++++++ package.json | 12 +++++++++++- 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index bff5e83..c1e667d 100644 --- a/README.md +++ b/README.md @@ -9,6 +9,21 @@ [![npm](https://img.shields.io/npm/dm/level-js.svg)](https://www.npmjs.com/package/level-js) [![JavaScript Style Guide](https://img.shields.io/badge/code_style-standard-brightgreen.svg)](https://standardjs.com) +## Table of Contents + +
Click to expand + +- [Background](#background) +- [Example](#example) +- [Browser Support](#browser-support) +- [Install](#install) +- [API](#api) +- [Running Tests](#running-tests) +- [Big Thanks](#big-thanks) +- [License](#license) + +
+ ## Background Here are the goals of `level-js`: diff --git a/package.json b/package.json index 83a39c6..39c3291 100644 --- a/package.json +++ b/package.json @@ -39,13 +39,23 @@ "buffer": "~5.1.0", "level-community": "~3.0.0", "remark-cli": "^5.0.0", + "remark-collapse": "~0.1.2", "remark-git-contributors": "~0.2.0", + "remark-toc": "~5.0.0", "standard": "^11.0.1", "tape": "^4.0.0" }, "remarkConfig": { "plugins": { - "remark-git-contributors": "level-community" + "remark-git-contributors": "level-community", + "remark-toc": { + "maxDepth": 2, + "tight": true + }, + "remark-collapse": { + "test": "Table of Contents", + "summary": "Click to expand" + } } }, "repository": { From 89b1a2fed6b1072b94742821b487574bb9e2c2e7 Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Sat, 16 Jun 2018 16:49:02 +0200 Subject: [PATCH 4/8] also run remark on UPGRADING.md (just for style) --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index 39c3291..22a90a3 100644 --- a/package.json +++ b/package.json @@ -8,7 +8,7 @@ "scripts": { "test": "standard && airtap --local --no-coverage test/index.js", "test-browsers": "standard && airtap --sauce-connect --loopback airtap.local --no-coverage test/index.js", - "remark": "remark README.md CONTRIBUTORS.md -o" + "remark": "remark README.md CONTRIBUTORS.md UPGRADING.md -o" }, "files": [ "index.js", From e161fee93d5a7f5384b92a03dc119e2fb6f28e8b Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Sat, 16 Jun 2018 17:36:23 +0200 Subject: [PATCH 5/8] move and expand type docs from UPGRADING to README --- README.md | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++ UPGRADING.md | 94 +++++++++++++++-------------------------- 2 files changed, 148 insertions(+), 61 deletions(-) diff --git a/README.md b/README.md index c1e667d..d7b5347 100644 --- a/README.md +++ b/README.md @@ -16,6 +16,7 @@ - [Background](#background) - [Example](#example) - [Browser Support](#browser-support) +- [Type Support](#type-support) - [Install](#install) - [API](#api) - [Running Tests](#running-tests) @@ -58,10 +59,104 @@ db.put('hello', Buffer.from('world'), function (err) { }) ``` +In ES6 browsers: + +```js +const levelup = require('levelup') +const leveljs = require('level-js') +const db = levelup(leveljs('bigdata')) + +await db.put('hello', Buffer.from('world')) +const value = await db.get('hello') +``` + ## Browser Support [![Sauce Test Status](https://saucelabs.com/browser-matrix/level-js.svg)](https://saucelabs.com/u/level-js) +## Type Support + +Unlike [`leveldown`][leveldown], `level-js` does not stringify keys or values. This means that in addition to strings and Buffers you can store almost any JavaScript type without the need for [`encoding-down`][encoding-down]. + +### Values + +All value types of the [structured clone algorithm][structured-clone-algorithm] are supported except for `null` and `undefined`. Depending on the environment, this includes: + +- Number, including `NaN`, `Infinity` and `-Infinity` +- String, Boolean, Date, RegExp, Array, Object +- ArrayBuffer or a view thereof (typed arrays); +- Map, Set, Blob, File, FileList, ImageData (limited support). + +In addition `level-js` stores [`Buffer`][buffer] values without transformation. This works in all target environments because `Buffer` is a subclass of `Uint8Array`, meaning such values can be passed to `IndexedDB` as-is. + +When getting or iterating binary values, regardless of whether they were stored as a `Buffer`, `ArrayBuffer` or a view thereof, values will return as a `Buffer`. This behavior can be disabled, in which case `ArrayBuffer` returns as `ArrayBuffer`, typed arrays return as typed arrays and `Buffer` returns as `Uint8Array`: + +```js +db.get('key', { asBuffer: false }) +db.iterator({ valueAsBuffer: false }) +``` + +If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `put` or `batch`. For example, IE does not support typed array values. At the time of writing, Chrome is the only browser that supports all types listed above. + +Due to the special meaning that `null` and `undefined` have in `abstract-leveldown` iterators and Node.js streams, values of this type are converted to empty strings prior to storage. + +### Keys + +All key types of IndexedDB Second Edition are supported. Depending on the environment, this includes: + +- Number, including `Infinity` and `-Infinity`, but not `NaN` +- Date, except invalid (`NaN`) +- String +- ArrayBuffer or a view thereof (typed arrays); +- Array, except cyclical, empty and sparse arrays. Elements must be valid types themselves. + +In addition you can use [`Buffer`][buffer] keys, giving `level-js` the same power as implementations like `leveldown` and `memdown`. When iterating binary keys, regardless of whether it was stored as a `Buffer`, `ArrayBuffer` or a view thereof, keys will return as a `Buffer`. This behavior can be disabled, in which case binary keys will always return as `ArrayBuffer`: + +```js +db.iterator({ keyAsBuffer: false }) +``` + +Note that this behavior is slightly different from values due to the way that IndexedDB works. IndexedDB stores binary _values_ using the structured clone algorithm, which preserves views, but it stores binary _keys_ as an array of octets, so that it is able to compare and sort differently typed keys. + +If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `get`, `put`, `del`, `batch` or an iterator. Exceptions are: + +- `null` and `undefined`: rejected early by `abstract-leveldown` +- Boolean and `NaN`: though invalid per the IndexedDB specification, they are converted to strings for `abstract-leveldown` compatibility; +- Binary and array keys: if not supported by the environment, `level-js` falls back to `String(key)`. + +### Normalization + +If you desire normalization for keys and values (e.g. to stringify numbers), wrap `level-js` with [`encoding-down`][encoding-down]. Alternatively install [`level-browserify`][level-browserify] which conveniently bundles [`levelup`][levelup], `level-js` and `encoding-down`. Such an approach is also recommended if you want to achieve universal (isomorphic) behavior or to smooth over type differences between browsers. For example, you could have [`leveldown`][leveldown] in a backend and `level-js` in the frontend. + +Another reason you might want to use `encoding-down` is that the structured clone algorithm, while rich in types, can be slower than `JSON.stringify`. + +### Buffer vs ArrayBuffer + +For interoperability it is recommended to use `Buffer` as your binary type. While we recognize that Node.js core modules are moving towards supporting `ArrayBuffer` and views thereof, `Buffer` remains the primary binary type in the Level ecosystem. + +That said: if you want to `put()` an `ArrayBuffer` you can! Just know that it will come back as a `Buffer` by default. If you want to `get()` or iterate stored `ArrayBuffer` data as an `ArrayBuffer`, you have a few options. Without `encoding-down`: + +```js +const db = levelup(leveljs('mydb')) + +// Yields an ArrayBuffer, Buffer and ArrayBuffer +const value1 = await db.get('key', { asBuffer: false }) +const value2 = await db.get('key') +const value3 = value2.buffer +``` + +With `encoding-down` (or `level-browserify`) you can use the `id` encoding to selectively bypass encodings: + +```js +const encode = require('encoding-down') +const db = levelup(encode(leveljs('mydb'), { valueEncoding: 'binary' })) + +// Yields an ArrayBuffer, Buffer and ArrayBuffer +const value1 = await db.get('key', { valueEncoding: 'id' }) +const value2 = await db.get('key') +const value3 = value2.buffer +``` + ## Install ```bash @@ -110,3 +205,23 @@ Cross-browser Testing Platform and Open Source ♥ Provided by [Sauce Labs](http [MIT](./LICENSE.md) © 2012-present [Max Ogden](https://github.com/maxogden) and [Contributors](./CONTRIBUTORS.md). [level-badge]: http://leveldb.org/img/badge.svg + +[indexeddb]: https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API + +[leveldb]: https://github.com/google/leveldb + +[buffer]: https://nodejs.org/api/buffer.html + +[awesome]: https://github.com/Level/awesome + +[abstract-leveldown]: https://github.com/Level/abstract-leveldown + +[levelup]: https://github.com/Level/levelup + +[leveldown]: https://github.com/Level/leveldown + +[level-browserify]: https://github.com/Level/level-browserify + +[encoding-down]: https://github.com/Level/encoding-down + +[structured-clone-algorithm]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm diff --git a/UPGRADING.md b/UPGRADING.md index e2346e5..7e9d7da 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -1,12 +1,12 @@ # Upgrade Guide -This document describes breaking changes and how to upgrade. For a complete list of changes including minor and patch releases, please refer to the [changelog](CHANGELOG.md). +This document describes breaking changes and how to upgrade. For a complete list of changes including minor and patch releases, please refer to the [changelog][changelog]. ## V3 -This release brings `level-js` up to par with latest [`levelup`] (v2), [`abstract-leveldown`] (v5) and IndexedDB Second Edition. It targets modern `browserify`, preferring [`Buffer`] (which is the primary binary type in the Level ecosystem) over `ArrayBuffer`. Lastly, [`IDBWrapper`] has been replaced with straight IndexedDB code. +This release brings `level-js` up to par with latest [`levelup`][levelup] (v2), [`abstract-leveldown`][abstract-leveldown] (v5) and IndexedDB Second Edition. It targets modern `browserify` preferring [`Buffer`][buffer] over `ArrayBuffer`. Lastly, [`IDBWrapper`][idbwrapper] has been replaced with straight IndexedDB code. -### Usage with [`levelup`] +### Usage with [`levelup`][levelup] Usage has changed to: @@ -23,16 +23,16 @@ From the old: const db = levelup('mydb', { db: leveljs }) ``` -Friendly reminder: encodings have moved from `levelup` to `encoding-down`. To get identical functionality to `levelup` < 2, please use the `level-browserify` convenience package or wrap `level-js` with `encoding-down`: +Friendly reminder: encodings have moved from [`levelup`][levelup] to [`encoding-down`][encoding-down]. To get identical functionality to `levelup < 2` please use the [`level-browserify`][level-browserify] convenience package or wrap `level-js` with `encoding-down`: ```js -const encoding = require('encoding-down') -const db = levelup(encoding(leveljs('mydb'))) +const encode = require('encoding-down') +const db = levelup(encode(leveljs('mydb'))) ``` ### New database prefix -The default prefix of the [`IDBDatabase`](https://developer.mozilla.org/en-US/docs/Web/API/IDBDatabase) name has changed from `IDBWrapper-` to `level-js-`. To access databases created using `level-js` < 3, pass a custom prefix to the `level-js` constructor: +The default prefix of the [`IDBDatabase`][idbdatabase] name has changed from `IDBWrapper-` to `level-js-`. To access databases created using `level-js < 3`, pass a custom prefix to the `level-js` constructor: ```js const db = levelup(leveljs('mydb', { prefix: 'IDBWrapper-' })) @@ -40,82 +40,54 @@ const db = levelup(leveljs('mydb', { prefix: 'IDBWrapper-' })) ### Browser support -As a result of removing [`IDBWrapper`], only modern browsers with a non-prefixed `window.indexedDB` are supported in this release. The current test matrix of `level-js` includes the latest versions of Chrome, Firefox, Safari, Edge and IE. +As a result of removing [`IDBWrapper`][idbwrapper], only modern browsers with a non-prefixed `window.indexedDB` are supported in this release. The current test matrix of `level-js` includes the latest versions of Chrome, Firefox, Safari, Edge and IE. :fire: Internet Explorer 10 is no longer supported. -### Support all value types of the [structured clone algorithm](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm) +### Type support -This means you can store almost any JS type without the need for [`encoding-down`]. Depending on the environment, this includes: +All value types of the [structured clone algorithm][structured-clone-algorithm] and all key types of IndexedDB Second Edition are now supported. This means you can store almost any JavaScript type without the need for [`encoding-down`][encoding-down]. In addition, you can use [`Buffer`][buffer] for both keys and values. For details and caveats please see the [readme][readme]. -- Number, including `NaN`, `Infinity` and `-Infinity` -- String, Boolean, Date, RegExp, Array, Object -- ArrayBuffer or a view thereof (typed arrays); -- Map, Set, Blob, File, FileList, ImageData (limited support). - -In addition `level-js` now stores `Buffer` values without transformation. This works in all target environments because modern `Buffer` is a subclass of `Uint8Array`, meaning such values can be passed to `IndexedDB` as-is. - -If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `put` or `batch`. For example, IE does not support typed array values. At the time of writing, Chrome is the only browser that supports all types listed above. - -Due to the special meaning that `null` and `undefined` have in `abstract-leveldown` iterators and Node.js streams, values of this type are converted to empty strings prior to storage. - -:point_up: The structured clone algorithm can be slower than `JSON.stringify`, so don't be too quick to eschew `encoding-down`. - -### Support all key types of IndexedDB Second Edition +### No backpressure -Depending on the environment, this includes: +In `level-js`, iterators are powered by IndexedDB cursors. To fulfill [`abstract-leveldown`][abstract-leveldown] snapshot guarantees (reads not being affected by simultaneous writes) cursors are started immediately and continuously read from, filling an in-memory cache. -- Number, including `Infinity` and `-Infinity`, but not `NaN` -- Date, except invalid (`NaN`) -- String -- ArrayBuffer or a view thereof (typed arrays); -- Array, except cyclical, empty and sparse arrays. Elements must be valid types themselves. +Though `level-js` now passes the full [`abstract-leveldown`][abstract-leveldown] test suite, fulfulling the snapshot guarantee means a loss of backpressure. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees. -In addition you can use `Buffer` keys, giving `level-js` the same power as implementations like `leveldown` and `memdown`. If the environment does not support a type, it will throw an error which `level-js` catches and passes to the callbacks of `get`, `put`, `del`, `batch` or an iterator. Exceptions are: +### Removed `raw` option -- `null` and `undefined`: rejected early by `abstract-leveldown` -- Boolean and `NaN`: though invalid per the IndexedDB specification, they are converted to strings for `abstract-leveldown` compatibility; -- Binary and array keys: if not supported by the environment, `level-js` falls back to `String(key)`. +Because `level-js` no longer stringifies values, the `raw` option (which bypassed conversion) became unnecessary and has been removed. If you use [`level-browserify`][level-browserify] or [`levelup`][levelup] with [`encoding-down`][encoding-down], you can store and retrieve raw values (as returned by IndexedDB) using the `id` encoding. Please refer to the [readme][readme] for an example. -### No backpressure +### New `destroy()` function signature -In `level-js`, iterators are powered by IndexedDB cursors. To fulfill `abstract-leveldown` snapshot guarantees (reads not being affected by simultaneous writes) cursors are started immediately and continuously read from, filling an in-memory cache. +Previously, a `level-js` instance could be passed to `destroy()`: -Though `level-js` now passes the full [`abstract-leveldown`] test suite, fulfulling the snapshot guarantee means a loss of backpressure. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees. +```js +leveljs.destroy(db, callback) +``` -### Remove `raw` option +This was useful to destroy a database that used a custom prefix. The new signature is `destroy(location[, prefix], callback)`. -Because `level-js` no longer stringifies values, the `raw` option (which bypassed conversion) became unnecessary and has been removed. If you use `level-browserify` or `levelup` with `encoding-down`, you can store and/or retrieve raw values (as returned by IndexedDB) using the `id` encoding: +### Strict `.batch(array)` -```js -const db = levelup(encoding(leveljs('mydb'), { valueEncoding: 'id' })) -``` +The upgrade to [`abstract-leveldown`][abstract-leveldown] comes with a [breaking change](https://github.com/Level/abstract-leveldown/commit/a2621ad70571f6ade9d2be42632ece042e068805) for the array version of `.batch()`. This change ensures all elements in the batch array are objects. If you previously passed arrays to `.batch()` that contained `undefined` or `null`, they would be silently ignored. Now this will produce an error. -Or per operation: +[readme]: README.md -```js -const db = levelup(encoding(leveljs('mydb'))) -const value = await db.get('key', { valueEncoding: 'id' }) -``` +[changelog]: CHANGELOG.md -Note that if you stored binary data as a Buffer or typed array, with the `id` encoding you will get the data back as an `ArrayBuffer`. Conversely, with the `binary` encoding or no encoding at all, any binary data you put in will come back as a `Buffer`. This is true for both keys and values, in an environment that supports all types. +[buffer]: https://nodejs.org/api/buffer.html -### New `destroy()` function signature +[idbwrapper]: https://www.npmjs.com/package/idb-wrapper -Previously, a `level-js` instance could be passed to `destroy()`: +[abstract-leveldown]: https://github.com/Level/abstract-leveldown -```js -leveljs.destroy(db, callback) -``` +[levelup]: https://github.com/Level/levelup -This was useful to destroy a database that used a custom prefix. The new signature is `destroy(location[, prefix], callback)`. +[encoding-down]: https://github.com/Level/encoding-down -### Strict `.batch(array)` +[level-browserify]: https://github.com/Level/level-browserify -The upgrade to [`abstract-leveldown`] comes with a [breaking change](https://github.com/Level/abstract-leveldown/commit/a2621ad70571f6ade9d2be42632ece042e068805) for the array version of `.batch()`. This change ensures all elements in the batch array are objects. If you previously passed arrays to `.batch()` that contained `undefined` or `null`, they would be silently ignored. Now this will produce an error. +[idbdatabase]: https://developer.mozilla.org/en-US/docs/Web/API/IDBDatabase -[`Buffer`]: https://nodejs.org/api/buffer.html -[`IDBWrapper`]: https://www.npmjs.com/package/idb-wrapper -[`abstract-leveldown`]: https://github.com/Level/abstract-leveldown -[`levelup`]: https://github.com/Level/levelup -[`encoding-down`]: https://github.com/Level/encoding-down +[structured-clone-algorithm]: https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm From 12d6828b54e1c0f76705b6f5cfb142d5af642c42 Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Sat, 16 Jun 2018 17:36:58 +0200 Subject: [PATCH 6/8] reuse reference links and shorten goals --- README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index d7b5347..42f49b2 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,8 @@ # level-js -> An [`abstract-leveldown`](https://github.com/Level/abstract-leveldown) compliant store on top of [IndexedDB](https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API), which is in turn implemented on top of [LevelDB](https://github.com/google/leveldb) which brings this whole shebang full circle. +> An [`abstract-leveldown`][abstract-leveldown] compliant store on top of [IndexedDB][indexeddb], which is in turn implemented on top of [LevelDB][leveldb] which brings this whole shebang full circle. -[![level badge][level-badge]](https://github.com/level/awesome) +[![level badge][level-badge]][awesome] [![npm](https://img.shields.io/npm/v/level-js.svg)](https://www.npmjs.com/package/level-js) [![npm next](https://img.shields.io/npm/v/level-js/next.svg)](https://www.npmjs.com/package/level-js) [![Travis](https://secure.travis-ci.org/Level/level-js.svg?branch=master)](http://travis-ci.org/Level/level-js) @@ -30,14 +30,14 @@ Here are the goals of `level-js`: - Store large amounts of data in modern browsers -- Pass the full `abstract-leveldown` test suite -- Support [Buffer](https://nodejs.org/api/buffer.html) values (in all target environments) -- Support all key types of IndexedDB Second Edition, including binary keys (depends on environment) -- Support all value types of the structured clone algorithm (depends on environment) except for `null` and `undefined` +- Pass the full [`abstract-leveldown`][abstract-leveldown] test suite +- Support [`Buffer`][buffer] keys and values +- Support all key types of IndexedDB Second Edition +- Support all value types of the [structured clone algorithm][structured-clone-algorithm] except for `null` and `undefined` - Be as fast as possible -- Sync with [multilevel](https://github.com/juliangruber/multilevel) over either ASCII or binary transports. +- Sync with [multilevel](https://github.com/juliangruber/multilevel) over ASCII or binary transports. -Being `abstract-leveldown` compliant means you can use many of the [Level modules](https://github.com/Level/awesome/) on top of this library. For some demos of it working, see @brycebaril's presentation [Path of the NodeBases Jedi](http://brycebaril.github.io/nodebase_jedi/#/vanilla). +Being `abstract-leveldown` compliant means you can use many of the [Level modules][awesome] on top of this library. For some demos of it working, see @brycebaril's presentation [Path of the NodeBases Jedi](http://brycebaril.github.io/nodebase_jedi/#/vanilla). ## Example From 4482f871ceaf0f9b17e56b68d536e6b247f396d0 Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Sat, 16 Jun 2018 18:10:18 +0200 Subject: [PATCH 7/8] it was => they were --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 42f49b2..60154fe 100644 --- a/README.md +++ b/README.md @@ -110,7 +110,7 @@ All key types of IndexedDB Second Edition are supported. Depending on the enviro - ArrayBuffer or a view thereof (typed arrays); - Array, except cyclical, empty and sparse arrays. Elements must be valid types themselves. -In addition you can use [`Buffer`][buffer] keys, giving `level-js` the same power as implementations like `leveldown` and `memdown`. When iterating binary keys, regardless of whether it was stored as a `Buffer`, `ArrayBuffer` or a view thereof, keys will return as a `Buffer`. This behavior can be disabled, in which case binary keys will always return as `ArrayBuffer`: +In addition you can use [`Buffer`][buffer] keys, giving `level-js` the same power as implementations like `leveldown` and `memdown`. When iterating binary keys, regardless of whether they were stored as `Buffer`, `ArrayBuffer` or a view thereof, keys will return as a `Buffer`. This behavior can be disabled, in which case binary keys will always return as `ArrayBuffer`: ```js db.iterator({ keyAsBuffer: false }) From 06e2fbb79d3554687e65032445e24e36a761856e Mon Sep 17 00:00:00 2001 From: Vincent Weevers Date: Sun, 17 Jun 2018 10:39:43 +0200 Subject: [PATCH 8/8] fulfulling => fulfilling --- UPGRADING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/UPGRADING.md b/UPGRADING.md index 7e9d7da..305c576 100644 --- a/UPGRADING.md +++ b/UPGRADING.md @@ -52,7 +52,7 @@ All value types of the [structured clone algorithm][structured-clone-algorithm] In `level-js`, iterators are powered by IndexedDB cursors. To fulfill [`abstract-leveldown`][abstract-leveldown] snapshot guarantees (reads not being affected by simultaneous writes) cursors are started immediately and continuously read from, filling an in-memory cache. -Though `level-js` now passes the full [`abstract-leveldown`][abstract-leveldown] test suite, fulfulling the snapshot guarantee means a loss of backpressure. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees. +Though `level-js` now passes the full [`abstract-leveldown`][abstract-leveldown] test suite, fulfilling the snapshot guarantee means a loss of backpressure. Memory consumption might increase if an iterator is not consumed fast enough. A future release will have an option to favor backpressure over snapshot guarantees. ### Removed `raw` option