diff --git a/doc/api/errors.md b/doc/api/errors.md index 391bb25d25cc5c..c4e7da6f628a33 100644 --- a/doc/api/errors.md +++ b/doc/api/errors.md @@ -624,16 +624,16 @@ An attempt was made to register something that is not a function as an The type of an asynchronous resource was invalid. Note that users are also able to define their own types if using the public embedder API. + +### ERR_BROTLI_COMPRESSION_FAILED + +Data passed to a Brotli stream was not successfully compressed. + ### ERR_BROTLI_INVALID_PARAM An invalid parameter key was passed during construction of a Brotli stream. - -### ERR_BROTLI_COMPRESSION_FAILED - -Data passed to a Brotli stream was not successfully compressed. - ### ERR_BUFFER_CONTEXT_NOT_AVAILABLE diff --git a/doc/api/zlib.md b/doc/api/zlib.md index 7b8f9187efb774..cbf3ebd180f87b 100644 --- a/doc/api/zlib.md +++ b/doc/api/zlib.md @@ -5,7 +5,7 @@ > Stability: 2 - Stable The `zlib` module provides compression functionality implemented using Gzip and -Deflate/Inflate. It can be accessed using: +Deflate/Inflate, as well as Brotli. It can be accessed using: ```js const zlib = require('zlib'); @@ -54,8 +54,8 @@ and/or unrecoverable and catastrophic memory fragmentation. ## Compressing HTTP requests and responses -The `zlib` module can be used to implement support for the `gzip` and `deflate` -content-encoding mechanisms defined by +The `zlib` module can be used to implement support for the `gzip`, `deflate` +and `br` content-encoding mechanisms defined by [HTTP](https://tools.ietf.org/html/rfc7230#section-4.2). The HTTP [`Accept-Encoding`][] header is used within an http request to identify @@ -76,12 +76,15 @@ const fs = require('fs'); const request = http.get({ host: 'example.com', path: '/', port: 80, - headers: { 'Accept-Encoding': 'gzip,deflate' } }); + headers: { 'Accept-Encoding': 'br,gzip,deflate' } }); request.on('response', (response) => { const output = fs.createWriteStream('example.com_index.html'); switch (response.headers['content-encoding']) { - // Or, just use zlib.createUnzip() to handle both cases + case 'br': + response.pipe(zlib.createBrotliDecompress()).pipe(output); + break; + // Or, just use zlib.createUnzip() to handle both of the following cases: case 'gzip': response.pipe(zlib.createGunzip()).pipe(output); break; @@ -117,6 +120,9 @@ http.createServer((request, response) => { } else if (/\bgzip\b/.test(acceptEncoding)) { response.writeHead(200, { 'Content-Encoding': 'gzip' }); raw.pipe(zlib.createGzip()).pipe(response); + } else if (/\bbr\b/.test(acceptEncoding)) { + response.writeHead(200, { 'Content-Encoding': 'br' }); + raw.pipe(zlib.createBrotliCompress()).pipe(response); } else { response.writeHead(200, {}); raw.pipe(response); @@ -136,6 +142,7 @@ const buffer = Buffer.from('eJzT0yMA', 'base64'); zlib.unzip( buffer, + // For Brotli, the equivalent is zlib.constants.BROTLI_OPERATION_FLUSH. { finishFlush: zlib.constants.Z_SYNC_FLUSH }, (err, buffer) => { if (!err) { @@ -156,6 +163,8 @@ decompressed result is valid. +### For zlib-based streams + From `zlib/zconf.h`, modified to Node.js's usage: The memory requirements for deflate are (in bytes): @@ -194,6 +203,16 @@ fewer calls to `zlib` because it will be able to process more data on each `write` operation. So, this is another factor that affects the speed, at the cost of memory usage. +### For Brotli-based streams + +There are equivalents to the zlib options for Brotli-based streams, although +these options have different ranges than the zlib ones: + +- zlib’s `level` option matches Brotli’s `BROTLI_PARAM_QUALITY` option. +- zlib’s `windowBits` option matches Brotli’s `BROTLI_PARAM_LGWIN` option. + +See [below][Brotli parameters] for more details on Brotli-specific options. + ## Flushing Calling [`.flush()`][] on a compression stream will make `zlib` return as much @@ -231,6 +250,8 @@ added: v0.5.8 +### zlib constants + All of the constants defined in `zlib.h` are also defined on `require('zlib').constants`. In the normal course of operations, it will not be necessary to use these constants. They are documented so that their presence is @@ -281,6 +302,77 @@ Compression strategy. * `zlib.constants.Z_FIXED` * `zlib.constants.Z_DEFAULT_STRATEGY` +### Brotli constants + + +There are several options and other constants available for Brotli-based +streams: + +#### Flush operations + +The following values are valid flush operations for Brotli-based streams: + +* `zlib.constants.BROTLI_OPERATION_PROCESS` (default for all operations) +* `zlib.constants.BROTLI_OPERATION_FLUSH` (default when calling `.flush()`) +* `zlib.constants.BROTLI_OPERATION_FINISH` (default for the last chunk) +* `zlib.constants.BROTLI_OPERATION_EMIT_METADATA` + * This particular operation may be hard to use in a Node.js context, + as the streaming layer makes it hard to know which data will end up + in this frame. Also, there is currently no way to consume this data through + the Node.js API. + +#### Compressor options + +There are several options that can be set on Brotli encoders, affecting +compression efficiency and speed. Both the keys and the values can be accessed +as properties of the `zlib.constants` object. + +The most important options are: + +* `BROTLI_PARAM_MODE` + * `BROTLI_MODE_GENERIC` (default) + * `BROTLI_MODE_TEXT`, adjusted for UTF-8 text + * `BROTLI_MODE_FONT`, adjusted for WOFF 2.0 fonts +* `BROTLI_PARAM_QUALITY` + * Ranges from `BROTLI_MIN_QUALITY` to `BROTLI_MAX_QUALITY`, + with a default of `BROTLI_DEFAULT_QUALITY`. +* `BROTLI_PARAM_SIZE_HINT` + * Integer value representing the expected input size; + defaults to `0` for an unknown input size. + +The following flags can be set for advanced control over the compression +algorithm and memory usage tuning: + +* `BROTLI_PARAM_LGWIN` + * Ranges from `BROTLI_MIN_WINDOW_BITS` to `BROTLI_MAX_WINDOW_BITS`, + with a default of `BROTLI_DEFAULT_WINDOW`, or up to + `BROTLI_LARGE_MAX_WINDOW_BITS` if the `BROTLI_PARAM_LARGE_WINDOW` flag + is set. +* `BROTLI_PARAM_LGBLOCK` + * Ranges from `BROTLI_MIN_INPUT_BLOCK_BITS` to `BROTLI_MAX_INPUT_BLOCK_BITS`. +* `BROTLI_PARAM_DISABLE_LITERAL_CONTEXT_MODELING` + * Boolean flag that decreases compression ratio in favour of + decompression speed. +* `BROTLI_PARAM_LARGE_WINDOW` + * Boolean flag enabling “Large Window Brotli” mode (not compatible with the + Brotli format as standardized in [RFC 7932][]). +* `BROTLI_PARAM_NPOSTFIX` + * Ranges from `0` to `BROTLI_MAX_NPOSTFIX`. +* `BROTLI_PARAM_NDIRECT` + * Ranges from `0` to `15 << NPOSTFIX` in steps of `1 << NPOSTFIX`. + +#### Decompressor options + +These advanced options are available for controlling decompression: + +* `BROTLI_DECODER_PARAM_DISABLE_RING_BUFFER_REALLOCATION` + * Boolean flag that affects internal memory allocation patterns. +* `BROTLI_DECODER_PARAM_LARGE_WINDOW` + * Boolean flag enabling “Large Window Brotli” mode (not compatible with the + Brotli format as standardized in [RFC 7932][]). + ## Class: Options -Each class takes an `options` object. All options are optional. +Each zlib-based class takes an `options` object. All options are optional. Note that some options are only relevant when compressing, and are ignored by the decompression classes. @@ -317,6 +409,47 @@ ignored by the decompression classes. See the description of `deflateInit2` and `inflateInit2` at for more information on these. +## Class: BrotliOptions + + + + +Each Brotli-based class takes an `options` object. All options are optional. + +* `flush` {integer} **Default:** `zlib.constants.BROTLI_OPERATION_PROCESS` +* `finishFlush` {integer} **Default:** `zlib.constants.BROTLI_OPERATION_FINISH` +* `chunkSize` {integer} **Default:** `16 * 1024` +* `params` {Object} Key-value object containing indexed [Brotli parameters][]. + +For example: + +```js +const stream = zlib.createBrotliCompress({ + chunkSize: 32 * 1024, + params: { + [zlib.constants.BROTLI_PARAM_MODE]: zlib.constants.BROTLI_MODE_TEXT, + [zlib.constants.BROTLI_PARAM_QUALITY]: 4, + [zlib.constants.BROTLI_PARAM_SIZE_HINT]: fs.statSync(inputFile).size + } +}); +``` + +## Class: zlib.BrotliCompress + + +Compress data using the Brotli algorithm. + +## Class: zlib.BrotliDecompress + + +Decompress data using the Brotli algorithm. + ## Class: zlib.Deflate Not exported by the `zlib` module. It is documented here because it is the base @@ -440,7 +577,8 @@ Close the underlying handle. added: v0.5.8 --> -* `kind` **Default:** `zlib.constants.Z_FULL_FLUSH` +* `kind` **Default:** `zlib.constants.Z_FULL_FLUSH` for zlib-based streams, + `zlib.constants.BROTLI_OPERATION_FLUSH` for Brotli-based streams. * `callback` {Function} Flush pending data. Don't call this frivolously, premature flushes negatively @@ -460,6 +598,8 @@ added: v0.11.4 * `strategy` {integer} * `callback` {Function} +This function is only available for zlib-based streams, i.e. not Brotli. + Dynamically update the compression level and compression strategy. Only applicable to deflate algorithm. @@ -478,6 +618,24 @@ added: v7.0.0 Provides an object enumerating Zlib-related constants. +## zlib.createBrotliCompress([options]) + + +* `options` {brotli options} + +Creates and returns a new [`BrotliCompress`][] object. + +## zlib.createBrotliDecompress([options]) + + +* `options` {brotli options} + +Creates and returns a new [`BrotliDecompress`][] object. + ## zlib.createDeflate([options]) +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} +* `callback` {Function} + +### zlib.brotliCompressSync(buffer[, options]) + +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} + +Compress a chunk of data with [`BrotliCompress`][]. + +### zlib.brotliDecompress(buffer[, options], callback) + +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} +* `callback` {Function} + +### zlib.brotliDecompressSync(buffer[, options]) + +* `buffer` {Buffer|TypedArray|DataView|ArrayBuffer|string} +* `options` {brotli options} + +Decompress a chunk of data with [`BrotliDecompress`][]. + ### zlib.deflate(buffer[, options], callback)