Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds RESP3 to protocol specification #2120

Merged
merged 35 commits into from
Aug 15, 2023
Merged
Changes from 2 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
a2089da
Adds RESP3 to protocol specification
itamarhaber Sep 13, 2022
2f07005
Spelling and whitelisting
itamarhaber Sep 13, 2022
ebaa2b7
Edits
itamarhaber Sep 13, 2022
42db0a8
Typo
itamarhaber Sep 13, 2022
cea81c7
Apply suggestions from code review
itamarhaber Dec 29, 2022
4a14719
Merge branch 'master' into fix-1920
itamarhaber Dec 29, 2022
8e0dcb6
Applies code review comments
itamarhaber Dec 29, 2022
fb3c92c
Edit
itamarhaber Dec 29, 2022
10f3d7e
worlist and edits
itamarhaber Dec 29, 2022
88bddcc
Adds NaNs
itamarhaber Dec 30, 2022
a840f7b
Edits
itamarhaber Dec 30, 2022
8ebebbd
Renames blob error to bulk error
itamarhaber Dec 30, 2022
554536a
wordlist
itamarhaber Dec 30, 2022
c7d58c8
Renames errors to simple errors
itamarhaber Dec 30, 2022
7b57825
Normalize ~nAn
itamarhaber Jan 5, 2023
25310cf
Adds protected-mode protocol exception
itamarhaber Feb 28, 2023
9012299
Wordlist
itamarhaber Feb 28, 2023
8655530
Review suggestion
itamarhaber Feb 28, 2023
d6bf3e3
Adds RESP2 map pattern
itamarhaber Mar 12, 2023
7abae3e
Adds double exponent part
itamarhaber Mar 12, 2023
0435303
Reverts fat thumbs
itamarhaber Mar 12, 2023
05e458a
Adds a singular tips
itamarhaber Mar 15, 2023
01da619
Update docs/reference/protocol-spec.md
itamarhaber Mar 15, 2023
6280997
Exponent optionality
itamarhaber Mar 15, 2023
bb11d60
Align
itamarhaber Mar 15, 2023
d921e47
CRLF
itamarhaber Mar 15, 2023
60770f3
uNDO
itamarhaber Mar 15, 2023
58c690a
Parameterize it
itamarhaber Mar 15, 2023
b324085
Wordlist
itamarhaber Mar 15, 2023
3e3cb2b
Breaks down integer's and double's encodings
itamarhaber Mar 15, 2023
fd32f3d
Make it formal, ASCII, base-10 crystal clear
itamarhaber Mar 16, 2023
9ad2bd4
Excludes streaming types
itamarhaber Mar 16, 2023
6482f97
Update docs/reference/protocol-spec.md
itamarhaber Mar 17, 2023
23ae7ec
CR https://github.com/redis/redis-doc/pull/2120#discussion_r1140109937
itamarhaber Mar 17, 2023
03e1905
Merge branch 'master' into fix-1920
itamarhaber Aug 15, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 48 additions & 33 deletions docs/reference/protocol-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,11 @@ Using RESP with Redis 1.2 was optional and had mainly served the purpose of work
In Redis 2.0, the protocol's next version, a.k.a RESP2, became the standard communication method for clients with the Redis server.

[RESP3](https://github.com/redis/redis-specifications/blob/master/protocol/RESP3.md) is a superset of RESP2 that mainly aims to make a client author's life a little bit easier.
Redis 6.0 introduced experimental opt-in support to a subset of RESP3's features.
Redis 6.0 introduced experimental opt-in support of RESP3's features (excluding streaming strings and streaming aggregates).
In addition, the introduction of the `HELLO` command allows clients to handshake and upgrade the connection's protocol version (see [Client handshake](#client-handshake)).

Up to and including Redis 7, both RESP2 and RESP3 clients can invoke all core commands. However, some commands return differently-typed replies for different protocol versions.
Up to and including Redis 7, both RESP2 and RESP3 clients can invoke all core commands.
However, commands may return differently typed replies for different protocol versions.

Future versions of Redis may change the default protocol version, but it is unlikely that RESP2 will become entirely deprecated.
It is possible, however, that new features in upcoming versions will require the use of RESP3.
Expand Down Expand Up @@ -85,13 +86,23 @@ Redis generally uses RESP as a [request-response](#request-response-model) proto
* The server replies with a RESP type.
The reply's type is determined by the command's implementation and possibly by the client's protocol version.

We categorize every RESP data type as either _simple_ or _aggregate_.
Simple types are similar to scalars in programming languages that represent plain literal values, for example, Booleans.
Aggregates, such as Arrays and Maps, can have varying numbers of sub-elements and nesting levels.
RESP is **always** ASCII-encoded (UTF-8/Latin-1).

The `\r\n` (CRLF) is the protocol's _terminator_, which **always** separates its parts.

The first byte in an RESP-serialized payload identifies its type.
The first byte in an RESP-serialized payload always identifies its type.
Subsequent bytes constitute the type's contents.
The `\r\n` (CRLF) is the protocol's _terminator_, which always separates its parts.

We categorize every RESP data type as either _simple_, _bulk_ or _aggregate_.

Simple types are similar to scalars in programming languages that represent plain literal values. Booleans and Integers are such examples.

RESP strings are either _simple_ or _bulk_.
Simple strings never contain carriage return (`\r`) or line feed (`\n`) characters.
Bulk strings can contain any ASCII character and may also be referred to as _binary_ or _blob_.
Note that bulk strings may be further encoded and decoded, e.g. with a wide multibyte encoding, by the client.

Aggregates, such as Arrays and Maps, can have varying numbers of sub-elements and nesting levels.

The following table summarizes the RESP data types that Redis supports:

Expand All @@ -116,7 +127,7 @@ The following table summarizes the RESP data types that Redis supports:

### Simple strings
Simple strings are encoded as a plus (`+`) character, followed by a string.
The string mustn't contain a CR or LF character (no newlines are allowed), and is terminated by CRLF (i.e., `\r\n`).
The string mustn't contain a CR or LF character (no newlines are allowed) and is terminated by CRLF (i.e., `\r\n`).

Simple strings transmit short, non-binary strings with minimal overhead.
For example, many Redis commands reply with just "OK" on success.
Expand Down Expand Up @@ -162,15 +173,15 @@ Also, simpler client implementations can return a generic error value, such as `
<a name="integer-reply"></a>

### Integers
This type is a CRLF-terminated string that represents a signed 64-bit integer.
This type is a CRLF-terminated string that represents a signed, base-10, 64-bit integer.

RESP encodes integers in the following way:

:[<sign>]<integer>\r\n
:[<+|->]<value>\r\n

* The colon (`:`) as the first byte.
* An optional plus (`+`) or minus (`-`) as the sign.
* One or more decimal digits (`0`..`9`) as the integer's value.
* One or more decimal digits (`0`..`9`) as the integer's unsigned, base-10 value.
* The CRLF terminator.

For example, `:0\r\n` and `:1000\r\n` are integer replies (of zero and one thousand, respectively).
Expand All @@ -196,7 +207,7 @@ RESP encodes bulk strings in the following way:
$<length>\r\n<data>\r\n

* The dollar sign (`$`) as the first byte.
* The length, or the number of bytes, composing the string (a prefixed length).
* One or more decimal digits (`0`..`9`) as the string's length, in bytes, as an unsigned, base-10 value.
* The CRLF terminator.
* The data.
* A final CRLF.
Expand Down Expand Up @@ -237,7 +248,7 @@ RESP Arrays' encoding uses the following format:
*<number-of-elements>\r\n<element-1>...<element-n>

* An asterisk (`*`) as the first byte.
* The number of elements in the array as the string representation of an integer.
* One or more decimal digits (`0`..`9`) as the number of elements in the array as an unsigned, base-10 value.
* The CRLF terminator.
* An additional RESP type for every element of the array.

Expand Down Expand Up @@ -350,7 +361,7 @@ The null type, introduced in RESP3, aims to fix this wrong.
### Booleans
RESP booleans are encoded as follows:

#<boolean>\r\n
#<t|f>\r\n

* The octothorpe character (`#`) as the first byte.
* A `t` character for true values, or an `f` character for false ones.
Expand All @@ -362,13 +373,13 @@ RESP booleans are encoded as follows:
The Double RESP type encodes a double-precision floating point value.
Doubles are encoded as follows:

,[<sign>]<integral>[.<fractional>][<E|e>[sign]<exponent>]\r\n
,[<+|->]<integral>[.<fractional>][<E|e>[sign]<exponent>]\r\n

* The comma character (`,`) as the first byte.
* An optional plus (`+`) or minus (`-`) as the sign.
* The integral value that's represented by one or more decimal digits (`0`..`9`).
* The optional fractional value that's represented by a dot (`.`), followed by one or more decimal digits (`0`..`9`).
* The optional exponent value begins with a capital or lowercase letter E (`E` or `e`), followed by an optional plus (`+`) or minus (`-`) as the exponent's sign, ending with one or more decimal digits (`0`..`9`).
* One or more decimal digits (`0`..`9`) as an unsigned, base-10 integral value.
* An optional dot (`.`), followed by one or more decimal digits (`0`..`9`) as an unsigned, base-10 fractional value.
* An optional capital or lowercase letter E (`E` or `e`), followed by an optional plus (`+`) or minus (`-`) as the exponent's sign, ending with one or more decimal digits (`0`..`9`) as an unsigned, base-10 exponent value.
* The CRLF terminator.

Here's the encoding of the number 1.23:
Expand All @@ -395,17 +406,18 @@ This type can encode integer values outside the range of signed 64-bit integers.

Big numbers use the following encoding:

(<big-number>\r\n
([+|-]<number>\r\n

* The left parenthesis character (`(`) as the first byte.
* The number.
* An optional plus (`+`) or minus (`-`) as the sign.
* One or more decimal digits (`0`..`9`) as an unsigned, base-10 value.
* The CRLF terminator.

Example:

(3492890328409238509324850943850943825024385\r\n

Big numbers can be positive or negative but can't include decimals.
Big numbers can be positive or negative but can't include fractionals.
Client libraries written in languages with a big number type should return a big number.
When big numbers aren't supported, the client should return a string and, when possible, signal to the caller that the reply is a big integer (depending on the API used by the client library).

Expand All @@ -419,7 +431,7 @@ It is encoded as:
!<length>\r\n<error>\r\n

* An exclamation mark (`!`) as the first byte.
* The length of the error message in bytes.
* One or more decimal digits (`0`..`9`) as the error's length, in bytes, as an unsigned, base-10 value.
* The CRLF terminator.
* The error itself.
* A final CRLF.
Expand All @@ -436,17 +448,17 @@ For instance, the error "SYNTAX invalid syntax" is represented by the following
<a name="verbatim-string-reply">

### Verbatim strings
This type is similar to the [bulk string](#bulk-strings), with the addition of providing a hint about the data's format.
This type is similar to the [bulk string](#bulk-strings), with the addition of providing a hint about the data's encoding.

A verbatim string's RESP encoding is as follows:

=<length>\r\n<three-bytes>:<data>\r\n
=<length>\r\n<encoding>:<data>\r\n

* An equal sign (`=`) as the first byte.
* The data's bytes length.
* One or more decimal digits (`0`..`9`) as the string's total length, in bytes, as an unsigned, base-10 value.
* The CRLF terminator.
* Three (3) bytes of additional information about the data.
* The colon (`:`) character.
* Exactly three (3) bytes represent the data's encoding.
* The colon (`:`) character separates the encoding and data.
* The data.
* A final CRLF.

Expand All @@ -461,7 +473,7 @@ Some client libraries may ignore the difference between this type and the string
However, interactive clients, such as command line interfaces (e.g., [`redis-cli`](/docs/manual/cli)), can use this type and know that their output should be presented to the human user as is and without quoting the string.

For example, the Redis command `INFO` outputs a report that includes newlines.
When using RESP3, `redis-cli` displays it correctly because it is sent as a Verbatim String reply (with its three bytes being "raw").
When using RESP3, `redis-cli` displays it correctly because it is sent as a Verbatim String reply (with its three bytes being "txt").
When using RESP2, however, the `redis-cli` is hard-coded to look for the `INFO` command to ensure its correct display to the user.

<a name="map-reply"></a>
Expand All @@ -474,9 +486,9 @@ It is encoded as follows:
%<number-of-entries>\r\n<key-1><value-1>...<key-n><value-n>

* A percent character (`%`) as the first byte.
* The number of entries, or key-value tuples, as the string representation of an integer.
* One or more decimal digits (`0`..`9`) as the number of entries, or key-value tuples, in the map as an unsigned, base-10 value.
* The CRLF terminator.
* Two additional RESP types for every key and value in the Map.
* Two additional RESP types for every key and value in the map.

For example, the following JSON object:

Expand Down Expand Up @@ -516,7 +528,7 @@ RESP set's encoding is:
~<number-of-elements>\r\n<element-1>...<element-n>

* A tilde (`~`) as the first byte.
* The number of elements in the set as the string representation of an integer.
* One or more decimal digits (`0`..`9`) as the number of elements in the set as an unsigned, base-10 value.
* The CRLF terminator.
* An additional RESP type for every element of the Set.

Expand All @@ -534,7 +546,7 @@ Push events are encoded similarly to [arrays](#arrays), differing only in their
><number-of-elements>\r\n<element-1>...<element-n>

* A greater-than sign (`>`) as the first byte.
* The number of elements as the string representation of an integer.
* One or more decimal digits (`0`..`9`) as the number of elements in the message as an unsigned, base-10 value.
* The CRLF terminator.
* An additional RESP type for every element of the push event.

Expand Down Expand Up @@ -681,4 +693,7 @@ While comparable in performance to a binary protocol, the Redis protocol is sign

* For testing purposes, use [Lua's type conversions](/topics/lua-api#lua-to-resp3-type-conversion) to have Redis reply with any RESP2/RESP3 needed.
As an example, a RESP3 double can be generated like so:
EVAL "return { double = tonumber(ARGV[1]) }" 0 1e0
```
EVAL "return { double = tonumber(ARGV[1]) }" 0 1e0
```