diff --git a/docs/reference/protocol-spec.md b/docs/reference/protocol-spec.md index 732e81b01a..bbb9f331cc 100644 --- a/docs/reference/protocol-spec.md +++ b/docs/reference/protocol-spec.md @@ -1,214 +1,274 @@ --- -title: "RESP protocol spec" +title: "Redis serialization protocol specification" linkTitle: "Protocol spec" weight: 4 -description: Redis serialization protocol (RESP) specification +description: Redis serialization protocol (RESP) is the wire protocol that clients implement aliases: - /topics/protocol --- -Redis clients use a protocol called **RESP** (REdis Serialization Protocol) to communicate with the Redis server. While the protocol was designed specifically for Redis, it can be used for other client-server software projects. +To communicate with the Redis server, Redis clients use a protocol called REdis Serialization Protocol (RESP). +While the protocol was designed specifically for Redis, you can use it for other client-server software projects. -RESP is a compromise between the following things: +RESP is a compromise among the following considerations: * Simple to implement. * Fast to parse. * Human readable. -RESP can serialize different data types like integers, strings, and arrays. There is also a specific type for errors. Requests are sent from the client to the Redis server as arrays of strings that represent the arguments of the command to execute. Redis replies with a command-specific data type. +RESP can serialize different data types including integers, strings, and arrays. +It also features an error-specific type. +A client sends a request to the Redis server as an array of strings. +The array's contents are the command and its arguments that the server should execute. +The server's reply type is command-specific. -RESP is binary-safe and does not require processing of bulk data transferred from one process to another because it uses prefixed-length to transfer bulk data. +RESP is binary-safe and uses prefixed length to transfer bulk data so it does not require processing bulk data transferred from one process to another. -Note: the protocol outlined here is only used for client-server communication. Redis Cluster uses a different binary protocol in order to exchange messages between nodes. +RESP is the protocol you should implement in your Redis client. -## Network layer +{{% alert title="Note" color="info" %}} +The protocol outlined here is used only for client-server communication. +[Redis Cluster](/docs/reference/cluster-spec) uses a different binary protocol for exchanging messages between nodes. +{{% /alert %}} -A client connects to a Redis server by creating a TCP connection to the port 6379. +## RESP versions +Support for the first version of the RESP protocol was introduced in Redis 1.2. +Using RESP with Redis 1.2 was optional and had mainly served the purpose of working the kinks out of the protocol. -While RESP is technically non-TCP specific, the protocol is only used with TCP connections (or equivalent stream-oriented connections like Unix sockets) in the context of Redis. +In Redis 2.0, the protocol's next version, a.k.a RESP2, became the standard communication method for clients with the Redis server. -## Request-Response model +[RESP3](https://github.com/redis/redis-specifications/blob/master/protocol/RESP3.md) is a superset of RESP2 that mainly aims to make a client author's life a little bit easier. +Redis 6.0 introduced experimental opt-in support of RESP3's features (excluding streaming strings and streaming aggregates). +In addition, the introduction of the `HELLO` command allows clients to handshake and upgrade the connection's protocol version (see [Client handshake](#client-handshake)). -Redis accepts commands composed of different arguments. -Once a command is received, it is processed and a reply is sent back to the client. +Up to and including Redis 7, both RESP2 and RESP3 clients can invoke all core commands. +However, commands may return differently typed replies for different protocol versions. -This is the simplest model possible; however, there are two exceptions: +Future versions of Redis may change the default protocol version, but it is unlikely that RESP2 will become entirely deprecated. +It is possible, however, that new features in upcoming versions will require the use of RESP3. + +## Network layer +A client connects to a Redis server by creating a TCP connection to its port (the default is 6379). -* Redis supports pipelining (covered later in this document). So it is possible for clients to send multiple commands at once and wait for replies later. -* When a Redis client subscribes to a Pub/Sub channel, the protocol changes semantics and becomes a *push* protocol. The client no longer requires sending commands because the server will automatically send new messages to the client (for the channels the client is subscribed to) as soon as they are received. +While RESP is technically non-TCP specific, the protocol is used exclusively with TCP connections (or equivalent stream-oriented connections like Unix sockets) in the context of Redis. -Excluding these two exceptions, the Redis protocol is a simple request-response protocol. +## Request-Response model +The Redis server accepts commands composed of different arguments. +Then, the server processes the command and sends the reply back to the client. + +This is the simplest model possible; however, there are some exceptions: + +* Redis requests can be [pipelined](#multiple-commands-and-pipelining). + Pipelining enables clients to send multiple commands at once and wait for replies later. +* When a RESP2 connection subscribes to a [Pub/Sub](/docs/manual/pubsub) channel, the protocol changes semantics and becomes a *push* protocol. + The client no longer requires sending commands because the server will automatically send new messages to the client (for the channels the client is subscribed to) as soon as they are received. +* The `MONITOR` command. + Invoking the `MONITOR` command switches the connection to an ad-hoc push mode. + The protocol of this mode is not specified but is obvious to parse. +* [Protected mode](/docs/management/security/#protected-mode). + Connections opened from a non-loopback address to a Redis while in protected mode are denied and terminated by the server. + Before terminating the connection, Redis unconditionally sends `-DENIED` reply, irregardless of whether the client writes to the socket. +* The [RESP3 Push type](#resp3-pushes). + As the name suggests, a push type allows the server to send out-of-band data to the connection. + The server may push data at any time, and the data isn't necessarily related to specific commands executed by the client. + +Excluding these exceptions, the Redis protocol is a simple request-response protocol. ## RESP protocol description +RESP is essentially a serialization protocol that supports several data types. +In RESP, the first byte of data determines its type. -The RESP protocol was introduced in Redis 1.2, but it became the -standard way for talking with the Redis server in Redis 2.0. -This is the protocol you should implement in your Redis client. +Redis generally uses RESP as a [request-response](#request-response-model) protocol in the following way: -RESP is actually a serialization protocol that supports the following -data types: Simple Strings, Errors, Integers, Bulk Strings, and Arrays. +* Clients send commands to a Redis server as an [array](#arrays) of [bulk strings](#bulk-strings). + The first (and sometimes also the second) bulk string in the array is the command's name. + Subsequent elements of the array are the arguments for the command. +* The server replies with a RESP type. + The reply's type is determined by the command's implementation and possibly by the client's protocol version. -Redis uses RESP as a request-response protocol in the -following way: +RESP is a binary protocol that uses control sequences encoded in standard ASCII. +The `A` character, for example, is encoded with the binary byte of value 65. +Similarly, the characters CR (`\r`), LF (`\n`) and SP (` `) have binary byte values of 13, 10 and 32, respectively. -* Clients send commands to a Redis server as a RESP Array of Bulk Strings. -* The server replies with one of the RESP types according to the command implementation. +The `\r\n` (CRLF) is the protocol's _terminator_, which **always** separates its parts. -In RESP, the first byte determines the data type: +The first byte in an RESP-serialized payload always identifies its type. +Subsequent bytes constitute the type's contents. -* For **Simple Strings**, the first byte of the reply is "+" -* For **Errors**, the first byte of the reply is "-" -* For **Integers**, the first byte of the reply is ":" -* For **Bulk Strings**, the first byte of the reply is "$" -* For **Arrays**, the first byte of the reply is "`*`" +We categorize every RESP data type as either _simple_, _bulk_ or _aggregate_. -RESP can represent a Null value using a special variation of Bulk Strings or Array as specified later. +Simple types are similar to scalars in programming languages that represent plain literal values. Booleans and Integers are such examples. -In RESP, different parts of the protocol are always terminated with "\r\n" (CRLF). +RESP strings are either _simple_ or _bulk_. +Simple strings never contain carriage return (`\r`) or line feed (`\n`) characters. +Bulk strings can contain any binary data and may also be referred to as _binary_ or _blob_. +Note that bulk strings may be further encoded and decoded, e.g. with a wide multi-byte encoding, by the client. - +Aggregates, such as Arrays and Maps, can have varying numbers of sub-elements and nesting levels. -## RESP Simple Strings +The following table summarizes the RESP data types that Redis supports: -Simple Strings are encoded as follows: a plus character, followed by a string that cannot contain a CR or LF character (no newlines are allowed), and terminated by CRLF (that is "\r\n"). +| RESP data type | Minimal protocol version | Category | First byte | +| --- | --- | --- | --- | +| [Simple strings](#simple-strings) | RESP2 | Simple | `+` | +| [Simple Errors](#simple-errors) | RESP2 | Simple | `-` | +| [Integers](#integers) | RESP2 | Simple | `:` | +| [Bulk strings](#bulk-strings) | RESP2 | Aggregate | `$` | +| [Arrays](#arrays) | RESP2 | Aggregate | `*` | +| [Nulls](#nulls) | RESP3 | Simple | `_` | +| [Booleans](#booleans) | RESP3 | Simple | `#` | +| [Doubles](#doubles) | RESP3 | Simple | `,` | +| [Big numbers](#big-numbers) | RESP3 | Simple | `(` | +| [Bulk errors](#bulk-errors) | RESP3 | Aggregate | `!` | +| [Verbatim strings](#verbatim-strings) | RESP3 | Aggregate | `=` | +| [Maps](#maps) | RESP3 | Aggregate | `%` | +| [Sets](#sets) | RESP3 | Aggregate | `~` | +| [Pushes](#pushes) | RESP3 | Aggregate | `>` | -Simple Strings are used to transmit non binary-safe strings with minimal overhead. For example, many Redis commands reply with just "OK" on success. The RESP Simple String is encoded with the following 5 bytes: + - "+OK\r\n" +### Simple strings +Simple strings are encoded as a plus (`+`) character, followed by a string. +The string mustn't contain a CR (`\r`) or LF (`\n`) character and is terminated by CRLF (i.e., `\r\n`). -In order to send binary-safe strings, use RESP Bulk Strings instead. +Simple strings transmit short, non-binary strings with minimal overhead. +For example, many Redis commands reply with just "OK" on success. +The encoding of this Simple String is the following 5 bytes: -When Redis replies with a Simple String, a client library should respond with a string composed of the first character after the '+' -up to the end of the string, excluding the final CRLF bytes. + +OK\r\n - +When Redis replies with a simple string, a client library should return to the caller a string value composed of the first character after the `+` up to the end of the string, excluding the final CRLF bytes. -## RESP Errors +To send binary strings, use [bulk strings](#bulk-strings) instead. -RESP has a specific data type for errors. They are similar to -RESP Simple Strings, but the first character is a minus '-' character instead -of a plus. The real difference between Simple Strings and Errors in RESP is that clients treat errors -as exceptions, and the string that composes -the Error type is the error message itself. + + +### Simple errors +RESP has specific data types for errors. +Simple errors, or simply just errors, are similar to [simple strings](#simple-strings), but their first character is the minus (`-`) character. +The difference between simple strings and errors in RESP is that clients should treat errors as exceptions, whereas the string encoded in the error type is the error message itself. The basic format is: - "-Error message\r\n" + -Error message\r\n -Error replies are only sent when something goes wrong, for instance if -you try to perform an operation against the wrong data type, or if the command -does not exist. The client should raise an exception when it receives an Error reply. +Redis replies with an error only when something goes wrong, for example, when you try to operate against the wrong data type, or when the command does not exist. +The client should raise an exception when it receives an Error reply. The following are examples of error replies: - -ERR unknown command 'helloworld' + -ERR unknown command 'asdf' -WRONGTYPE Operation against a key holding the wrong kind of value -The first word after the "-", up to the first space or newline, represents -the kind of error returned. This is just a convention used by Redis and is not -part of the RESP Error format. +The first upper-case word after the `-`, up to the first space or newline, represents the kind of error returned. +This word is called an _error prefix_. +Note that the error prefix is a convention used by Redis rather than part of the RESP error type. -For example, `ERR` is the generic error, while `WRONGTYPE` is a more specific -error that implies that the client tried to perform an operation against the -wrong data type. This is called an **Error Prefix** and is a way to allow -the client to understand the kind of error returned by the server without checking the exact error message. +For example, in Redis, `ERR` is a generic error, whereas `WRONGTYPE` is a more specific error that implies that the client attempted an operation against the wrong data type. +The error prefix allows the client to understand the type of error returned by the server without checking the exact error message. -A client implementation may return different types of exceptions for different -errors or provide a generic way to trap errors by directly providing -the error name to the caller as a string. +A client implementation can return different types of exceptions for various errors, or provide a generic way for trapping errors by directly providing the error name to the caller as a string. -However, such a feature should not be considered vital as it is rarely useful, and a limited client implementation may simply return a generic error condition, such as `false`. +However, such a feature should not be considered vital as it is rarely useful. +Also, simpler client implementations can return a generic error value, such as `false`. -## RESP Integers +### Integers +This type is a CRLF-terminated string that represents a signed, base-10, 64-bit integer. -This type is just a CRLF-terminated string that represents an integer, -prefixed by a ":" byte. For example, ":0\r\n" and ":1000\r\n" are integer replies. +RESP encodes integers in the following way: -Many Redis commands return RESP Integers, like `INCR`, `LLEN`, and `LASTSAVE`. + :[<+|->]\r\n -There is no special meaning for the returned integer. It is just an -incremental number for `INCR`, a UNIX time for `LASTSAVE`, and so forth. However, -the returned integer is guaranteed to be in the range of a signed 64-bit integer. +* The colon (`:`) as the first byte. +* An optional plus (`+`) or minus (`-`) as the sign. +* One or more decimal digits (`0`..`9`) as the integer's unsigned, base-10 value. +* The CRLF terminator. -Integer replies are also used in order to return true or false. -For instance, commands like `EXISTS` or `SISMEMBER` will return 1 for true -and 0 for false. +For example, `:0\r\n` and `:1000\r\n` are integer replies (of zero and one thousand, respectively). -Other commands like `SADD`, `SREM`, and `SETNX` will return 1 if the operation -was actually performed and 0 otherwise. +Many Redis commands return RESP integers, including `INCR`, `LLEN`, and `LASTSAVE`. +An integer, by itself, has no special meaning other than in the context of the command that returned it. +For example, it is an incremental number for `INCR`, a UNIX timestamp for `LASTSAVE`, and so forth. +However, the returned integer is guaranteed to be in the range of a signed 64-bit integer. -The following commands will reply with an integer: `SETNX`, `DEL`, -`EXISTS`, `INCR`, `INCRBY`, `DECR`, `DECRBY`, `DBSIZE`, `LASTSAVE`, -`RENAMENX`, `MOVE`, `LLEN`, `SADD`, `SREM`, `SISMEMBER`, `SCARD`. +In some cases, integers can represent true and false Boolean values. +For instance, `SISMEMBER` returns 1 for true and 0 for false. + +Other commands, including `SADD`, `SREM`, and `SETNX`, return 1 when the data changes and 0 otherwise. - -## RESP Bulk Strings +### Bulk strings +A bulk string represents a single binary string. +The string can be of any size, but by default, Redis limits it to 512 MB (see the `proto-max-bulk-len` configuration directive). -Bulk Strings are used in order to represent a single binary-safe -string up to 512 MB in length. +RESP encodes bulk strings in the following way: -Bulk Strings are encoded in the following way: + $\r\n\r\n -* A "$" byte followed by the number of bytes composing the string (a prefixed length), terminated by CRLF. -* The actual string data. +* The dollar sign (`$`) as the first byte. +* One or more decimal digits (`0`..`9`) as the string's length, in bytes, as an unsigned, base-10 value. +* The CRLF terminator. +* The data. * A final CRLF. So the string "hello" is encoded as follows: - "$5\r\nhello\r\n" + $5\r\nhello\r\n -An empty string is encoded as: +The empty string's encoding is: - "$0\r\n\r\n" + $0\r\n\r\n -RESP Bulk Strings can also be used in order to signal non-existence of a value -using a special format to represent a Null value. In this -format, the length is -1, and there is no data. Null is represented as: + + +#### Null bulk strings +Whereas RESP3 has a dedicated data type for [null values](#nulls), RESP2 has no such type. +Instead, due to historical reasons, the representation of null values in RESP2 is via predetermined forms of the [bulk strings](#bulk-strings) and [arrays](#arrays) types. - "$-1\r\n" +The null bulk string represents a non-existing value. +The `GET` command returns the Null Bulk String when the target key doesn't exist. -This is called a **Null Bulk String**. +It is encoded as a bulk string with the length of negative one (-1), like so: + + $-1\r\n -The client library API should not return an empty string, but a nil object, -when the server replies with a Null Bulk String. -For example, a Ruby library should return 'nil' while a C library should -return NULL (or set a special flag in the reply object). +A Redis client should return a nil object when the server replies with a null bulk string rather than the empty string. +For example, a Ruby library should return `nil` while a C library should return `NULL` (or set a special flag in the reply object). -## RESP Arrays +### Arrays +Clients send commands to the Redis server as RESP arrays. +Similarly, some Redis commands that return collections of elements use arrays as their replies. +An example is the `LRANGE` command that returns elements of a list. -Clients send commands to the Redis server using RESP Arrays. Similarly, -certain Redis commands, that return collections of elements to the client, -use RESP Arrays as their replies. An example is the `LRANGE` command that -returns elements of a list. +RESP Arrays' encoding uses the following format: -RESP Arrays are sent using the following format: + *\r\n... -* A `*` character as the first byte, followed by the number of elements in the array as a decimal number, followed by CRLF. -* An additional RESP type for every element of the Array. +* An asterisk (`*`) as the first byte. +* One or more decimal digits (`0`..`9`) as the number of elements in the array as an unsigned, base-10 value. +* The CRLF terminator. +* An additional RESP type for every element of the array. So an empty Array is just the following: - "*0\r\n" + *0\r\n -While an array of two RESP Bulk Strings "hello" and "world" is encoded as: +Whereas the encoding of an array consisting of the two bulk strings "hello" and "world" is: - "*2\r\n$5\r\nhello\r\n$5\r\nworld\r\n" + *2\r\n$5\r\nhello\r\n$5\r\nworld\r\n -As you can see after the `*CRLF` part prefixing the array, the other -data types composing the array are just concatenated one after the other. +As you can see, after the `*CRLF` part prefixing the array, the other data types that compose the array are concatenated one after the other. For example, an Array of three integers is encoded as follows: - "*3\r\n:1\r\n:2\r\n:3\r\n" + *3\r\n:1\r\n:2\r\n:3\r\n -Arrays can contain mixed types, so it's not necessary for the -elements to be of the same type. For instance, a list of four -integers and a bulk string can be encoded as follows: +Arrays can contain mixed data types. +For instance, the following encoding is of a list of four integers and a bulk string: *5\r\n :1\r\n @@ -218,28 +278,14 @@ integers and a bulk string can be encoded as follows: $5\r\n hello\r\n -(The reply was split into multiple lines for clarity). +(The raw RESP encoding is split into multiple lines for readability). -The first line the server sent is `*5\r\n` in order to specify that five -replies will follow. Then every reply constituting the items of the -Multi Bulk reply are transmitted. +The first line the server sent is `*5\r\n`. +This numeric value tells the client that five reply types are about to follow it. +Then, every successive reply constitutes an element in the array. -Null Arrays exist as well and are an alternative way to -specify a Null value (usually the Null Bulk String is used, but for historical -reasons we have two formats). - -For instance, when the `BLPOP` command times out, it returns a Null Array -that has a count of `-1` as in the following example: - - "*-1\r\n" - -A client library API should return a null object and not an empty Array when -Redis replies with a Null Array. This is necessary to distinguish -between an empty list and a different condition (for instance the timeout -condition of the `BLPOP` command). - -Nested arrays are possible in RESP. For example a nested array of two arrays -is encoded as follows: +All of the aggregate RESP types support nesting. +For example, a nested array of two arrays is encoded as follows: *2\r\n *3\r\n @@ -250,16 +296,38 @@ is encoded as follows: +Hello\r\n -World\r\n -(The format was split into multiple lines to make it easier to read). +(The raw RESP encoding is split into multiple lines for readability). + +The above encodes a two-elements array. +The first element is an array that, in turn, contains three integers (1, 2, 3). +The second element is another array containing a simple string and an error. + +{{% alert title="Multi bulk reply" color="info" %}} +In some places, the RESP Array type may be referred to as _multi bulk_. +The two are the same. +{{% /alert %}} + + + +#### Null arrays +Whereas RESP3 has a dedicated data type for [null values](#nulls), RESP2 has no such type. Instead, due to historical reasons, the representation of null values in RESP2 is via predetermined forms of the [Bulk Strings](#bulk-strings) and [arrays](#arrays) types. -The above RESP data type encodes a two-element Array consisting of an Array that contains three Integers (1, 2, 3) and an array of a Simple String and an Error. +Null arrays exist as an alternative way of representing a null value. +For instance, when the `BLPOP` command times out, it returns a null array. -## Null elements in Arrays +The encoding of a null array is that of an array with the length of -1, i.e.: -Single elements of an Array may be Null. This is used in Redis replies to signal that these elements are missing and not empty strings. This -can happen with the SORT command when used with the GET _pattern_ option -if the specified key is missing. Example of an Array reply containing a -Null element: + *-1\r\n + +When Redis replies with a null array, the client should return a null object rather than an empty array. +This is necessary to distinguish between an empty list and a different condition (for instance, the timeout condition of the `BLPOP` command). + +#### Null elements in arrays +Single elements of an array may be [null bulk string](#null-bulk-strings). +This is used in Redis replies to signal that these elements are missing and not empty strings. This can happen, for example, with the `SORT` command when used with the `GET pattern` option +if the specified key is missing. + +Here's an example of an array reply containing a null element: *3\r\n $5\r\n @@ -268,25 +336,287 @@ Null element: $5\r\n world\r\n -The second element is a Null. The client library should return something -like this: +Above, the second element is null. +The client library should return to its caller something like this: ["hello",nil,"world"] -Note that this is not an exception to what was said in the previous sections, but -an example to further specify the protocol. + + +### Nulls +The null data type represents non-existent values. + +Nulls' encoding is the underscore (`_`) character, followed by the CRLF terminator (`\r\n`). +Here's Null's raw RESP encoding: + + _\r\n + +{{% alert title="Null Bulk String, Null Arrays and Nulls" color="info" %}} +Due to historical reasons, RESP2 features two specially crafted values for representing null values of bulk strings and arrays. +This duality has always been a redundancy that added zero semantical value to the protocol itself. + +The null type, introduced in RESP3, aims to fix this wrong. +{{% /alert %}}}} + + + +### Booleans +RESP booleans are encoded as follows: + + #\r\n + +* The octothorpe character (`#`) as the first byte. +* A `t` character for true values, or an `f` character for false ones. +* The CRLF terminator. + + + +### Doubles +The Double RESP type encodes a double-precision floating point value. +Doubles are encoded as follows: + + ,[<+|->][.][[sign]]\r\n + +* The comma character (`,`) as the first byte. +* An optional plus (`+`) or minus (`-`) as the sign. +* One or more decimal digits (`0`..`9`) as an unsigned, base-10 integral value. +* An optional dot (`.`), followed by one or more decimal digits (`0`..`9`) as an unsigned, base-10 fractional value. +* An optional capital or lowercase letter E (`E` or `e`), followed by an optional plus (`+`) or minus (`-`) as the exponent's sign, ending with one or more decimal digits (`0`..`9`) as an unsigned, base-10 exponent value. +* The CRLF terminator. + +Here's the encoding of the number 1.23: + + ,1.23\r\n + +Because the fractional part is optional, the integer value of ten (10) can, therefore, be RESP-encoded both as an integer as well as a double: + + :10\r\n + ,10\r\n + +In such cases, the Redis client should return native integer and double values, respectively, providing that these types are supported by the language of its implementation. + +The positive infinity, negative infinity and NaN values are encoded as follows: + + ,inf\r\n + ,-inf\r\n + ,nan\r\n + + + +### Big numbers +This type can encode integer values outside the range of signed 64-bit integers. + +Big numbers use the following encoding: + + ([+|-]\r\n + +* The left parenthesis character (`(`) as the first byte. +* An optional plus (`+`) or minus (`-`) as the sign. +* One or more decimal digits (`0`..`9`) as an unsigned, base-10 value. +* The CRLF terminator. + +Example: + + (3492890328409238509324850943850943825024385\r\n + +Big numbers can be positive or negative but can't include fractionals. +Client libraries written in languages with a big number type should return a big number. +When big numbers aren't supported, the client should return a string and, when possible, signal to the caller that the reply is a big integer (depending on the API used by the client library). + + + +### Bulk errors +This type combines the purpose of [simple errors](#simple-errors) with the expressive power of [bulk strings](#bulk-strings). + +It is encoded as: + + !\r\n\r\n + +* An exclamation mark (`!`) as the first byte. +* One or more decimal digits (`0`..`9`) as the error's length, in bytes, as an unsigned, base-10 value. +* The CRLF terminator. +* The error itself. +* A final CRLF. + +As a convention, the error begins with an uppercase (space-delimited) word that conveys the error message. + +For instance, the error "SYNTAX invalid syntax" is represented by the following protocol encoding: -## Send commands to a Redis server + !21\r\n + SYNTAX invalid syntax\r\n -Now that you are familiar with the RESP serialization format, you can use it to help write a Redis client library. We can further specify -how the interaction between the client and the server works: +(The raw RESP encoding is split into multiple lines for readability). -* A client sends the Redis server a RESP Array consisting of only Bulk Strings. + + +### Verbatim strings +This type is similar to the [bulk string](#bulk-strings), with the addition of providing a hint about the data's encoding. + +A verbatim string's RESP encoding is as follows: + + =\r\n:\r\n + +* An equal sign (`=`) as the first byte. +* One or more decimal digits (`0`..`9`) as the string's total length, in bytes, as an unsigned, base-10 value. +* The CRLF terminator. +* Exactly three (3) bytes represent the data's encoding. +* The colon (`:`) character separates the encoding and data. +* The data. +* A final CRLF. + +Example: + + =15\r\n + txt:Some string\r\n + +(The raw RESP encoding is split into multiple lines for readability). + +Some client libraries may ignore the difference between this type and the string type and return a native string in both cases. +However, interactive clients, such as command line interfaces (e.g., [`redis-cli`](/docs/manual/cli)), can use this type and know that their output should be presented to the human user as is and without quoting the string. + +For example, the Redis command `INFO` outputs a report that includes newlines. +When using RESP3, `redis-cli` displays it correctly because it is sent as a Verbatim String reply (with its three bytes being "txt"). +When using RESP2, however, the `redis-cli` is hard-coded to look for the `INFO` command to ensure its correct display to the user. + + + +### Maps +The RESP map encodes a collection of key-value tuples, i.e., a dictionary or a hash. + +It is encoded as follows: + + %\r\n... + +* A percent character (`%`) as the first byte. +* One or more decimal digits (`0`..`9`) as the number of entries, or key-value tuples, in the map as an unsigned, base-10 value. +* The CRLF terminator. +* Two additional RESP types for every key and value in the map. + +For example, the following JSON object: + + { + "first": 1, + "second": 2 + } + +Can be encoded in RESP like so: + + %2\r\n + +first\r\n + :1\r\n + +second\r\n + :2\r\n + +(The raw RESP encoding is split into multiple lines for readability). + +Both map keys and values can be any of RESP's types. + +Redis clients should return the idiomatic dictionary type that their language provides. +However, low-level programming languages (such as C, for example) will likely return an array along with type information that indicates to the caller that it is a dictionary. + +{{% alert title="Map pattern in RESP2" color="info" %}} +RESP2 doesn't have a map type. +Maps in RESP2 are represented by arrays, in which each element is a key-value tuple. +Each tuple is an array with two elements, these being the key and the value. +{{% /alert %}} + + + +### Sets +Sets are somewhat like [Arrays](#arrays) but are unordered and should only contain unique elements. + +RESP set's encoding is: + + ~\r\n... + +* A tilde (`~`) as the first byte. +* One or more decimal digits (`0`..`9`) as the number of elements in the set as an unsigned, base-10 value. +* The CRLF terminator. +* An additional RESP type for every element of the Set. + +Clients should return the native set type if it is available in their programming language. +Alternatively, in the absence of a native set type, an array coupled with type information can be used (in C, for example). + + + +### Pushes +RESP's pushes contain out-of-band data. +They are an exception to the protocol's request-response model and provide a generic _push mode_ for connections. + +Push events are encoded similarly to [arrays](#arrays), differing only in their first byte: + + >\r\n... + +* A greater-than sign (`>`) as the first byte. +* One or more decimal digits (`0`..`9`) as the number of elements in the message as an unsigned, base-10 value. +* The CRLF terminator. +* An additional RESP type for every element of the push event. + +Pushed data may precede or follow any of RESP's data types but never inside them. +That means a client won't find push data in the middle of a map reply, for example. +It also means that pushed data may appear before or after a command's reply, as well as by itself (without calling any command). + +Clients should react to pushes by invoking a callback that implements their handling of the pushed data. + +## Client handshake +New RESP connections should begin the session by calling the `HELLO` command. +This practice accomplishes two things: + +1. It allows servers to be backward compatible with RESP2 versions. + This is needed in Redis to make the transition to version 3 of the protocol gentler. +2. The `HELLO` command returns information about the server and the protocol that the client can use for different goals. + +The `HELLO` command has the following high-level syntax: + + HELLO [optional-arguments] + +The first argument of the command is the protocol version we want the connection to be set. +By default, the connection starts in RESP2 mode. +If we specify a connection version that is too big and unsupported by the server, it should reply with a `-NOPROTO` error. Example: + + Client: HELLO 4 + Server: -NOPROTO sorry, this protocol version is not supported. + +At that point, the client may retry with a lower protocol version. + +Similarly, the client can easily detect a server that is only able to speak RESP2: + + Client: HELLO 3 + Server: -ERR unknown command 'HELLO' + +The client can then proceed and use RESP2 to communicate with the server. + +Note that even if the protocol's version is supported, the `HELLO` command may return an error, perform no action and remain in RESP2 mode. +For example, when used with invalid authentication credentials in the command's optional `!AUTH` clause: + + Client: HELLO 3 AUTH default mypassword + Server: -ERR invalid password + (the connection remains in RESP2 mode) + +A successful reply to the `HELLO` command is a map reply. +The information in the reply is partly server-dependent, but certain fields are mandatory for all the RESP3 implementations: +* **server**: "redis" (or other software name). +* **version**: the server's version. +* **proto**: the highest supported version of the RESP protocol. + +In Redis' RESP3 implementation, the following fields are also emitted: + +* **id**: the connection's identifier (ID). +* **mode**: "standalone", "sentinel" or "cluster". +* **role**: "master" or "replica". +* **modules**: list of loaded modules as an Array of Bulk Strings. + +## Sending commands to a Redis server +Now that you are familiar with the RESP serialization format, you can use it to help write a Redis client library. +We can further specify how the interaction between the client and the server works: + +* A client sends the Redis server an [array](#arrays) consisting of only bulk strings. * A Redis server replies to clients, sending any valid RESP data type as a reply. -So for example a typical interaction could be the following. +So, for example, a typical interaction could be the following. -The client sends the command **LLEN mylist** in order to get the length of the list stored at key *mylist*. Then the server replies with an Integer reply as in the following example (C: is the client, S: the server). +The client sends the command `LLEN mylist` to get the length of the list stored at the key _mylist_. +Then the server replies with an [integer](#integers) reply as in the following example (`C:` is the client, `S:` the server). C: *2\r\n C: $4\r\n @@ -299,51 +629,44 @@ The client sends the command **LLEN mylist** in order to get the length of the l As usual, we separate different parts of the protocol with newlines for simplicity, but the actual interaction is the client sending `*2\r\n$4\r\nLLEN\r\n$6\r\nmylist\r\n` as a whole. ## Multiple commands and pipelining - -A client can use the same connection in order to issue multiple commands. -Pipelining is supported so multiple commands can be sent with a single -write operation by the client, without the need to read the server reply -of the previous command before issuing the next one. +A client can use the same connection to issue multiple commands. +Pipelining is supported, so multiple commands can be sent with a single write operation by the client. +The client can skip reading replies and continue to send the commands one after the other. All the replies can be read at the end. For more information, see [Pipelining](/topics/pipelining). ## Inline commands +Sometimes you may need to send a command to the Redis server but only have `telnet` available. +While the Redis protocol is simple to implement, it is not ideal for interactive sessions, and `redis-cli` may not always be available. +For this reason, Redis also accepts commands in the _inline command_ format. -Sometimes you may need to send a command -to the Redis server but only have `telnet` available. While the Redis protocol is simple to implement, it is -not ideal to use in interactive sessions, and `redis-cli` may not always be -available. For this reason, Redis also accepts commands in the **inline command** format. - -The following is an example of a server/client chat using an inline command -(the server chat starts with S:, the client chat with C:) +The following example demonstrates a server/client exchange using an inline command (the server chat starts with `S:`, the client chat with `C:`): C: PING S: +PONG -The following is an example of an inline command that returns an integer: +Here's another example of an inline command where the server returns an integer: C: EXISTS somekey S: :0 -Basically, you write space-separated arguments in a telnet session. -Since no command starts with `*` that is instead used in the unified request -protocol, Redis is able to detect this condition and parse your command. +Basically, to issue an inline command, you write space-separated arguments in a telnet session. +Since no command starts with `*` (the identifying byte of RESP Arrays), Redis detects this condition and parses your command inline. -## High performance parser for the Redis protocol +## High-performance parser for the Redis protocol -While the Redis protocol is human readable and easy to implement, it can -be implemented with a performance similar to that of a binary protocol. +While the Redis protocol is human-readable and easy to implement, its implementation can exhibit performance similar to that of a binary protocol. -RESP uses prefixed lengths to transfer bulk data, so there is -never a need to scan the payload for special characters, like with JSON, nor to quote the payload that needs to be sent to the -server. +RESP uses prefixed lengths to transfer bulk data. +That makes scanning the payload for special characters unnecessary (unlike parsing JSON, for example). +For the same reason, quoting and escaping the payload isn't needed. -The Bulk and Multi Bulk lengths can be processed with code that performs -a single operation per character while at the same time scanning for the -CR character, like the following C code: +Reading the length of aggregate types (for example, bulk strings or arrays) can be processed with code that performs a single operation per character while at the same time scanning for the CR character. -``` +Example (in C): + +```c #include int main(void) { @@ -362,11 +685,17 @@ int main(void) { } ``` -After the first CR is identified, it can be skipped along with the following -LF without any processing. Then the bulk data can be read using a single -read operation that does not inspect the payload in any way. Finally, -the remaining CR and LF characters are discarded without any processing. +After the first CR is identified, it can be skipped along with the following LF without further processing. +Then, the bulk data can be read with a single read operation that doesn't inspect the payload in any way. +Finally, the remaining CR and LF characters are discarded without additional processing. + +While comparable in performance to a binary protocol, the Redis protocol is significantly more straightforward to implement in most high-level languages, reducing the number of bugs in client software. + +## Tips for Redis client authors + +* For testing purposes, use [Lua's type conversions](/topics/lua-api#lua-to-resp3-type-conversion) to have Redis reply with any RESP2/RESP3 needed. + As an example, a RESP3 double can be generated like so: + ``` + EVAL "return { double = tonumber(ARGV[1]) }" 0 1e0 + ``` -While comparable in performance to a binary protocol, the Redis protocol is -significantly simpler to implement in most high-level languages, -reducing the number of bugs in client software. diff --git a/wordlist b/wordlist index 504bc19b16..60fe619c87 100644 --- a/wordlist +++ b/wordlist @@ -70,6 +70,7 @@ BigNumber BitOp Bitfields Bitwise +Booleans brpop C1 C2 @@ -271,6 +272,7 @@ NTP NUMA NX NaN +NaNs Nehalem node-redis NoSQL @@ -532,6 +534,7 @@ bitop bitwise bitwise bool +booleans breakpoint broadcasted bt @@ -655,6 +658,7 @@ foo0 foo1 foo2 formatter +fractionals frequencyonly fsSL fsync @@ -794,6 +798,7 @@ node-redlock noeviction non-TCP non-TLS +non-loopback non-reachability non-virtualized nonprintable @@ -807,6 +812,7 @@ numkeys nullarray nx observability +octothorpe odown ok oldval @@ -984,6 +990,7 @@ suboptimal subsequence substring sudo +superset swapdb swappability syncd @@ -998,6 +1005,7 @@ tmp tmux toolkits topologies +tonumber tradeoff tradeoffs transactional @@ -1024,6 +1032,7 @@ unregisters untrusted untuned unwatches +uppercased urandom used_memory_scripts_eval userSession