merge nips 12, 16, 20 and 33 into nip 01 (#703)

Co-authored-by: Viktor Vsk <me@viktorvsk.com>

01.md

@@ -4,7 +4,7 @@ NIP-01
 Basic protocol flow description
 -------------------------------
 
-`draft` `mandatory` `author:fiatjaf` `author:distbit` `author:scsibug` `author:kukks` `author:jb55` `author:semisol`
+`draft` `mandatory` `author:fiatjaf` `author:distbit` `author:scsibug` `author:kukks` `author:jb55` `author:semisol` `author:cameri` `author:Giszmo`
 
 This NIP defines the basic protocol that should be implemented by everybody. New NIPs may add new optional (or mandatory) fields and messages and features to the structures and flows described here.
 
@@ -19,11 +19,10 @@ The only object type that exists is the `event`, which has the following format
   "id": <32-bytes lowercase hex-encoded sha256 of the serialized event data>,
   "pubkey": <32-bytes lowercase hex-encoded public key of the event creator>,
   "created_at": <unix timestamp in seconds>,
-  "kind": <integer>,
+  "kind": <integer between 0 and 65535>,
   "tags": [
-    ["e", <32-bytes hex of the id of another event>, <recommended relay URL>],
-    ["p", <32-bytes hex of a pubkey>, <recommended relay URL>],
-    ... // other kinds of tags may be included later
+    [<arbitrary string>...],
+    ...
   ],
   "content": <arbitrary string>,
   "sig": <64-bytes hex of the signature of the sha256 hash of the serialized event data, which is the same as the "id" field>
@@ -43,9 +42,57 @@ To obtain the `event.id`, we `sha256` the serialized event. The serialization is
 ]
 ```
 
+### Tags
+
+Each tag is an array of strings of arbitrary size, with some conventions around them. Take a look at the example below:
+
+```json
+{
+  ...,
+  "tags": [
+    ["e", "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36", "wss://nostr.example.com"],
+    ["p", "f7234bd4c1394dda46d09f35bd384dd30cc552ad5541990f98844fb06676e9ca"],
+    ["a", "30023:f7234bd4c1394dda46d09f35bd384dd30cc552ad5541990f98844fb06676e9ca:abcd", "wss://nostr.example.com"],
+    ["alt", "reply"],
+    ...
+  ],
+  ...
+}
+```
+
+The first element of the tag array is referred to as the tag _name_ or _key_ and the second as the tag _value_. So we can safely say that the event above has an `e` tag set to `"5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"`, an `alt` tag set to `"reply"` and so on. All elements after the second do not have a conventional name.
+
+This NIP defines 3 standard tags that can be used accross all event kinds with the same meaning. They are as follows:
+
+- The `e` tag, used to refer to an event: `["e", <32-bytes hex of the id of another event>, <recommended relay URL, optional>]`
+- The `p` tag, used to refer to another user: `["p", <32-bytes hex of a pubkey>, <recommended relay URL, optional>]`
+- The `a` tag, used to refer to a parameterized replaceable event: `["a", <kind integer>:<32-bytes hex of a pubkey>:<d tag value>, <recommended relay URL, optional>]`
+
+As a convention, all single-letter key tags are expected to be indexed by relays, such that it is possible, for example, to query or subscribe to events that reference the event `"5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"` by using the `{"#e": "5c83da77af1dec6d7289834998ad7aafbd9e2191396d75ec3cc27f5a77226f36"}` filter.
+
+### Kinds
+
+Kinds specify how clients should interpret the meaning of each event and the other fields of each event (e.g. an `"r"` tag may have a meaning in an event of kind 1 and an entirely different meaning in an event of kind 10002). Each NIP may define the meaning of a set of kinds that weren't defined elsewhere. This NIP defines two basic kinds:
+
+- `0`: **metadata**: the `content` is set to a stringified JSON object `{name: <username>, about: <string>, picture: <url, string>}` describing the user who created the event. A relay may delete older events once it gets a new one for the same pubkey.
+- `1`: **text note**: the `content` is set to the **plaintext** content of a note (anything the user wants to say). Content that must be parsed, such as Markdown and HTML, should not be used. Clients should also not parse content as those.
+
+And also a convention for kind ranges that allow for easier experimentation and flexibility of relay implementation:
+
+- for kind `n` such that `1000 <= n < 10000`, events are **regular**, which means they're all expected to be stored by relays.
+- for kind `n` such that `10000 <= n < 20000`, events are **replaceable**, which means that, for each combination of `pubkey` and `kind`, only the latest event is expected to be stored by relays, older versions are expected to be discarded.
+- for kind `n` such that `20000 <= n < 30000`, events are **ephemeral**, which means they are not expected to be stored by relays.
+- for kind `n` such that `30000 <= n < 40000`, events are **parameterized replaceable**, which means that, for each combination of `pubkey`, `kind` and the `d` tag, only the latest event is expected to be stored by relays, older versions are expected to be discarded.
+
+These are just conventions and relay implementations may differ.
+
 ## Communication between clients and relays
 
-Relays expose a websocket endpoint to which clients can connect.
+Relays expose a websocket endpoint to which clients can connect. Clients should open a single websocket connection to each relay and use it for all their subscriptions.
+
+### Meaning of WebSocket status codes
+
+- When a websocket is closed by the relay with a status code `4000` that means the client shouldn't try to connect again.
 
 ### From client to relay: sending events and creating subscriptions
 
@@ -61,11 +108,10 @@ Clients can send 3 types of messages, which must be JSON arrays, according to th
 
 ```json
 {
-  "ids": <a list of event ids or prefixes>,
-  "authors": <a list of pubkeys or prefixes, the pubkey of an event must be one of these>,
+  "ids": <a list of event ids>,
+  "authors": <a list of lowercase pubkeys, the pubkey of an event must be one of these>,
   "kinds": <a list of a kind numbers>,
-  "#e": <a list of event ids that are referenced in an "e" tag>,
-  "#p": <a list of pubkeys that are referenced in a "p" tag>,
+  "#<single-letter>": <a list of event ids that are referenced in the tag specified by the single letter>,
   "since": <an integer unix timestamp in seconds, events must be newer than this to pass>,
   "until": <an integer unix timestamp in seconds, events must be older than this to pass>,
   "limit": <maximum number of events to be returned in the initial query>
@@ -74,42 +120,38 @@ Clients can send 3 types of messages, which must be JSON arrays, according to th
 
 Upon receiving a `REQ` message, the relay SHOULD query its internal database and return events that match the filter, then store that filter and send again all future events it receives to that same websocket until the websocket is closed. The `CLOSE` event is received with the same `<subscription_id>` or a new `REQ` is sent using the same `<subscription_id>`, in which case it should overwrite the previous subscription.
 
-Filter attributes containing lists (such as `ids`, `kinds`, or `#e`) are JSON arrays with one or more values.  At least one of the array's values must match the relevant field in an event for the condition itself to be considered a match.  For scalar event attributes such as `kind`, the attribute from the event must be contained in the filter list.  For tag attributes such as `#e`, where an event may have multiple values, the event and filter condition values must have at least one item in common.
+Filter attributes containing lists (`ids`, `authors`, `kinds` and tag filters like `#e`) are JSON arrays with one or more values. At least one of the arrays' values must match the relevant field in an event for the condition to be considered a match. For scalar event attributes such as `authors` and `kind`, the attribute from the event must be contained in the filter list. In the case of tag attributes such as `#e`, for which an event may have multiple values, the event and filter condition values must have at least one item in common.
 
-The `ids` and `authors` lists contain lowercase hexadecimal strings, which may either be an exact 64-character match, or a prefix of the event value.  A prefix match is when the filter string is an exact string prefix of the event value.  The use of prefixes allows for more compact filters where a large number of values are queried, and can provide some privacy for clients that may not want to disclose the exact authors or events they are searching for.
+The `ids`, `authors`, `#e` and `#p` filter lists MUST contain exact 64-character lowercase hex values.
 
 The `since` and `until` properties can be used to specify the time range of events returned in the subscription. If a filter includes the `since` property, events with `created_at` greater than or equal to `since` are considered to match the filter. The `until` property is similar except that `created_at` must be less than or equal to `until`. In short, an event matches a filter if `since <= created_at <= until` holds.
 
 All conditions of a filter that are specified must match for an event for it to pass the filter, i.e., multiple conditions are interpreted as `&&` conditions.
 
 A `REQ` message may contain multiple filters. In this case, events that match any of the filters are to be returned, i.e., multiple filters are to be interpreted as `||` conditions.
 
-The `limit` property of a filter is only valid for the initial query and can be ignored afterward. When `limit: n` is present it is assumed that the events returned in the initial query will be the last `n` events ordered by the `created_at`. It is safe to return less events than `limit` specifies, but it is expected that relays do not return (much) more events than requested so clients don't get unnecessarily overwhelmed by data.
+The `limit` property of a filter is only valid for the initial query and MUST be ignored afterwards. When `limit: n` is present it is assumed that the events returned in the initial query will be the last `n` events ordered by the `created_at`. It is safe to return less events than `limit` specifies, but it is expected that relays do not return (much) more events than requested so clients don't get unnecessarily overwhelmed by data.
 
 ### From relay to client: sending events and notices
 
-Relays can send 3 types of messages, which must also be JSON arrays, according to the following patterns:
+Relays can send 4 types of messages, which must also be JSON arrays, according to the following patterns:
 
   * `["EVENT", <subscription_id>, <event JSON as defined above>]`, used to send events requested by clients.
+  * `["OK", <event_id>, <true|false>, <message>]`, used to indicate acceptance or denial of an `EVENT` message.
   * `["EOSE", <subscription_id>]`, used to indicate the _end of stored events_ and the beginning of events newly received in real-time.
   * `["NOTICE", <message>]`, used to send human-readable error messages or other things to clients.
 
 This NIP defines no rules for how `NOTICE` messages should be sent or treated.
 
-`EVENT` messages MUST be sent only with a subscription ID related to a subscription previously initiated by the client (using the `REQ` message above).
-
-## Basic Event Kinds
-
-  - `0`: `set_metadata`: the `content` is set to a stringified JSON object `{name: <username>, about: <string>, picture: <url, string>}` describing the user who created the event. A relay may delete past `set_metadata` events once it gets a new one for the same pubkey.
-  - `1`: `text_note`: the `content` is set to the **plaintext** content of a note (anything the user wants to say). Content that must be parsed, such as Markdown and HTML, should not be used. Clients should also not parse content as those.
-  - `2`: `recommend_server`: the `content` is set to the URL (e.g., `wss://somerelay.com`) of a relay the event creator wants to recommend to its followers.
-
-A relay may choose to treat different message kinds differently, and it may or may not choose to have a default way to handle kinds it doesn't know about.
-
-## Other Notes:
-
-- Clients should not open more than one websocket to each relay. One channel can support an unlimited number of subscriptions, so clients should do that.
-- The `tags` array can store a case-sensitive tag name as the first element of each subarray, plus arbitrary information afterward (always as strings). This NIP defines `"p"` — meaning "pubkey", which points to a pubkey of someone that is referred to in the event —, and `"e"` — meaning "event", which points to the id of an event this event is quoting, replying to or referring to somehow. See [NIP-10](10.md) for a detailed description of "e" and "p" tags.
-- The `<recommended relay URL>` item present on the `"e"` and `"p"` tags is an optional (could be set to `""`) URL of a relay the client could attempt to connect to fetch the tagged event or other events from a tagged profile. It MAY be ignored, but it exists to increase censorship resistance and make the spread of relay addresses more seamless across clients.
-- Clients should use the created_at field to judge the age of a metadata event and completely replace older metadata events with newer metadata events regardless of the order in which they arrive.  Clients should not merge any filled fields within older metadata events into empty fields of newer metadata events.
-- When a websocket is closed by the relay with a status code 4000 that means the client shouldn't try to connect again.
+- `EVENT` messages MUST be sent only with a subscription ID related to a subscription previously initiated by the client (using the `REQ` message above).
+- `OK` messages MUST be sent in response to `EVENT` messages received from clients, they must have the 3rd parameter set to `true` when an event has been accepted by the relay, `false` otherwise. The 4th parameter MAY be empty when the 3rd is `true`, otherwise it MUST be a string containing a machine-readable single-word prefix followed by a `:` and then a human-readable message. The standardized machine-readable prefixes are: `duplicate`, `pow`, `blocked`, `rate-limited`, `invalid`, and `error` for when none of that fits. Some examples:
+
+  * `["OK", "b1a649ebe8...", true, ""]`
+  * `["OK", "b1a649ebe8...", true, "pow: difficulty 25>=24"]`
+  * `["OK", "b1a649ebe8...", true, "duplicate: already have this event"]`
+  * `["OK", "b1a649ebe8...", false, "blocked: you are banned from posting here"]`
+  * `["OK", "b1a649ebe8...", false, "blocked: please register your pubkey at https://my-expensive-relay.example.com"]`
+  * `["OK", "b1a649ebe8...", false, "rate-limited: slow down there chief"]`
+  * `["OK", "b1a649ebe8...", false, "invalid: event creation date is too far off from the current time. Is your system clock in sync?"]`
+  * `["OK", "b1a649ebe8...", false, "pow: difficulty 26 is less than 30"]`
+  * `["OK", "b1a649ebe8...", false, "error: could not connect to the database"]`

11.md

@@ -118,7 +118,7 @@ field of any event. This is a count of unicode characters. After
 serializing into JSON it may be larger (in bytes), and is still
 subject to the `max_message_length`, if defined.
 
-- `min_pow_difficulty`: new events will require at least this difficulty of PoW, 
+- `min_pow_difficulty`: new events will require at least this difficulty of PoW,
 based on [NIP-13](13.md), or they will be rejected by this server.
 
 - `auth_required`: this relay requires [NIP-42](42.md) authentication
@@ -129,7 +129,7 @@ Even if set to False, authentication may be required for specific actions.
 
 ### Event Retention ###
 
-There may be a cost associated with storing data forever, so relays 
+There may be a cost associated with storing data forever, so relays
 may wish to state retention times. The values stated here are defaults
 for unauthenticated users and visitors. Paid users would likely have
 other policies.
@@ -151,7 +151,7 @@ all, and preferably an error will be provided when those are received.
 }
 ```
 
-`retention` is a list of specifications: each will apply to either all kinds, or 
+`retention` is a list of specifications: each will apply to either all kinds, or
 a subset of kinds. Ranges may be specified for the kind field as a tuple of inclusive
 start and end values. Events of indicated kind (or all) are then limited to a `count`
 and/or time period.
@@ -161,8 +161,7 @@ a specific `kind` number, by giving a retention time of zero for those `kind` va
 While that is unfortunate, it does allow clients to discover servers that will
 support their protocol quickly via a single HTTP fetch.
 
-There is no need to specify retention times for _ephemeral events_ as defined
-in [NIP-16](16.md) since they are not retained.
+There is no need to specify retention times for _ephemeral events_ since they are not retained.
 
 
 ### Content Limitations ###
@@ -271,7 +270,7 @@ A URL pointing to an image to be used as an icon for the relay. Recommended to b
 As of 2 May 2023 the following `curl` command provided these results.
 
 	>curl -H "Accept: application/nostr+json" https://eden.nostr.land
-	
+
 	{"name":"eden.nostr.land",
 	"description":"Eden Nostr Land - Toronto 1-01",
 	"pubkey":"00000000827ffaa94bfea288c3dfce4422c794fbb96625b6b31e9049f729d700",
@@ -293,5 +292,5 @@ As of 2 May 2023 the following `curl` command provided these results.
 				  "payment_required":true},
 	"payments_url":"https://eden.nostr.land/invoices",
 	"fees":{"admission":[{"amount":5000000,"unit":"msats"}],
-	        "publication":[]}}, 
+	        "publication":[]}},
     "icon": "https://nostr.build/i/53866b44135a27d624e99c6165cabd76ac8f72797209700acb189fce75021f47.jpg"