From e3cbc8d32961783b81b596eb6035970dfff74da6 Mon Sep 17 00:00:00 2001 From: "Masih H. Derkani" Date: Wed, 15 Feb 2023 22:30:51 +0000 Subject: [PATCH 1/4] IPIP-379 Delegated Naming HTTP API Propose HTTP APIs for delegated naming system that enables offloading of naming system to other processes or servers. Relates to: - https://github.com/ipfs/specs/pull/377 Fixes #343 --- src/ipips/ipip-0337.md | 4 +- src/ipips/ipip-0379.md | 95 +++++++++++++++++ src/routing/http-ipns-routing.md | 177 +++++++++++++++++++++++++++++++ 3 files changed, 274 insertions(+), 2 deletions(-) create mode 100644 src/ipips/ipip-0379.md create mode 100644 src/routing/http-ipns-routing.md diff --git a/src/ipips/ipip-0337.md b/src/ipips/ipip-0337.md index 4438ef74c..d0d853885 100644 --- a/src/ipips/ipip-0337.md +++ b/src/ipips/ipip-0337.md @@ -70,7 +70,7 @@ To understand the design rationale, it is important to consider the concrete Ref So this API proposal makes the following changes: - The Delegated Content Routing API is defined using HTTP semantics, and can be implemented without introducing Reframe concepts nor IPLD -- There is a clear distinction between the RPC protocol (HTTP) and the API (Deleged Content Routing) +- There is a clear distinction between the RPC protocol (HTTP) and the API (Delegated Content Routing) - "Method names" and cache-relevant parameters are pushed into the URL path - Streaming support is removed, and default response size limits are added. - We will add streaming support in a subsequent IPIP, but we are trying to minimize the scope of this IPIP to what is immediately useful @@ -80,7 +80,7 @@ So this API proposal makes the following changes: - Multiaddrs use the [human-readable format](https://github.com/multiformats/multiaddr#specification) that is used in existing tools and Kubo CLI commands such as `ipfs id` or `ipfs swarm peers` - Byte array values, such as signatures, are multibase-encoded strings (with an `m` prefix indicating Base64) - The "Identify" method and "message groups" are not included -- The "GetIPNS" and "PutIPNS" methods are not included +- The "GetIPNS" and "PutIPNS" methods are not included. Instead, a separate Delegated IPNS HTTP API aims to firstly facilitate naming system delegation, and secondly, pave the way for future iterations of IPNS with less interdependency with content routing. For more information, see Delegated IPNS HTTP API (:cite[ipip-0379]). ### User benefit diff --git a/src/ipips/ipip-0379.md b/src/ipips/ipip-0379.md new file mode 100644 index 000000000..6a6ee670a --- /dev/null +++ b/src/ipips/ipip-0379.md @@ -0,0 +1,95 @@ +--- +title: "IPIP-0379: Delegated IPNS HTTP API" +date: 2023-02-13 +ipip: proposal +editors: + - name: Masih H. Derkani + github: masih + - name: Marcin Rataj + github: lidel + url: https://lidel.org/ +relatedIssues: + - https://github.com/ipfs/specs/issues/343 + - https://github.com/ipfs/specs/pull/337 + - https://github.com/ipfs/specs/pull/377 +order: 379 +tags: ['ipips'] +--- + +- Start Date: 2023-02-13 +- Related Issues: + +## Summary + +This IPIP specifies a set of HTTP APIs to offload naming system onto another process or server. + +## Motivation + +Expanding on the motivations of :cite[ipip-0337], the work here concentrates on delegation of _naming system_ over HTTP APIs. Naming is part of the core IPFS DHT functionality. +The performance of naming system over the IPFS DHT can suffer from long delays due to churn of records and quorum requirements. + +## HTTP API Specification + +See :cite[http-ipns-routing] specification. + +## Design rationale + +The rationale for delegated IPNS over HTTP APIs closely follows the reasoning listed in :cite[ipip-0337]. + +The document proposes the following: +- Use of HTTP semantics for publication and resolution of naming records. +- Preference for human-readable request and response encoding, such as JSON format +- Optional backward compatibility support for the existing ProtocolBuffer format using `Content-Type: application/vnd.ipfs.ipns-record` +- Use of extra headers in `OPTIONS` response to communicate the supported capabilities and limitations, namely: + - `X-Ipns-Allow-Max-Size` -- to signal maximum supported IPNS record size + - `X-Ipns-Allow-Protobuf` -- to signal whether the server supports ProtocolBuffer formatted records. +- Streaming interaction is not supported. + +### User benefit + +The ability of offload naming onto another process or server via an idiomatic and simple HTTP API opens up an opportunity to not only reduce the burden of naming system management, but also introduce alternative naming systems across other routing systems, namely IPNI. + +### Compatibility + +#### Backwards Compatibility + +##### Serialization Format + +:cite[ipns-record] use ProtocolBuffer serialisation format. +This format is widely in use in IPNS over PubSub and DHT routing systems. +One of the motivations of this document is to introduce simple to use HTTP APIs and ultimately reduce barrier for interaction across alternative systems. +Further, interoperability across the existing and HTTP APIs is also desirable in order to reduce the barrier for adoption of the delegated HTTP APIs. + +The specification here maintains backwards compatibility in terms of record serialisation, with preference for human-readable formats such as JSON. +To maximize interoperability with existing ecosystem, the canonical IPNS record serialization format :cite[ipns-record] (`0x0300`) can be requested with content type `application/vnd.ipfs.ipns-record`. + +##### Reframe + +See "Backwards Compatibility" section of :cite[ipip-0337]. + +#### Forwards Compatibility + +See "Forwards Compatibility" section of :cite[ipip-0337]. + +### Security + +All interaction over the APIs should use TLS to protect against third-party observation and tampering. +Additionally, the IPNS records are signed by the publisher's identity and contain sequence number to avoid replay attacks. + +To avoid Denial of Service attack, maximum IPNS record size of `10 KiB` applies. +Implements are permitted to set a lower limit. If lower than the default maximum, the limit should be discoverable via `OPTIONS` request with header key `X-Ipns-Allow-Max-Size` with value specified as the number of bytes. + +Similarly, a client may check if a server supports ProtocolBuffer formatted records by checking the `X-Ipns-Allow-Protobuf` header key in response to `OPTIONS` request. If present the header value must be either `true` or `false` the absence of the header indicates that ProtocolBuffer formatted records are not supported. + +Privacy in delegated IPNS is out of scope for this work. +- The usual JSON parsing rules apply. To prevent potential Denial of Service (DoS) attack, clients should ignore responses larger than 100 providers and introduce a byte size limit that is applicable to their use case. + +### Alternatives + +See: +- [Libp2p-based IPNS](https://github.com/ipfs/go-ipns). +- Reframe; find out more on "Design Rationale" section of :cite[ipip-0337]. + +### Copyright + +Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). diff --git a/src/routing/http-ipns-routing.md b/src/routing/http-ipns-routing.md new file mode 100644 index 000000000..bafd1c371 --- /dev/null +++ b/src/routing/http-ipns-routing.md @@ -0,0 +1,177 @@ +--- +title: Delegated IPNS HTTP API +description: > + Delegated IPNS is a mechanism for IPFS implementations to offload naming system to another process + or server. This includes naming resolution as well as publication of new naming records. This + document describes an HTTP API through which such functionality is facilitated. +date: 2023-03-22 +maturity: wip +editors: + - name: Masih H. Derkani + github: masih +order: 0 +tags: ['routing'] +--- + +Delegated IPNS is a mechanism for IPFS implementations to offload naming system to another process +or server. This includes naming resolution as well as publication of new naming records. This +document describes an HTTP API through which such functionality is facilitated. + +## API Specification + +The Delegated IPNS HTTP API uses the `application/json` content type by default. + +As such, human-readable encodings of types are preferred. This spec may be updated in the future +with a compact `application/cbor` encoding, in which case compact encodings of the various types +would be used. + +## Common Data Types + +### IPNS Record + +The following snippet outlines the JSON schema of IPNS records: + +```json +{ + "Signature": "", + "Payload": { + "Value": "", + "Sequence": 0, + "Validity": { + "EOL": { + "Timestamp": 0, + "AdvisoryTTL": 0 + } + }, + "PublicKey": "", + "ExtendedData": {} + } +} +``` + +Where: + +- `Signature` is the multibase-encoded signature of the sha256 hash of the `Payload` field, signed + using the private key that corresponds to the `PublicKey` in the `Payload` if present. And + Otherwise, the private key associcated to the IPNS record key. Signing details for specific key + types should + follow [libp2p/peerid specs](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#key-types), + unless stated otherwise. +- `Payload` is the content of the IPNS record as specified + by :cite[ipns-record] specification: +- `Value` is the string representation of the IPNS path, + e.g. `ipns/{ipns-key}`, `/ipns/example.com`, `/ipfs/baf...`, etc. +- `Sequence` represents the current version of the record starting from `0`. +- `Validity` captures the mechanism by which the record is validated. Each validity type reserves a + field key under this object. + - `EOL` donates that the validity type is EOL, containing: + - `Timestamp` represents the time in the future at which the record expires with nanoseconds + precision represented as an ASCII string that follows notation + from :cite[rfc3339]. + - `AdvisoryTTL` represents an optional field that hints at how long the record should be + cached. +- `PublicKey` represents the optional public key used to sign the record. This field is only + required if it cannot be extracted from the IPNS name, e.g. in the case of legacy RSA keys. +- `ExtendedData` represents the extensible data as arbitrary JSON object. + +## Versioning + +The path predix `/v1` donates the version number of the HTTP API. Backwards-incompatible change must +increment the version number. + +## API + +### `GET /naming/v1/records/{ipns-name}` + +**Path Parameters** + +- `ipns-name` the IPNS name to resolve. + +**Response Status Codes** + +- `200` (OK): indicates that the response body containing the IPNS record that corresponds to the + IPNS name. +- `404` (Not Found): indicates that no matching records are found. +- `400` (Bad Request): indicates that the given IPNS name is not valid. +- `429` (Too Many Requests): indicates that the caller is issuing requests too many request and may + retry after the time specified + at [Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) response + header has elapsed. +- `501` (Not Implemented): indicates that the server does not support resolution of IPNS records + +**Response Body** + +The response body contains the [IPNS record](#ipns-record). + +```json +{ + "Signature": "", + "Payload": { + "Value": "", + "Sequence": 0, + "Validity": { + "EOL": { + "Timestamp": 0, + "AdvisoryTTL": 0 + } + }, + "PublicKey": "", + "ExtendedData": {} + } +} +``` + +### `PUT /naming/v1/records/{ipns-name}` + +**Path Parameters** + +- `ipns-name` the IPNS name to publish. The name must match `Value` in request body. + +**Request Body** + +```json +{ + "Signature": "", + "Payload": { + "Value": "", + "Sequence": 0, + "Validity": { + "EOL": { + "Timestamp": 0, + "AdvisoryTTL": 0 + } + }, + "PublicKey": "", + "ExtendedData": {} + } +} +``` + +**Response Status Codes** + +- `200` (OK): indicates that the response body containing the IPNS record that corresponds to the + IPNS name. +- `404` (Not Found): indicates that no matching records are found. +- `400` (Bad Request): indicates that the given IPNS record is not valid. +- `429` (Too Many Requests): indicates that the caller is issuing requests too many request and may + retry after the time specified + at [Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) response + header has elapsed. +- `501` (Not Implemented): indicates that the server does not support publication of IPNS records + +## CORS and Web Browsers + +Browser interoperability requires implementations to support +[CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS). + +JavaScript client running on a third-party Origin must be able to send HTTP request to the endpoints +defined in this specification, and read the received values. This means HTTP server implementing +this API must: + +1. support [CORS preflight requests](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) sent as HTTP OPTIONS, and +2. always respond with headers that remove CORS limits, allowing every site to query the API for results: + +```plaintext +Access-Control-Allow-Origin: * +Access-Control-Allow-Methods: GET, PUT, OPTIONS +``` From 4a03ecdcb6dc29f828efec33de0ea774682f554b Mon Sep 17 00:00:00 2001 From: Henrique Dias Date: Thu, 18 May 2023 15:16:58 +0200 Subject: [PATCH 2/4] docs: merge delegated naming with Routing V1 Spec - suggestions from code review - removes application/json from ipip-379 - removes custom headers from ipip-379 - cleans duplicated errors - punctuation and others --- src/ipips/ipip-0337.md | 2 +- src/ipips/ipip-0379.md | 41 +++---- src/routing/http-ipns-routing.md | 177 ------------------------------- src/routing/http-routing-v1.md | 105 ++++++++++++------ 4 files changed, 87 insertions(+), 238 deletions(-) delete mode 100644 src/routing/http-ipns-routing.md diff --git a/src/ipips/ipip-0337.md b/src/ipips/ipip-0337.md index d0d853885..d7ba81d25 100644 --- a/src/ipips/ipip-0337.md +++ b/src/ipips/ipip-0337.md @@ -70,7 +70,7 @@ To understand the design rationale, it is important to consider the concrete Ref So this API proposal makes the following changes: - The Delegated Content Routing API is defined using HTTP semantics, and can be implemented without introducing Reframe concepts nor IPLD -- There is a clear distinction between the RPC protocol (HTTP) and the API (Delegated Content Routing) +- There is a clear distinction between the [Kubo RPC](https://docs.ipfs.tech/reference/kubo/rpc/) and the vendor-agnostic Routing V1 HTTP API (introduced in :cite[ipip-0377]). - "Method names" and cache-relevant parameters are pushed into the URL path - Streaming support is removed, and default response size limits are added. - We will add streaming support in a subsequent IPIP, but we are trying to minimize the scope of this IPIP to what is immediately useful diff --git a/src/ipips/ipip-0379.md b/src/ipips/ipip-0379.md index 6a6ee670a..7cd83d59f 100644 --- a/src/ipips/ipip-0379.md +++ b/src/ipips/ipip-0379.md @@ -12,13 +12,12 @@ relatedIssues: - https://github.com/ipfs/specs/issues/343 - https://github.com/ipfs/specs/pull/337 - https://github.com/ipfs/specs/pull/377 +xref: + - ipns-record order: 379 tags: ['ipips'] --- -- Start Date: 2023-02-13 -- Related Issues: - ## Summary This IPIP specifies a set of HTTP APIs to offload naming system onto another process or server. @@ -28,9 +27,9 @@ This IPIP specifies a set of HTTP APIs to offload naming system onto another pro Expanding on the motivations of :cite[ipip-0337], the work here concentrates on delegation of _naming system_ over HTTP APIs. Naming is part of the core IPFS DHT functionality. The performance of naming system over the IPFS DHT can suffer from long delays due to churn of records and quorum requirements. -## HTTP API Specification +## Detailed design -See :cite[http-ipns-routing] specification. +See :cite[http-routing-v1] specification. ## Design rationale @@ -38,11 +37,7 @@ The rationale for delegated IPNS over HTTP APIs closely follows the reasoning li The document proposes the following: - Use of HTTP semantics for publication and resolution of naming records. -- Preference for human-readable request and response encoding, such as JSON format -- Optional backward compatibility support for the existing ProtocolBuffer format using `Content-Type: application/vnd.ipfs.ipns-record` -- Use of extra headers in `OPTIONS` response to communicate the supported capabilities and limitations, namely: - - `X-Ipns-Allow-Max-Size` -- to signal maximum supported IPNS record size - - `X-Ipns-Allow-Protobuf` -- to signal whether the server supports ProtocolBuffer formatted records. +- Use of existing :ref[IPNS Record] serialization format through `Content-Type: application/vnd.ipfs.ipns-record`. - Streaming interaction is not supported. ### User benefit @@ -51,44 +46,32 @@ The ability of offload naming onto another process or server via an idiomatic an ### Compatibility -#### Backwards Compatibility +See "Compatibility" section of :cite[ipip-0337]. -##### Serialization Format +#### Serialization Format -:cite[ipns-record] use ProtocolBuffer serialisation format. +:cite[ipns-record] uses `application/vnd.ipfs.ipns-record` protobuf serialization format. This format is widely in use in IPNS over PubSub and DHT routing systems. One of the motivations of this document is to introduce simple to use HTTP APIs and ultimately reduce barrier for interaction across alternative systems. Further, interoperability across the existing and HTTP APIs is also desirable in order to reduce the barrier for adoption of the delegated HTTP APIs. -The specification here maintains backwards compatibility in terms of record serialisation, with preference for human-readable formats such as JSON. To maximize interoperability with existing ecosystem, the canonical IPNS record serialization format :cite[ipns-record] (`0x0300`) can be requested with content type `application/vnd.ipfs.ipns-record`. -##### Reframe - -See "Backwards Compatibility" section of :cite[ipip-0337]. - -#### Forwards Compatibility - -See "Forwards Compatibility" section of :cite[ipip-0337]. - ### Security All interaction over the APIs should use TLS to protect against third-party observation and tampering. -Additionally, the IPNS records are signed by the publisher's identity and contain sequence number to avoid replay attacks. - -To avoid Denial of Service attack, maximum IPNS record size of `10 KiB` applies. -Implements are permitted to set a lower limit. If lower than the default maximum, the limit should be discoverable via `OPTIONS` request with header key `X-Ipns-Allow-Max-Size` with value specified as the number of bytes. +Additionally, the IPNS records must be validated according to the rules stated in :cite[ipns-record] before further processing. -Similarly, a client may check if a server supports ProtocolBuffer formatted records by checking the `X-Ipns-Allow-Protobuf` header key in response to `OPTIONS` request. If present the header value must be either `true` or `false` the absence of the header indicates that ProtocolBuffer formatted records are not supported. +To avoid Denial of Service attack, maximum IPNS record size defined in :cite[ipns-record] applies. Privacy in delegated IPNS is out of scope for this work. -- The usual JSON parsing rules apply. To prevent potential Denial of Service (DoS) attack, clients should ignore responses larger than 100 providers and introduce a byte size limit that is applicable to their use case. ### Alternatives See: + - [Libp2p-based IPNS](https://github.com/ipfs/go-ipns). -- Reframe; find out more on "Design Rationale" section of :cite[ipip-0337]. +- Reframe; it was deprecated and now removed. Historical reasons can be found in :cite[ipip-0337]. ### Copyright diff --git a/src/routing/http-ipns-routing.md b/src/routing/http-ipns-routing.md deleted file mode 100644 index bafd1c371..000000000 --- a/src/routing/http-ipns-routing.md +++ /dev/null @@ -1,177 +0,0 @@ ---- -title: Delegated IPNS HTTP API -description: > - Delegated IPNS is a mechanism for IPFS implementations to offload naming system to another process - or server. This includes naming resolution as well as publication of new naming records. This - document describes an HTTP API through which such functionality is facilitated. -date: 2023-03-22 -maturity: wip -editors: - - name: Masih H. Derkani - github: masih -order: 0 -tags: ['routing'] ---- - -Delegated IPNS is a mechanism for IPFS implementations to offload naming system to another process -or server. This includes naming resolution as well as publication of new naming records. This -document describes an HTTP API through which such functionality is facilitated. - -## API Specification - -The Delegated IPNS HTTP API uses the `application/json` content type by default. - -As such, human-readable encodings of types are preferred. This spec may be updated in the future -with a compact `application/cbor` encoding, in which case compact encodings of the various types -would be used. - -## Common Data Types - -### IPNS Record - -The following snippet outlines the JSON schema of IPNS records: - -```json -{ - "Signature": "", - "Payload": { - "Value": "", - "Sequence": 0, - "Validity": { - "EOL": { - "Timestamp": 0, - "AdvisoryTTL": 0 - } - }, - "PublicKey": "", - "ExtendedData": {} - } -} -``` - -Where: - -- `Signature` is the multibase-encoded signature of the sha256 hash of the `Payload` field, signed - using the private key that corresponds to the `PublicKey` in the `Payload` if present. And - Otherwise, the private key associcated to the IPNS record key. Signing details for specific key - types should - follow [libp2p/peerid specs](https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#key-types), - unless stated otherwise. -- `Payload` is the content of the IPNS record as specified - by :cite[ipns-record] specification: -- `Value` is the string representation of the IPNS path, - e.g. `ipns/{ipns-key}`, `/ipns/example.com`, `/ipfs/baf...`, etc. -- `Sequence` represents the current version of the record starting from `0`. -- `Validity` captures the mechanism by which the record is validated. Each validity type reserves a - field key under this object. - - `EOL` donates that the validity type is EOL, containing: - - `Timestamp` represents the time in the future at which the record expires with nanoseconds - precision represented as an ASCII string that follows notation - from :cite[rfc3339]. - - `AdvisoryTTL` represents an optional field that hints at how long the record should be - cached. -- `PublicKey` represents the optional public key used to sign the record. This field is only - required if it cannot be extracted from the IPNS name, e.g. in the case of legacy RSA keys. -- `ExtendedData` represents the extensible data as arbitrary JSON object. - -## Versioning - -The path predix `/v1` donates the version number of the HTTP API. Backwards-incompatible change must -increment the version number. - -## API - -### `GET /naming/v1/records/{ipns-name}` - -**Path Parameters** - -- `ipns-name` the IPNS name to resolve. - -**Response Status Codes** - -- `200` (OK): indicates that the response body containing the IPNS record that corresponds to the - IPNS name. -- `404` (Not Found): indicates that no matching records are found. -- `400` (Bad Request): indicates that the given IPNS name is not valid. -- `429` (Too Many Requests): indicates that the caller is issuing requests too many request and may - retry after the time specified - at [Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) response - header has elapsed. -- `501` (Not Implemented): indicates that the server does not support resolution of IPNS records - -**Response Body** - -The response body contains the [IPNS record](#ipns-record). - -```json -{ - "Signature": "", - "Payload": { - "Value": "", - "Sequence": 0, - "Validity": { - "EOL": { - "Timestamp": 0, - "AdvisoryTTL": 0 - } - }, - "PublicKey": "", - "ExtendedData": {} - } -} -``` - -### `PUT /naming/v1/records/{ipns-name}` - -**Path Parameters** - -- `ipns-name` the IPNS name to publish. The name must match `Value` in request body. - -**Request Body** - -```json -{ - "Signature": "", - "Payload": { - "Value": "", - "Sequence": 0, - "Validity": { - "EOL": { - "Timestamp": 0, - "AdvisoryTTL": 0 - } - }, - "PublicKey": "", - "ExtendedData": {} - } -} -``` - -**Response Status Codes** - -- `200` (OK): indicates that the response body containing the IPNS record that corresponds to the - IPNS name. -- `404` (Not Found): indicates that no matching records are found. -- `400` (Bad Request): indicates that the given IPNS record is not valid. -- `429` (Too Many Requests): indicates that the caller is issuing requests too many request and may - retry after the time specified - at [Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) response - header has elapsed. -- `501` (Not Implemented): indicates that the server does not support publication of IPNS records - -## CORS and Web Browsers - -Browser interoperability requires implementations to support -[CORS](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS). - -JavaScript client running on a third-party Origin must be able to send HTTP request to the endpoints -defined in this specification, and read the received values. This means HTTP server implementing -this API must: - -1. support [CORS preflight requests](https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request) sent as HTTP OPTIONS, and -2. always respond with headers that remove CORS limits, allowing every site to query the API for results: - -```plaintext -Access-Control-Allow-Origin: * -Access-Control-Allow-Methods: GET, PUT, OPTIONS -``` diff --git a/src/routing/http-routing-v1.md b/src/routing/http-routing-v1.md index 3015d66b3..747711225 100644 --- a/src/routing/http-routing-v1.md +++ b/src/routing/http-routing-v1.md @@ -1,23 +1,27 @@ --- title: Routing V1 HTTP API description: > - Delegated content routing is a mechanism for IPFS implementations to use for - offloading content routing to another process. This specification describes + Delegated routing is a mechanism for IPFS implementations to use for offloading + content routing and naming to another process/server. This specification describes an HTTP API for delegated content routing. date: 2023-03-22 maturity: reliable editors: - name: Gus Eggert - github: guseggert + github: guseggert + - name: Masih H. Derkani + github: masih +xref: + - ipns-record order: 0 tags: ['routing'] --- -"Delegated content routing" is a mechanism for IPFS implementations to use for offloading content routing to another process/server. This specification describes an HTTP API for delegated content routing. +Delegated routing is a mechanism for IPFS implementations to use for offloading content routing and naming to another process/server. This specification describes an HTTP API for delegated content routing. ## API Specification -The Delegated Content Routing Routing HTTP API uses the `application/json` content type by default. +The Routing HTTP API uses the `application/json` content type by default. For :ref[IPNS Names], the verifiable [`application/vnd.ipfs.ipns-record`][application/vnd.ipfs.ipns-record] content type is used. As such, human-readable encodings of types are preferred. This specification may be updated in the future with a compact `application/cbor` encoding, in which case compact encodings of the various types would be used. @@ -63,27 +67,35 @@ Where: Specifications for some transfer protocols are provided in the "Transfer Protocols" section. +### IPNS Records + +:ref[IPNS Records] are serialized using the verifiable [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) format. For more details, read the specification :cite[ipns-record]. + ## API -### `GET /routing/v1/providers/{CID}` +### `GET /routing/v1/providers/{cid}` + +#### Path Parameters -#### Response codes +- `cid` is the [CID](https://github.com/multiformats/cid) to fetch provider records for. -- `200` (OK): the response body contains 0 or more records -- `404` (Not Found): must be returned if no matching records are found -- `422` (Unprocessable Entity): request does not conform to schema or semantic constraints +#### Response Status Codes + +- `200` (OK): the response body contains 0 or more records. +- `404` (Not Found): must be returned if no matching records are found. +- `422` (Unprocessable Entity): request does not conform to schema or semantic constraints. #### Response Body ```json { - "Providers": [ - { - "Protocol": "", - "Schema": "", - ... - } - ] + "Providers": [ + { + "Protocol": "", + "Schema": "", + ... + } + ] } ``` @@ -91,6 +103,36 @@ Response limit: 100 providers Each object in the `Providers` list is a *read provider record*. +### `GET /routing/v1/ipns/{name}` + +#### Path Parameters + +- `name` is the :ref[IPNS Name] to resolve. + +#### Response Status Codes + +- `200` (OK): the response body contains the :ref[IPNS Record] for the given :ref[IPNS Name]. +- `404` (Not Found): must be returned if no matching records are found. + +#### Response Body + +The response body contains a [`application/vnd.ipfs.ipns-record`][application/vnd.ipfs.ipns-record] serialized :ref[IPNS Record]. + +### `PUT /routing/v1/ipns/{name}` + +#### Path Parameters + +- `name` is the :ref[IPNS Name] to publish. + +#### Request Body + +The content body must be a [`application/vnd.ipfs.ipns-record`][application/vnd.ipfs.ipns-record] serialized :ref[IPNS Record], which matches the `name` path parameter. + +#### Response Status Codes + +- `200` (OK): the provided :ref[IPNS Record] was published. +- `400` (Bad Request): the provided :ref[IPNS Record] or :ref[IPNS Name] are not valid. + ## Pagination This API does not support pagination, but optional pagination can be added in a backwards-compatible spec update. @@ -101,9 +143,9 @@ This API does not currently support streaming, however it can be added in the fu ## Error Codes -- `501` (Not Implemented): must be returned if a method/path is not supported -- `429` (Too Many Requests): may be returned along with optional [Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header to indicate to the caller that it is issuing requests too quickly -- `400` (Bad Request): must be returned if an unknown path is requested +- `400` (Bad Request): must be returned if an unknown path is requested. +- `429` (Too Many Requests): may be returned along with optional [Retry-After](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After) header to indicate to the caller that it is issuing requests too quickly. +- `501` (Not Implemented): must be returned if a method/path is not supported. ## CORS and Web Browsers @@ -136,10 +178,10 @@ Specification: [ipfs/specs/BITSWAP.md](https://github.com/ipfs/specs/blob/main/B ```json { - "Protocol": "transport-bitswap", - "Schema": "bitswap", - "ID": "12D3K...", - "Addrs": ["/ip4/..."] + "Protocol": "transport-bitswap", + "Schema": "bitswap", + "ID": "12D3K...", + "Addrs": ["/ip4/..."] } ``` @@ -159,13 +201,13 @@ Specification: [ipfs/go-graphsync/blob/main/docs/architecture.md](https://github ```json { - "Protocol": "transport-graphsync-filecoinv1", - "Schema": "graphsync-filecoinv1", - "ID": "12D3K...", - "Addrs": ["/ip4/..."], - "PieceCID": "", - "VerifiedDeal": true, - "FastRetrieval": true + "Protocol": "transport-graphsync-filecoinv1", + "Schema": "graphsync-filecoinv1", + "ID": "12D3K...", + "Addrs": ["/ip4/..."], + "PieceCID": "", + "VerifiedDeal": true, + "FastRetrieval": true } ``` @@ -180,3 +222,4 @@ Specification: [ipfs/go-graphsync/blob/main/docs/architecture.md](https://github [multiaddr]: https://github.com/multiformats/multiaddr#specification [peer-id]: https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md [peer-id-representation]: https://github.com/libp2p/specs/blob/master/peer-ids/peer-ids.md#string-representation +[application/vnd.ipfs.ipns-record]: https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record From 6d49dfbf9874d0d47b11b308c893a9505bf3d3b4 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Wed, 31 May 2023 19:53:43 +0200 Subject: [PATCH 3/4] chore(ipip-379): final editorials --- src/ipips/ipip-0379.md | 37 ++++++++++++++++++++++------------ src/routing/http-routing-v1.md | 23 ++++++++++++--------- 2 files changed, 38 insertions(+), 22 deletions(-) diff --git a/src/ipips/ipip-0379.md b/src/ipips/ipip-0379.md index 7cd83d59f..6cc550f31 100644 --- a/src/ipips/ipip-0379.md +++ b/src/ipips/ipip-0379.md @@ -20,39 +20,49 @@ tags: ['ipips'] ## Summary -This IPIP specifies a set of HTTP APIs to offload naming system onto another process or server. +This IPIP specifies `/routing/v1/ipns` HTTP API to offload naming system onto another process or server. ## Motivation -Expanding on the motivations of :cite[ipip-0337], the work here concentrates on delegation of _naming system_ over HTTP APIs. Naming is part of the core IPFS DHT functionality. +One of the motivations of this document is to introduce simple to use HTTP APIs and ultimately reduce barrier for interaction across alternative systems. + +Expanding on the motivations of :cite[ipip-0337], the work here concentrates on delegation of IPNS over HTTP API. Naming is part of the core IPFS DHT functionality. The performance of naming system over the IPFS DHT can suffer from long delays due to churn of records and quorum requirements. ## Detailed design -See :cite[http-routing-v1] specification. +Add `/routing/v1/ipns` to the existing :cite[http-routing-v1] specification. ## Design rationale The rationale for delegated IPNS over HTTP APIs closely follows the reasoning listed in :cite[ipip-0337]. The document proposes the following: -- Use of HTTP semantics for publication and resolution of naming records. -- Use of existing :ref[IPNS Record] serialization format through `Content-Type: application/vnd.ipfs.ipns-record`. -- Streaming interaction is not supported. +- HTTP `GET` and `PUT` semantics for publication and resolution of IPNS records. +- Use of existing :ref[IPNS Record] serialization format as HTTP `Content-Type` [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record). ### User benefit -The ability of offload naming onto another process or server via an idiomatic and simple HTTP API opens up an opportunity to not only reduce the burden of naming system management, but also introduce alternative naming systems across other routing systems, namely IPNI. +The ability to offload naming to another process or server using a simple HTTP +API brings several benefits: + +- It reduces the resource drain on light clients such as JavaScript running on + a web page, mobile devices, IoT devices, and gateway HTTP services. +- It enables scaling of IPNS resolution separately from retrieval and + deserialization/verification services. +- Expands the utility of IPNS beyond IPFS nodes, contributes to its broader + adoption and impact by reducing integration costs. ### Compatibility -See "Compatibility" section of :cite[ipip-0337]. +See the "Compatibility" section of :cite[ipip-0337]. #### Serialization Format -:cite[ipns-record] uses `application/vnd.ipfs.ipns-record` protobuf serialization format. +Standard IPNS record serialization format is used, making it fully compatible with the existing IPNS ecosystem. + +:cite[ipns-record] uses [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) protobuf serialization format. This format is widely in use in IPNS over PubSub and DHT routing systems. -One of the motivations of this document is to introduce simple to use HTTP APIs and ultimately reduce barrier for interaction across alternative systems. Further, interoperability across the existing and HTTP APIs is also desirable in order to reduce the barrier for adoption of the delegated HTTP APIs. To maximize interoperability with existing ecosystem, the canonical IPNS record serialization format :cite[ipns-record] (`0x0300`) can be requested with content type `application/vnd.ipfs.ipns-record`. @@ -64,14 +74,15 @@ Additionally, the IPNS records must be validated according to the rules stated i To avoid Denial of Service attack, maximum IPNS record size defined in :cite[ipns-record] applies. -Privacy in delegated IPNS is out of scope for this work. +Privacy in delegated IPNS is out of scope for this IPIP. ### Alternatives See: - -- [Libp2p-based IPNS](https://github.com/ipfs/go-ipns). +- IPNS over (libp2p over HTTP). While it may be possible in the future, it has more narrow utility and way bigger implementation complexity than libp2p-agnostic HTTP API with protobuf GET/PUT. - Reframe; it was deprecated and now removed. Historical reasons can be found in :cite[ipip-0337]. +- JSON IPNS Record representation was descoped due to open question how signing should work and unclear use case ([discussion](https://github.com/ipfs/specs/pull/379#discussion_r1107898543)). +- API-specific max record size was descoped, as it would introduce interop problems harmful to the IPNS ecosystem. ### Copyright diff --git a/src/routing/http-routing-v1.md b/src/routing/http-routing-v1.md index 747711225..5e6e9e412 100644 --- a/src/routing/http-routing-v1.md +++ b/src/routing/http-routing-v1.md @@ -67,11 +67,7 @@ Where: Specifications for some transfer protocols are provided in the "Transfer Protocols" section. -### IPNS Records - -:ref[IPNS Records] are serialized using the verifiable [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) format. For more details, read the specification :cite[ipns-record]. - -## API +## Content Providers API ### `GET /routing/v1/providers/{cid}` @@ -103,35 +99,44 @@ Response limit: 100 providers Each object in the `Providers` list is a *read provider record*. +## IPNS API + ### `GET /routing/v1/ipns/{name}` #### Path Parameters -- `name` is the :ref[IPNS Name] to resolve. +- `name` is the :ref[IPNS Name] to resolve, encoded as CIDv1. #### Response Status Codes - `200` (OK): the response body contains the :ref[IPNS Record] for the given :ref[IPNS Name]. - `404` (Not Found): must be returned if no matching records are found. +- `406` (Not Acceptable): requested content type is missing or not supported. Error message returned in body should inform the user to retry with `Accept: application/vnd.ipfs.ipns-record`. + +#### Response Headers + +- `Etag`: a globally unique opaque string used for HTTP caching. MUST be derived from the protobuf record returned in the body. +- `Cache-Control: max-age={TTL}`: cache TTL returned with :ref[IPNS Record] that has `IpnsEntry.data[TTL] > 0`. When present, SHOULD match the TTL value from the record. When record was not found (HTTP 404) or has no TTL (value is `0`), implementation SHOULD default to `max-age=60`. #### Response Body -The response body contains a [`application/vnd.ipfs.ipns-record`][application/vnd.ipfs.ipns-record] serialized :ref[IPNS Record]. +The response body contains a :ref[IPNS Record] serialized using the verifiable [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) protobuf format. ### `PUT /routing/v1/ipns/{name}` #### Path Parameters -- `name` is the :ref[IPNS Name] to publish. +- `name` is the :ref[IPNS Name] to publish, encoded as CIDv1. #### Request Body -The content body must be a [`application/vnd.ipfs.ipns-record`][application/vnd.ipfs.ipns-record] serialized :ref[IPNS Record], which matches the `name` path parameter. +The content body must be a [`application/vnd.ipfs.ipns-record`][application/vnd.ipfs.ipns-record] serialized :ref[IPNS Record], with a valid signature matching the `name` path parameter. #### Response Status Codes - `200` (OK): the provided :ref[IPNS Record] was published. - `400` (Bad Request): the provided :ref[IPNS Record] or :ref[IPNS Name] are not valid. +- `406` (Not Acceptable): submitted content type is not supported. Error message returned in body should inform the user to retry with `Content-Type: application/vnd.ipfs.ipns-record`. ## Pagination From c734686458786e203b8519b1ec3dd12086654251 Mon Sep 17 00:00:00 2001 From: Marcin Rataj Date: Wed, 31 May 2023 20:41:36 +0200 Subject: [PATCH 4/4] ipns: reference IANA content type & lean records --- src/ipns/ipns-record.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/src/ipns/ipns-record.md b/src/ipns/ipns-record.md index 6ac3eba49..a9132f9a9 100644 --- a/src/ipns/ipns-record.md +++ b/src/ipns/ipns-record.md @@ -159,6 +159,8 @@ IPNS records are stored locally, as well as spread across the network, in order For storing this structured data at rest and on the wire, we use `IpnsEntry` encoded as [protobuf](https://github.com/google/protobuf), which is a language-neutral, platform neutral extensible mechanism for serializing structured data. The extensible part of IPNS Record is placed in `IpnsEntry.data` field, which itself is encoded using a strict and deterministic subset of CBOR named [DAG-CBOR](https://ipld.io/specs/codecs/dag-cbor/spec/). +This canonical serialization format uses the [`application/vnd.ipfs.ipns-record`](https://www.iana.org/assignments/media-types/application/vnd.ipfs.ipns-record) content type. + ```protobuf message IpnsEntry { enum ValidityType { @@ -196,10 +198,14 @@ message IpnsEntry { } ``` -Notes: +:::issue + +For legacy reasons, some values must be stored in both `IpnsEntry` protobuf **and** `IpnsEntry.data` CBOR. +This should not be ignored, as it impacts interoperability with old software. + +Opt-in lean IPNS Records are discussed in [ipfs/specs#376](https://github.com/ipfs/specs/issues/376). -- For legacy reasons, some values must be stored in both `IpnsEntry` protobuf and `IpnsEntry.data` CBOR. - This should not be ignored, as it impacts interoperability with old software. +::: ### Record Size Limit