IPIP-428: Allowing V2-Only Records in IPNS

src/ipips/ipip-0428.md

@@ -0,0 +1,176 @@
+---
+title: "IPIP-0428: Allowing V2-Only Records in IPNS"
+date: 2023-07-24
+ipip: ratified
+editors:
+  - name: Marcin Rataj
+    github: lidel
+    url: https://lidel.org/
+    affiliation:
+        name: Protocol Labs
+        url: https://protocol.ai/
+  - name: Henrique Dias
+    github: hacdias
+    url: https://hacdias.com/
+    affiliation:
+        name: Protocol Labs
+        url: https://protocol.ai/
+relatedIssues:
+  - https://github.com/ipfs/specs/issues/376
+  - https://github.com/ipfs/boxo/pull/339
+  - https://github.com/ipfs/kubo/pull/9932
+  - https://github.com/ipfs/js-ipns/pull/234
+order: 428
+tags: ['ipips']
+---
+
+## Summary
+
+Introduce support for creation and validation of compact, V2-only IPNS Records.
+
+## Motivation
+
+IPNS record creation and validation is overly complex due to the legacy of
+decisions made in 2021.
+
+The "V1+V2" record creation and validation was reverse-engineered and documented
+the current state in [ipfs/specs#319](https://github.com/ipfs/specs/pull/319),
+which created a base for specifications to improve upon.
+
+A quick refresher on how IPNS Record lifecycle works today (2023 Q2):
+
+- _Record Creation_ produces both V1 and V2 signatures, and the record has
+  duplicated values in both top level protobuf AND `data` CBOR field.
+
+- _Record Validation_ only cares about V2 signature, but still requires fields
+  related to V1 to be always present in a record and match values from CBOR in
+  `data` field, for the record to be considered valid.
+
+We've been producing and expecting these hybrid V1+V2 records [since 2021](https://github.com/ipfs/js-ipns/pull/121).
+
+An unfortunate result is that all mandatory values MUST be duplicated, even
+when both ends use a modern client that only cares about `signatureV2` that
+guards CBOR field, otherwise the record will not be valid.
+
+What this IPIP aims to improve is allow implementations to produce lean,
+V2-only IPNS records and ensure clients will interpret them as valid IPNS.
+
+## Detailed design
+
+Finish V2 story by making V2-Only records possible, namely:
+
+- Record Creation: document and default to lean V2-Only records, keep V1+V2 as legacy
+  backward-compatible variant.
+
+- Record Validation: adjust the algorithm to stop requiring V1 fields when there is no
+  `signatureV1`.
+
+For details, see the updated :cite[ipns-record] specification.
+
+## Design rationale
+
+For modern IPNS, the outer `IpnsEntry` protobuf should effectively only have
+two required fields: `data` and its `signatureV2`, and such record, as long
+signature is valid, should be accepted as valid.
+
+At the same time, we don't want to break existig software, especially software
+and hardware devices which use IPNS for pulling updates.
+
+We can get to that future in two steps:
+
+1. Reference implementations (boxo/ipns, js-ipns) will keep producing V1+V2
+   records as backward-compatible default, but we adjust validation algorithm
+   to allow V2-only records, and support creation of such records as opt-in in
+   modern implementations of IPFS+IPNS, like Kubo (GO) and Helia (JS).
+   - Namely, only check/require fields to be duplicated in top level protobuf IF
+     `signatureV1` is present in the `IpnsEntry` protobuf.
+     - IF there is no `signatureV1`, the V1 record would be invalid anyway.
+     - IF there is no `signatureV1` but `signatureV2` and `data` fields
+       are present and valid, the V2-only record should be considered valid.
+        - This will allow people to build V2-only systems that produce records that
+          are considered valid.
+
+2. At some point in the future, e.g. when we see the majority of the public
+   swarm supports V2-Only records, libraries like boxo/ipns, js-ipns and
+   implementations like Kubo will stop producing V1+V2 and switch to publishing
+   V2-only records that are protobuf with only two fields: Data
+   CBOR+signatureV2.
+
+### User benefit
+
+- End users: the main benefit for end user is the smaller size of IPNS Records and
+  less complexity during creation/validation of modern V2-only records.
+
+- Developers interested in IPNS: by making IPNS Record creation as simple as
+  "create DAG-CBOR with these fields, and sign it", and validation to
+  "signatureV2 should match the DAG-CBOR value and key". We've removed surface
+  for bugs, making it easier to reason about for use in greenfield projects.
+
+- IPFS ecosystem: lowering the complexity related to IPNS record creation and
+  validation makes it more likely for third-party interoperable IPNS
+  implementations to happen.
+
+### Compatibility
+
+- This is backward-compatible, we adjust validation logic to allow V2-only
+  records, but all V1+V2 records that are being used in the wild today are
+  still valid
+
+- V2-only rollout is not part of this IPIP.
+  - Our suggestion is to default to creating V1+V2 records for now, keeping
+    backward-compatibility with the old IPNS clients.
+
+  - Creation of V2-only records should be introduced as an explicit opt-in. It
+    is up to implementations to decide when it is feasible to default to
+    creating V2-only records on IPNS publish.
+
+### Security
+
+- `IpnsEntry.signatureV1` (protobuf field) is parsed only by legacy clients, modern ones ignore this value
+
+It is highly advised to implement validation conformance tests using the fixtures
+included at the end of this IPIP.
+
+### Alternatives
+
+Describe alternate designs that were considered and related work.
+
+1. Just switch to V2-only as the new default!
+   - No, this would be a breaking change. We have to do this in two steps,
+     because we've rushed the way V2 was introduced in 2021, and STILL require
+     field copying, even when `signatureV1` is missing. So technically there
+     was never "V2", it was more like "V1.5". Only with this IPIP, we finally
+     adjust validation to only care about CBOR values when there is no
+     `signatureV1`.
+
+2. Why keeping the outer protobuf envelope? Could we make IPNS DAG-CBOR-only?
+   - Due to how long it takes for decentralized network nodes to upgrade, we prefer evolution rather than revolution.
+   - Protobuf is a useful envelope for two reasons:
+     1. Ensures the opaque V2-only record can be passed and stored in existing infrastructure.
+     2. Allows us to evolve IPNS record ("V3") in the future without impacting existing infrastructure.
+
+## Test fixtures
+
+To make testing easier below are test vectors in form of IPNS records along
+with the expected verification results. These test records are valid for 100
+years, making them safe for use in CI tests.
+
+1. [V1-only](https://dweb.link/ipfs/bafybeifkipmlz2fehxda6y7x752uolfed7bdd46jzdammpfga5zrnkq33u/k51qzi5uqu5dm4tm0wt8srkg9h9suud4wuiwjimndrkydqm81cqtlb5ak6p7ku_v1.ipns-record) → record invalid
+2. [V1+V2](https://dweb.link/ipfs/bafybeifkipmlz2fehxda6y7x752uolfed7bdd46jzdammpfga5zrnkq33u/k51qzi5uqu5dlkw8pxuw9qmqayfdeh4kfebhmreauqdc6a7c3y7d5i9fi8mk9w_v1-v2.ipns-record) (both signatures valid) → record valid, value points at `/ipfs/bafkqaddwgevxmmraojswg33smq`
+3. [V1+V2](https://dweb.link/ipfs/bafybeifkipmlz2fehxda6y7x752uolfed7bdd46jzdammpfga5zrnkq33u/k51qzi5uqu5dlmit2tuwdvnx4sbnyqgmvbxftl0eo3f33wwtb9gr7yozae9kpw_v1-v2-broken-v1-value.ipns-record) (both signatures valid, but 'value' is different in V1 pb vs V2 CBOR) → record invalid
+4. [V1+V2](https://dweb.link/ipfs/bafybeifkipmlz2fehxda6y7x752uolfed7bdd46jzdammpfga5zrnkq33u/k51qzi5uqu5diamp7qnnvs1p1gzmku3eijkeijs3418j23j077zrkok63xdm8c_v1-v2-broken-signature-v2.ipns-record) (only signatureV1 valid) → record invalid
+5. [V1+V2](https://dweb.link/ipfs/bafybeifkipmlz2fehxda6y7x752uolfed7bdd46jzdammpfga5zrnkq33u/k51qzi5uqu5dilgf7gorsh9vcqqq4myo6jd4zmqkuy9pxyxi5fua3uf7axph4y_v1-v2-broken-signature-v1.ipns-record) (only signatureV2 valid) → record valid, value points at `/ipfs/bafkqahtwgevxmmrao5uxi2bamjzg623fnyqhg2lhnzqxi5lsmuqhmmi`
+6. [V2-only](https://dweb.link/ipfs/bafybeifkipmlz2fehxda6y7x752uolfed7bdd46jzdammpfga5zrnkq33u/k51qzi5uqu5dit2ku9mutlfgwyz8u730on38kd10m97m36bjt66my99hb6103f_v2.ipns-record) (no V1 fields) → record valid
+
+:::note
+
+Implementers can either write own tests against the above test vectors, or run
+[gateway-conformance](https://github.com/ipfs/gateway-conformance/) test suite,
+which includes tests for these vectors since
+[gateway-conformance/pull/157](https://github.com/ipfs/gateway-conformance/pull/157).
+
+:::
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).