IPIP 0001: Lightweight "RFC" Process for IPFS Specifications

.github/CODEOWNERS

@@ -1,4 +1,14 @@
-# Spec Stewards defined below are automatically requested for review when
-# someone opens a pull request that modifies area of their interest.
+# This is a CODEOWNERS file
+# Each line is a file pattern followed by one or more owners.
+# Order is important; the last matching pattern takes the most precedence.
+# See: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners
+
+# global IPIP
+IPIP/ @ipfs/specs-stewards
+
+# Selected Spec Stewards can be defined below to be automatically requested for
+# review when someone opens a pull request that modifies area of their
+# interest.
 
 http-gateways/  @lidel
+reframe/*HTTP*  @lidel

IPIP/0000-template.md

@@ -0,0 +1,68 @@
+# IPIP 0000: InterPlanetary Improvement Proposal Template
+
+<!-- IPIP number will be assigned by an editor. When opening a pull request to
+submit your IPIP, please use number 0000 and an abbreviated title in the filename,
+`0000-draft-title-abbrev.md`. -->
+
+- Start Date: YYYY-MM-DD
+- Related Issues:
+  - (add links here)
+
+## Summary
+
+<!--One paragraph explanation of the IPIP.-->
+This is the suggested template for new IPIPs.
+
+## Motivation
+
+AKA Problem Statement
+
+Clearly explain why the existing protocol specification is inadequate
+to address the problem that the IPIP solves.
+
+## Detailed design
+
+AKA Solution Proposal
+
+Describe the proposed solution and list all changes made to the specs repository.
+
+The resulting specification should be detailed enough to allow competing,
+interoperable implementations.
+
+When modifying an existing specification file, this section should provide a
+summary of changes. When adding new specification files, list all of them.
+
+## Test fixtures
+
+List relevant CIDs. Describe how implementations can use them to determine
+specification compliance. This section can be skipped if IPIP does not deal
+with the way IPFS handles content-addressed data, or the modified specification
+file already includes this information.
+
+## Design rationale
+
+The rationale fleshes out the specification by describing what motivated
+the design and why particular design decisions were made.
+
+Provide evidence of rough consensus and working code within the community,
+and discuss important objections or concerns raised during discussion.
+
+### User benefit
+
+How will end users benefit from this work?
+
+### Compatibility
+
+Explain the upgrade considerations for existing implementations.
+
+### Security
+
+Explain the security implications/considerations relevant to the proposed change.
+
+### Alternatives
+
+Describe alternate designs that were considered and related work.
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).

IPIP/0001-lightweight-improvement-proposal-process.md

@@ -0,0 +1,147 @@
+# IPIP 0001: Lightweight Improvement Process for IPFS Specifications
+
+- Start Date: 2022-06-10
+- Related Issues:
+  - [ipfs/specs/issues/286](https://github.com/ipfs/specs/issues/286)
+
+## Summary
+
+This _InterPlanetary Improvement Proposal_ (IPIP) introduces a lightweight
+"request for comments/change" process for the IPFS specifications
+[repository][1].
+
+[1]: https://github.com/ipfs/specs/
+
+## Motivation
+
+Today, protocol design discussions often take place in a repository of an IPFS
+implementation. These conversations are unintentionally obscured from the useful input of [Specs Stewards], other
+implementations, service operators and the wider IPFS Community.
+
+The IPFS Project needs a mechanism for proposing and evaluating specification
+improvements that are not tied to a specific programming language
+or implementation of IPFS.
+
+## Detailed design
+
+Adopt an informal IPIP process for the [ipfs/specs][1] repository, providing a
+minimal structure for opening, reviewing, and merging specification changes.
+
+The purpose of IPIP documents is to **document motivation** behind the change
+applied to the spec. **IPIP is not to be the spec itself**.
+
+To illustrate:
+- In order to understand how (hypothetical) WebDAV Gateway works, one would
+  read contents of specs in `ipfs/specs/WEBDAV_GATEWAY.md`.
+- IPIP in `ipfs/specs/IPIP/000N-webdav-gateway.md` would only include
+  **Motivation** and explainer why certain design decisions were made at a
+  certain point in time. Initial `IPIP/000N-webdav-gateway.md` would explain
+  why we added WebDAV spec in the first place.
+- If we realize the spec has a bug, we will evaluate the impact: adding more
+  details, test vectors, and editorials/cosmetics can be fixed without IPIP.
+- Things that could cause an interop issues require a PR with fix and IPIP in
+  `ipfs/specs/IPIP/000M-webdav-fix-for-foo.md` explaining why we make the
+  breaking spec change, compatibility/migration considerations etc.
+
+### Opening an improvement proposal (IPIP)
+
+Changes to IPFS specifications can be proposed by opening a Git pull-request
+(PR) against the `ipfs/specs` repository.
+
+In addition to specification changes, such PR must include a short **IPIP
+document** based on the template in [`ipfs/specs/IPIP/0000-template.md`](./0000-template.md).
+
+When a new specification file is added to the repo, it should be based on
+the template at [`ipfs/specs/template.md`](../template.md).
+
+### Reviewing IPIPs
+
+[Specs Stewards] will review new IPIP PRs during periodical (best-effort) triage.
+
+IPFS Community is encouraged to participate in the review process.
+
+IPIP can be either:
+- merged,
+- rejected (PR close without merging),
+- deferred (converting PR back to a draft).
+
+The final decision belongs to [Specs Stewards].
+
+### Merging IPIPs
+
+PR with a IPIP can be merged only after two [Specs Stewards] approve it and
+there are no objections from other Stewards.
+
+IPIP number is assigned before the PR merge.
+
+IPIP author and two approving [Specs Stewards] are added to `CODEOWNERS` file
+to be automatically asked to review any future changes to files added or
+modified by the IPIP.
+
+### Long-term plan
+
+[Specs Stewards] will adjust the process based on usage patterns.
+
+## Design rationale
+
+We want to empower IPFS community members and implementers with the ability to propose
+changes in a well-known way, without introducing much overhead.
+
+Guiding principles:
+- No new tooling
+  - Reuse Markdown, Git, and existing PR review process
+- Convention over Byzantine process
+  - Proposing a new IPIP should have low cognitive overhead, allowing us to
+    focus on specs
+  - Reuse existing Github developer accounts and reputation attached to them
+  - One should be able to create a valid IPIP without reading a long explainer
+    like this one. Looking at past IPIPs, and then copying a template and
+    opening a PR with the proposal should be more than enough.
+
+### User benefit
+
+End users will indirectly benefit from a healthy IPIP process being in place:
+
+- IPFS community members will be able to use IPIP drafts for evaluating ideas
+  before investing time into building new things.
+- The bar for creating a brand new IPFS implementation will be lowered, and
+  existing implementations will be able to propose improvements for others to
+  adopt. This removes the soft vendor lock-in present when the oldest
+  implementation is considered as the reference standard and source of truth.
+- IPFS implementers will have a better understanding of why certain design
+  decisions were made, and have both historical context and language-agnostic
+  specifications with test fixtures ready for use in their project, ensuring
+  a high level of interoperability.
+- More eyes looking at specifications will improve overall quality over time.
+
+As a result, IPFS will become easier to implement, useful in more contexts,
+and benefit more people.
+
+### Compatibility
+
+Existing contents of [ipfs/specs][1] repository act as the initial state
+against which IPIP PRs can be opened.
+
+### Security
+
+Existing Git-based review infrastructure, user accounts and reputation
+system will be reused.
+
+Merging IPIP will require approval from two [Specs Stewards].
+
+### Alternatives
+
+- Maintaining the status quo (no IPIP process) is not acceptable, as we want to
+  move specification discussions away from repositories of specific
+  implementations. We need a mechanism for discussing improvements that is not
+  tied to specific implementation or language.
+- Creating more elaborate IPIP process. This comes with increased overhead and
+  risk. Introducing a complex process requires deeper understanding of
+  community needs and pitfalls of preexisting processes, and since we don't
+  have any process in place, starting light, limits the risk.
+
+### Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
+
+[Specs Stewards]: https://github.com/orgs/ipfs/teams/specs-stewards/members

README.md

@@ -1,41 +1,53 @@
 # IPFS Specifications
 
-[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)
-[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)
-[![Matrix](https://img.shields.io/badge/matrix-%23ipfs%3Amatrix.org-blue.svg?style=flat-square)](https://matrix.to/#/#ipfs:matrix.org)
-[![IRC](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
-[![Discord](https://img.shields.io/discord/475789330380488707?color=blueviolet&label=discord&style=flat-square)](https://discord.gg/24fmuwR)
-
 > This repository contains the specs for the IPFS Protocol and associated subsystems.
 
+- [Documentation and Community](#documentation-and-community)
+- [Understanding badges](#understanding-the-meaning-of-the-spec-badges-and-their-lifecycle)
+- [Index](#index)
+- [Contribute](#contribute)
+  - [InterPlanetary Improvement Process (IPIP)](#interplanetary-improvement-process-ipip)
+
+## Documentation and Community
+
+Looking for user support?
+
+See [Documentation](https://docs.ipfs.io),
+[Discussion Forums](https://discuss.ipfs.io/), and other
+[Community Resources](https://docs.ipfs.io/community/) instead.
+
 ## Understanding the meaning of the spec badges and their lifecycle
 
 We use the following label system to identify the state of each spec:
 
-- ![](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) - A work-in-progress, possibly to describe an idea before actually committing to a full draft of the spec.
-- ![](https://img.shields.io/badge/status-draft-yellow.svg?style=flat-square) - A draft that is ready to review. It should be implementable.
-- ![](https://img.shields.io/badge/status-reliable-green.svg?style=flat-square) - A spec that has been adopted (implemented) and can be used as a reference point to learn how the system works.
-- ![](https://img.shields.io/badge/status-stable-brightgreen.svg?style=flat-square) - We consider this spec to close to final, it might be improved but the system it specifies should not change fundamentally.
-- ![](https://img.shields.io/badge/status-permanent-blue.svg?style=flat-square) - This spec will not change.
-- ![](https://img.shields.io/badge/status-deprecated-red.svg?style=flat-square) - This spec is no longer in use.
+- ![wip](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square) - A work-in-progress, possibly to describe an idea before actually committing to a full draft of the spec.
+- ![draft](https://img.shields.io/badge/status-draft-yellow.svg?style=flat-square) - A draft that is ready to review. It should be implementable.
+- ![reliable](https://img.shields.io/badge/status-reliable-green.svg?style=flat-square) - A spec that has been adopted (implemented) and can be used as a reference point to learn how the system works.
+- ![stable](https://img.shields.io/badge/status-stable-brightgreen.svg?style=flat-square) - We consider this spec to close to final, it might be improved but the system it specifies should not change fundamentally.
+- ![permanent](https://img.shields.io/badge/status-permanent-blue.svg?style=flat-square) - This spec will not change.
+- ![deprecated](https://img.shields.io/badge/status-deprecated-red.svg?style=flat-square) - This spec is no longer in use.
 
 Nothing in this spec repository is `permanent` or even `stable` yet. Most of the subsystems are still a `draft` or in `reliable` state.
 
 ## Index
 
-The specs contained in this repository are:
+The specs contained in this and related repositories are:
 
 - **IPFS Protocol:**
   - [Protocol Architecture Overview](./ARCHITECTURE.md) - the top-level spec and the stack
   - [Other IPFS Overviews](/overviews) - quick overviews of the various parts of IPFS
 - **User Interface (aka Public APIs):**
-  - [HTTP Gateways](./http-gateways/README.md) - implementation agnostic interfaces for accessing content-addressed data over HTTP
+  - [HTTP Gateways](./http-gateways/) - implementation agnostic interfaces for accessing content-addressed data over HTTP
+  - [REFRAME](./reframe/) - protocol designed for request-response messages that is sufficiently generic and extensible to evolve over time as new needs for it arise
   - IPFS implementations may provide additional interfaces, for example:
-    - [HTTP RPC API exposed by go-ipfs](https://docs.ipfs.io/reference/http/api/)
+    - [Legacy HTTP RPC API exposed by Kubo (go-ipfs)](https://docs.ipfs.io/reference/http/api/)
     - [Programmatic Core API for JavaScript](https://github.com/ipfs/js-ipfs/tree/master/docs/core-api#readme)
 - **Data Formats:**
   - [IPLD](https://ipld.io/specs/) - InterPlanetary Linked Data.
-  - [Merkle DAG (Deprecated)](./MERKLE_DAG.md)
+    - [DAG-CBOR](https://ipld.io/docs/codecs/known/dag-cbor/) -  binary format, supporting the complete IPLD Data Model, with excellent performance, and suitable for any job.
+    - [DAG-JSON](https://ipld.io/docs/codecs/known/dag-json/) - human-readable format, supporting almost the complete IPLD Data Model, and very convenient for interoperability, development, and debugging.
+    - [DAG-PB](https://ipld.io/docs/codecs/known/dag-pb/) - a binary format for specific limited structures of data, which is highly used in IPFS and [UnixFS](./UNIXFS.md).
+    - [CAR](https://ipld.io/specs/transport/car/) - transport format used to store content addressable objects in the form of IPLD block data as a sequence of bytes; typically as an [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car) file with a `.car` extension
   - Self Describing Formats ([multiformats](http://github.com/multiformats/multiformats)):
     - [multihash](https://github.com/multiformats/multihash) - self-describing hash digest format.
     - [multiaddr](https://github.com/multiformats/multiaddr) - self-describing addressing format.
@@ -45,7 +57,7 @@ The specs contained in this repository are:
   - [UnixFS](./UNIXFS.md)
   - [Mutable File System (the Files API)](./MUTABLE_FILE_SYSTEM.md) - Virtual File System interface, unix like, on top of the MerkleDAG
 - **Storage Layer:**
-  - Pinning
+  - [Pinning Service API](https://ipfs.github.io/pinning-services-api-spec/)
   - [Repo](./REPO.md) - IPFS node local repository spec
     - [FileSystem Repo](./REPO_FS.md) - IPFS node local repository spec
 - **Block Exchanges:**
@@ -57,12 +69,22 @@ The specs contained in this repository are:
   - [libp2p](https://github.com/libp2p/specs) - libp2p is a modular and extensible network stack, built and use by IPFS, but that it can be reused as a standalone project. Covers:
 - **Records, Naming and Record Systems:**
   - [IPNS](./IPNS.md) - InterPlanetary Naming System
+  - [DNSLink](https://dnslink.dev)
 - **Other/related/included:**
   - [PDD](https://github.com/ipfs/pdd) - Protocol Driven Development
 
 ## Contribute
 
-[![](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
+[![contribute](https://cdn.rawgit.com/jbenet/contribute-ipfs-gif/master/img/contribute.gif)](https://github.com/ipfs/community/blob/master/CONTRIBUTING.md)
 
 Suggestions, contributions, criticisms are welcome. Though please make sure to familiarize yourself deeply with IPFS, the models it adopts, and the principles it follows.
 This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md).
+
+### InterPlanetary Improvement Process (IPIP)
+
+- Want to propose a change to an existing specification?
+- Or add a new protocol?
+
+See [IPIP 0001: Lightweight Improvement Process for IPFS Specifications](./IPIP/0001-lightweight-improvement-proposal-process.md).
+
+Accepted proposals can be found in the [`IPIP/`](./IPIP/) directory.

template.md

@@ -0,0 +1,52 @@
+# Specification Template
+
+![wip](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square)
+
+**Author(s)**:
+- Name
+
+**Maintainer(s)**:
+- Name
+
+* * *
+
+**Abstract**
+
+One paragraph explanation of the specification
+
+## Organization of this document
+
+- [Introduction](#introduction)
+- [Specification](#specification)
+  - [Test fixtures](#test-fixtures)
+  - [Security](#security)
+  - [Privacy and User Control](#privacy-and-user-control)
+
+## Introduction
+
+Clearly explain why the specification exists, what is the problem solved here.
+
+## Specification
+
+Explain things in depth.
+
+The resulting specification should be detailed enough to allow competing,
+interoperable implementations.
+
+### Test fixtures
+
+List relevant CIDs. Describe how implementations can use them to determine
+specification compliance.
+
+### Security
+
+Explain the security implications/considerations relevant to the spec.
+
+### Privacy and User Control
+
+Note if there are any privacy or user control considerations that should be
+taken into account by the implementers.
+
+## Copyright
+
+Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).