From affd081c6b241d959f90b0c6d19f646fd372b305 Mon Sep 17 00:00:00 2001 From: David Dias Date: Wed, 17 Feb 2016 10:58:24 +0000 Subject: [PATCH] internals API --- api/README.md | 1 + api/internals/README.md | 91 ++++++++++++++++++++++++++++++++++ api/internals/blocks.md | 0 api/internals/merkle-dag.md | 97 +++++++++++++++++++++++++++++++++++++ 4 files changed, 189 insertions(+) create mode 100644 api/internals/README.md create mode 100644 api/internals/blocks.md create mode 100644 api/internals/merkle-dag.md diff --git a/api/README.md b/api/README.md index 3fb82cfec..d3c9bb368 100644 --- a/api/README.md +++ b/api/README.md @@ -29,6 +29,7 @@ This RFC is organized by chapters described on the *Table of contents* section. - [%N% Introduction]() - [%N% APIs]() + - [%N.M% Internals]() - [%N.M% Core]() - [%N.M% HTTP]() - [%N.M% CLI]() diff --git a/api/internals/README.md b/api/internals/README.md new file mode 100644 index 000000000..89679c335 --- /dev/null +++ b/api/internals/README.md @@ -0,0 +1,91 @@ +# Internals + +In addition to the APIs exposed to developers (cli, http and core), the IPFS spec provides a set of specifications for the IPFS subsystems (AKA modules), providing a clean understanding of the components that have to be constructed for a full IPFS implementation or even a clear knowledge of how to make a partial implementation, without compromissing due to the interdependencies of these subsystems. + +This spec is a `WIP` and should be treated like so. + +## Architecture + +In a birds eye view, the Internals of IPFS might look like this: + +``` + ▶ ┌───────────────────────────────────────────────────────────────────────────────┐ + │ IPFS Core │ + │ └───────────────────────────────────────────────────────────────────────────────┘ + │ + │ │ + │ + │ ┌──────────────┬──────────────┼────────────┬─────────────────┐ + │ │ │ │ │ + │ │ │ │ │ │ + ▼ │ ▼ │ ▼ + │ ┌──────────────────┐ │ ┌──────────────────┐ │ ┌──────────────────┐ + │ │ │ │ │ │ │ │ + │ │ Block Service │ │ │ DAG Service │ │ │ IPFS Repo │ + │ │ │ │ │ │ │ │ + │ └──────────────────┘ │ └──────────────────┘ │ └──────────────────┘ + │ │ │ │ + IPFS Core │ ▼ │ ┌────┴────┐ │ + ┌────────┐ │ ▼ ▼ │ + │ │ Block │ │ ┌────────┐┌────────┐ │ + └────────┘ │ │DAG Node││DAG Link│ │ + │ │ └────────┘└────────┘ │ + ┌──────────────────┐ │ │ ┌──────────────────┐ + │ │ │ │ │ │ │ + │ Bitswap │◀────┤ ├──────▶│ Importer │ + │ │ │ │ │ │ │ + └──────────────────┘ │ │ └──────────────────┘ + │ │ │ │ + │ │ ┌────┴────┐ + │ │ │ ▼ ▼ + │ │ ┌────────┐┌────────┐ + │ ┌──────────────────┐ │ │ │ layout ││chunker │ + │ │ │ ┌────────────┘ └────────┘└────────┘ + │ │ Files │◀────┘ │ + │ │ │ + │ └──────────────────┘ │ + ▶ │ + ▼ + ┌───────────────────────────────────────────────────────────────────────────────┐ + │ │ + │ │ + │ │ + │ libp2p │ + │ │ + │ │ + └───────────────────────────────────────────────────────────────────────────────┘ +``` + +TODO: + +- [ ] Add IPLD + +## Index + +- ipfs-blocks - Block Service and Block + - [spec]() + - [js](https://github.com/ipfs/js-ipfs-blocks) + - [go](https://github.com/ipfs/go-ipfs/tree/master/blocks) +- ipfs-merkle-dag - DAG Service and DAG Node (protobuf data format) + - [spec]() + - [js](https://github.com/vijayee/js-ipfs-merkle-dag) + - [go](https://github.com/ipfs/go-ipfs/tree/master/merkledag) +- ipfs-bitswap + - Defined by the [Exchange Spec](https://github.com/ipfs/specs/pull/53) + - [go](https://github.com/ipfs/go-ipfs/tree/master/exchange/bitswap) +- ipfs-data-importing + - Defined by the [Data Importing Spec](https://github.com/ipfs/specs/pull/57) + - [js](https://github.com/ipfs/js-ipfs-data-importing) + - [go](https://github.com/ipfs/go-ipfs/tree/master/unixfs/io) +- ipfs-repo + - Defined by the [Repo](https://github.com/ipfs/specs/pull/43) + - [js](https://github.com/ipfs/js-ipfs-repo) + - [go](https://github.com/ipfs/go-ipfs/tree/master/repo) +- ipfs-files + - [spec]() + - js - Currently, we support `jsipfs files add`, on the js-ipfs impl, but not the full files API impl + - [go](https://github.com/ipfs/go-ipfs/tree/master/mfs) +- ipfs-unixfs + - [spec]() + - [js](https://github.com/ipfs/js-ipfs-unixfs) + - [go](https://github.com/ipfs/go-ipfs/tree/master/unixfs) diff --git a/api/internals/blocks.md b/api/internals/blocks.md new file mode 100644 index 000000000..e69de29bb diff --git a/api/internals/merkle-dag.md b/api/internals/merkle-dag.md new file mode 100644 index 000000000..c81d774d1 --- /dev/null +++ b/api/internals/merkle-dag.md @@ -0,0 +1,97 @@ +Internal: ipfs-merkle-dag +========================= + +> This is the specification for the merkle-dag module used by IPFS. + +The merkle-dag module offers an interface to create MerkleDAG nodes and links and in addition implements the DAG Service, offering an interface to store and fetch Nodes that works with ipfs-blocks, and therefore local storage and network storage (through exchange). + +## API + +### DAG Service + +##### add + +> stores the node + +##### get + +> fetches a node by its multihash + +##### getRecursive + +> fetches a node and all of its links (if possible) + +##### remove + +> deletes a node + +### DAG Node + +##### createNode + +##### addNodeLink + +> creates a link on node A to node B by using node B to get its multihash + +##### addRawLink + +> creates a link on node A to node B by using direclty node B multihash + +##### updateNodeLink + +> updates a link on the node. *caution* this method returns a copy of the MerkleDAG node + +##### removeNodeLink + +> removes a link from the node by name + +##### removeNodeLinkByHash + +> removes a link from the node by the hash of the linked node + +##### copy + +> creates a copy of the MerkleDAG Node + +##### size + +> size of the node. This is a property, not a function + +##### multihash + +> returns the multihash (default: sha2-256) + +##### marshal + +> returns a protobuf serialized version, compatible with go-ipfs MerkleDAG + +##### unMarshal + +> desirializes a probuf serialized node + +##### (used internally) getPBNode + +> used internally + +##### (used internally) makeLink + +> used internally + +### DAG Link + + + +## Data Structures + +### DAG Node + +- protobuf schema - https://github.com/ipfs/go-ipfs/blob/master/merkledag/pb/merkledag.proto#L31-L39 + +### DAG Link + +- protobuf schema - https://github.com/ipfs/go-ipfs/blob/master/merkledag/pb/merkledag.proto#L18-L29 + +## Implementations + +- [js](https://github.com/vijayee/js-ipfs-merkle-dag) +- [go](https://github.com/ipfs/go-ipfs/tree/master/merkledag)