-
Notifications
You must be signed in to change notification settings - Fork 233
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #78 from ipfs/api/internals
WIP: IPFS Internals API
- Loading branch information
Showing
4 changed files
with
189 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| # Internals | ||
|
|
||
| In addition to the APIs exposed to developers (cli, http and core), the IPFS spec provides a set of specifications for the IPFS subsystems (AKA modules), providing a clean understanding of the components that have to be constructed for a full IPFS implementation or even a clear knowledge of how to make a partial implementation, without compromissing due to the interdependencies of these subsystems. | ||
|
|
||
| This spec is a `WIP` and should be treated like so. | ||
|
|
||
| ## Architecture | ||
|
|
||
| In a birds eye view, the Internals of IPFS might look like this: | ||
|
|
||
| ``` | ||
| ▶ ┌───────────────────────────────────────────────────────────────────────────────┐ | ||
| │ IPFS Core │ | ||
| │ └───────────────────────────────────────────────────────────────────────────────┘ | ||
| │ | ||
| │ │ | ||
| │ | ||
| │ ┌──────────────┬──────────────┼────────────┬─────────────────┐ | ||
| │ │ │ │ │ | ||
| │ │ │ │ │ │ | ||
| ▼ │ ▼ │ ▼ | ||
| │ ┌──────────────────┐ │ ┌──────────────────┐ │ ┌──────────────────┐ | ||
| │ │ │ │ │ │ │ │ | ||
| │ │ Block Service │ │ │ DAG Service │ │ │ IPFS Repo │ | ||
| │ │ │ │ │ │ │ │ | ||
| │ └──────────────────┘ │ └──────────────────┘ │ └──────────────────┘ | ||
| │ │ │ │ | ||
| IPFS Core │ ▼ │ ┌────┴────┐ │ | ||
| ┌────────┐ │ ▼ ▼ │ | ||
| │ │ Block │ │ ┌────────┐┌────────┐ │ | ||
| └────────┘ │ │DAG Node││DAG Link│ │ | ||
| │ │ └────────┘└────────┘ │ | ||
| ┌──────────────────┐ │ │ ┌──────────────────┐ | ||
| │ │ │ │ │ │ │ | ||
| │ Bitswap │◀────┤ ├──────▶│ Importer │ | ||
| │ │ │ │ │ │ │ | ||
| └──────────────────┘ │ │ └──────────────────┘ | ||
| │ │ │ │ | ||
| │ │ ┌────┴────┐ | ||
| │ │ │ ▼ ▼ | ||
| │ │ ┌────────┐┌────────┐ | ||
| │ ┌──────────────────┐ │ │ │ layout ││chunker │ | ||
| │ │ │ ┌────────────┘ └────────┘└────────┘ | ||
| │ │ Files │◀────┘ │ | ||
| │ │ │ | ||
| │ └──────────────────┘ │ | ||
| ▶ │ | ||
| ▼ | ||
| ┌───────────────────────────────────────────────────────────────────────────────┐ | ||
| │ │ | ||
| │ │ | ||
| │ │ | ||
| │ libp2p │ | ||
| │ │ | ||
| │ │ | ||
| └───────────────────────────────────────────────────────────────────────────────┘ | ||
| ``` | ||
|
|
||
| TODO: | ||
|
|
||
| - [ ] Add IPLD | ||
|
|
||
| ## Index | ||
|
|
||
| - ipfs-blocks - Block Service and Block | ||
| - [spec]() | ||
| - [js](https://github.com/ipfs/js-ipfs-blocks) | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/blocks) | ||
| - ipfs-merkle-dag - DAG Service and DAG Node (protobuf data format) | ||
| - [spec]() | ||
| - [js](https://github.com/vijayee/js-ipfs-merkle-dag) | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/merkledag) | ||
| - ipfs-bitswap | ||
| - Defined by the [Exchange Spec](https://github.com/ipfs/specs/pull/53) | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/exchange/bitswap) | ||
| - ipfs-data-importing | ||
| - Defined by the [Data Importing Spec](https://github.com/ipfs/specs/pull/57) | ||
| - [js](https://github.com/ipfs/js-ipfs-data-importing) | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/unixfs/io) | ||
| - ipfs-repo | ||
| - Defined by the [Repo](https://github.com/ipfs/specs/pull/43) | ||
| - [js](https://github.com/ipfs/js-ipfs-repo) | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/repo) | ||
| - ipfs-files | ||
| - [spec]() | ||
| - js - Currently, we support `jsipfs files add`, on the js-ipfs impl, but not the full files API impl | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/mfs) | ||
| - ipfs-unixfs | ||
| - [spec]() | ||
| - [js](https://github.com/ipfs/js-ipfs-unixfs) | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/unixfs) |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| Internal: ipfs-merkle-dag | ||
| ========================= | ||
|
|
||
| > This is the specification for the merkle-dag module used by IPFS. | ||
| The merkle-dag module offers an interface to create MerkleDAG nodes and links and in addition implements the DAG Service, offering an interface to store and fetch Nodes that works with ipfs-blocks, and therefore local storage and network storage (through exchange). | ||
|
|
||
| ## API | ||
|
|
||
| ### DAG Service | ||
|
|
||
| ##### add | ||
|
|
||
| > stores the node | ||
| ##### get | ||
|
|
||
| > fetches a node by its multihash | ||
| ##### getRecursive | ||
|
|
||
| > fetches a node and all of its links (if possible) | ||
| ##### remove | ||
|
|
||
| > deletes a node | ||
| ### DAG Node | ||
|
|
||
| ##### createNode | ||
|
|
||
| ##### addNodeLink | ||
|
|
||
| > creates a link on node A to node B by using node B to get its multihash | ||
| ##### addRawLink | ||
|
|
||
| > creates a link on node A to node B by using direclty node B multihash | ||
| ##### updateNodeLink | ||
|
|
||
| > updates a link on the node. *caution* this method returns a copy of the MerkleDAG node | ||
| ##### removeNodeLink | ||
|
|
||
| > removes a link from the node by name | ||
| ##### removeNodeLinkByHash | ||
|
|
||
| > removes a link from the node by the hash of the linked node | ||
| ##### copy | ||
|
|
||
| > creates a copy of the MerkleDAG Node | ||
| ##### size | ||
|
|
||
| > size of the node. This is a property, not a function | ||
| ##### multihash | ||
|
|
||
| > returns the multihash (default: sha2-256) | ||
| ##### marshal | ||
|
|
||
| > returns a protobuf serialized version, compatible with go-ipfs MerkleDAG | ||
| ##### unMarshal | ||
|
|
||
| > desirializes a probuf serialized node | ||
| ##### (used internally) getPBNode | ||
|
|
||
| > used internally | ||
| ##### (used internally) makeLink | ||
|
|
||
| > used internally | ||
| ### DAG Link | ||
|
|
||
|
|
||
|
|
||
| ## Data Structures | ||
|
|
||
| ### DAG Node | ||
|
|
||
| - protobuf schema - https://github.com/ipfs/go-ipfs/blob/master/merkledag/pb/merkledag.proto#L31-L39 | ||
|
|
||
| ### DAG Link | ||
|
|
||
| - protobuf schema - https://github.com/ipfs/go-ipfs/blob/master/merkledag/pb/merkledag.proto#L18-L29 | ||
|
|
||
| ## Implementations | ||
|
|
||
| - [js](https://github.com/vijayee/js-ipfs-merkle-dag) | ||
| - [go](https://github.com/ipfs/go-ipfs/tree/master/merkledag) |