diff --git a/README.md b/README.md index fc0dfb5fe0..9c06e130f5 100644 --- a/README.md +++ b/README.md @@ -92,6 +92,8 @@ We've come a long way, but this project is still in Alpha, lots of development i - [Network](#network) - [Node Management](#node-management) - [Static types and utils](#static-types-and-utils) + - [Glob source](#glob-source) + - [URL source](#url-source) - [FAQ](#faq) - [How to enable WebRTC support for js-ipfs in the Browser](#how-to-enable-webrtc-support-for-js-ipfs-in-the-browser) - [Is there WebRTC support for js-ipfs with Node.js?](#is-there-webrtc-support-for-js-ipfs-with-nodejs) @@ -302,9 +304,9 @@ a case, you should provide a way to trigger migrations manually.** |------|---------| | boolean or object | `true` | -Initialize the repo when creating the IPFS node. +Perform repo initialization steps when creating the IPFS node. -If you have already initialized a repo before creating your IPFS node (e.g. you are loading a repo that was saved to disk from a previous run of your program), you must make sure to set this to `false`. Note that *initializing* a repo is different from creating an instance of [`ipfs.Repo`](https://github.com/ipfs/js-ipfs-repo). The IPFS constructor sets many special properties when initializing a repo, so you should usually not try and call `repoInstance.init()` yourself. +Note that *initializing* a repo is different from creating an instance of [`ipfs.Repo`](https://github.com/ipfs/js-ipfs-repo). The IPFS constructor sets many special properties when initializing a repo, so you should usually not try and call `repoInstance.init()` yourself. Instead of a boolean, you may provide an object with custom initialization options. All properties are optional: @@ -314,13 +316,13 @@ Instead of a boolean, you may provide an object with custom initialization optio ```js // Generating a Peer ID: const PeerId = require('peer-id') - PeerId.create({ bits: 2048 }, (err, peerId) => { - // Generates a new Peer ID, complete with public/private keypair - // See https://github.com/libp2p/js-peer-id - }) + // Generates a new Peer ID, complete with public/private keypair + // See https://github.com/libp2p/js-peer-id + const peerId = await PeerId.create({ bits: 2048 }) ``` - `pass` (string) A passphrase to encrypt keys. You should generally use the [top-level `pass` option](#optionspass) instead of the `init.pass` option (this one will take its value from the top-level option if not set). - `profiles` (Array) Apply profile settings to config. +- `allowNew` (boolean, default: `true`) Set to `false` to disallow initialization if the repo does not already exist. ##### `options.start` @@ -409,6 +411,11 @@ Available delegate multiaddrs are: **Note**: If more than 1 delegate multiaddr is specified, the actual delegate will be randomly selected on startup. +**Note**: If you wish to use delegated routing and are creating your node _programmatically_ in Node.js or the browser you must `npm install libp2p-delegated-content-routing` and/or `npm install libp2p-delegated-peer-routing` and provide configured instances of them in [`options.libp2p`](#optionslibp2p). See the module repos for further instructions: + +- https://github.com/libp2p/js-libp2p-delegated-content-routing +- https://github.com/libp2p/js-libp2p-delegated-peer-routing + ##### `options.ipld` | Type | Default | @@ -550,10 +557,12 @@ The libp2p option allows you to build your libp2p node by configuration, or via You can see the bundle in action in the [custom libp2p example](examples/custom-libp2p). - `modules` (object): - - `transport` (Array<[libp2p.Transport](https://github.com/libp2p/interface-transport)>): An array of Libp2p transport classes/instances to use _instead_ of the defaults. See [libp2p/interface-transport](https://github.com/libp2p/interface-transport) for details. - - `peerDiscovery` (Array<[libp2p.PeerDiscovery](https://github.com/libp2p/interface-peer-discovery)>): An array of Libp2p peer discovery classes/instances to use _instead_ of the defaults. See [libp2p/peer-discovery](https://github.com/libp2p/interface-peer-discovery) for details. If passing a class, configuration can be passed using the config section below under the key corresponding to you module's unique `tag` (a static property on the class) + - `transport` (Array<[libp2p.Transport](https://github.com/libp2p/js-interfaces/tree/master/src/transport)>): An array of Libp2p transport classes/instances to use _instead_ of the defaults. See [libp2p/js-interfaces/transport](https://github.com/libp2p/js-interfaces/tree/master/src/transport) for details. + - `peerDiscovery` (Array<[libp2p.PeerDiscovery](https://github.com/libp2p/js-interfaces/tree/master/src/peer-discovery)>): An array of Libp2p peer discovery classes/instances to use _instead_ of the defaults. See [libp2p/js-interfaces/peer-discovery](https://github.com/libp2p/js-interfaces/tree/master/src/peer-discovery) for details. If passing a class, configuration can be passed using the config section below under the key corresponding to you module's unique `tag` (a static property on the class) - `dht` (object): a DHT implementation that enables PeerRouting and ContentRouting. Example [libp2p/js-libp2p-kad-dht](https://github.com/libp2p/js-libp2p-kad-dht) - `pubsub` (object): a Pubsub implementation on top of [libp2p/js-libp2p-pubsub](https://github.com/libp2p/js-libp2p-pubsub) + - `contentRouting` (Array<[libp2p.ContentRouting](https://github.com/libp2p/js-interfaces/tree/master/src/content-routing)>): An array of Libp2p content routing modules. See [libp2p/js-interfaces/content-routing](https://github.com/libp2p/js-interfaces/tree/master/src/content-routing) for details. + - `peerRouting` (Array<[libp2p.PeerRouting](https://github.com/libp2p/js-interfaces/tree/master/src/peer-routing)>): An array of Libp2p peer routing modules. See [libp2p/js-interfaces/peer-routing](https://github.com/libp2p/js-interfaces/tree/master/src/peer-routing) for details. - `config` (object): - `peerDiscovery` (object): - `autoDial` (boolean): Dial to discovered peers when under the Connection Manager min peer count watermark. (default `true`) @@ -739,48 +748,31 @@ node.on('stop', () => console.log('Node stopped!')) The IPFS core API provides all functionality that is not specific to setting up and starting or stopping a node. This API is available directly on an IPFS instance, on the command line (when using the CLI interface), and as an HTTP REST API. For a complete reference, see [![](https://img.shields.io/badge/interface--ipfs--core-API%20Docs-blue.svg)](https://github.com/ipfs/interface-ipfs-core). -All the API methods aside from streaming methods (ones that end in `ReadableStream` or `PullStream`) are asynchronous and return Promises, but _also_ accept callbacks. - The core API is grouped into several areas: #### Files - [Regular Files API](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md) - [`ipfs.add(data, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#add) - - [`ipfs.addPullStream([options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#addpullstream) - - [`ipfs.addReadableStream([options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#addreadablestream) - - [`ipfs.addFromStream(stream)`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#addfromstream) - - [`ipfs.addFromFs(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#addfromfs) - - [`ipfs.addFromUrl(url, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#addfromurl) - [`ipfs.cat(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#cat) - - [`ipfs.catPullStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#catpullstream) - - [`ipfs.catReadableStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#catreadablestream) - [`ipfs.get(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#get) - - [`ipfs.getPullStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#getpullstream) - - [`ipfs.getReadableStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#getreadablestream) - [`ipfs.ls(ipfsPath)`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#ls) - - [`ipfs.lsPullStream(ipfsPath)`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#lspullstream) - - [`ipfs.lsReadableStream(ipfsPath)`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#lsreadablestream) - [MFS (mutable file system) specific](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#mutable-file-system) + - [`ipfs.files.chmod(path, mode, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#fileschmod) - [`ipfs.files.cp([from, to])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filescp) - [`ipfs.files.flush([path])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesflush) - [`ipfs.files.ls([path], [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesls) - [`ipfs.files.mkdir(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesmkdir) - [`ipfs.files.mv([from, to])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesmv) - [`ipfs.files.read(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesread) - - [`ipfs.files.readPullStream(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesreadpullstream) - - [`ipfs.files.readReadableStream(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesreadreadablestream) - [`ipfs.files.rm(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesrm) - [`ipfs.files.stat(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filesstat) + - [`ipfs.files.touch(path, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#filestouch) - [`ipfs.files.write(path, content, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#fileswrite) - [refs](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/REFS.md) - [`ipfs.refs(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/REFS.md#refs) - - [`ipfs.refsReadableStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/REFS.md#refsreadablestream) - - [`ipfs.refsPullStream(ipfsPath, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/REFS.md#refspullstream) - [`ipfs.refs.local()`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/REFS.md#refslocal) - - [`ipfs.refs.localReadableStream()`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/REFS.md#refslocalreadablestream) - - [`ipfs.refs.localPullStream()`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/REFS.md#refslocalpullstream) #### Graph @@ -794,7 +786,7 @@ The core API is grouped into several areas: - [`ipfs.pin.ls([hash], [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/PIN.md#pinls) - [`ipfs.pin.rm(hash, [options])`](https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/PIN.md#pinrm) -- [object (legacy)](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/OBJECT.md) +- [object](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/OBJECT.md) - [`ipfs.object.new([template])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/OBJECT.md#objectnew) - [`ipfs.object.put(obj, [options])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/OBJECT.md#objectput) - [`ipfs.object.get(multihash, [options])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/OBJECT.md#objectget) @@ -877,8 +869,6 @@ The core API is grouped into several areas: - [`ipfs.id()`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/MISCELLANEOUS.md#id) - [`ipfs.version()`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/MISCELLANEOUS.md#version) - [`ipfs.ping(peerId, [options])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/MISCELLANEOUS.md#ping) - - [`ipfs.pingReadableStream(peerId, [options])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/MISCELLANEOUS.md#pingreadablestream) - - [`ipfs.pingPullStream(peerId, [options])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/MISCELLANEOUS.md#pingpullstream) - `ipfs.init([options])` - `ipfs.start()` - [`ipfs.stop()`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/MISCELLANEOUS.md#stop) @@ -895,8 +885,6 @@ The core API is grouped into several areas: - [stats](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/STATS.md) - [`ipfs.stats.bitswap()`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/STATS.md#statsbitswap) - [`ipfs.stats.bw([options])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/STATS.md#statsbw) - - [`ipfs.stats.bwPullStream([options]) -> Pull Stream`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/STATS.md#statsbwpullstream) - - [`ipfs.stats.bwReadableStream([options]) -> Readable Stream`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/STATS.md#statsbwreadablestream) - [`ipfs.stats.repo([options])`](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/STATS.md#statsrepo) - [config](https://github.com/ipfs/interface-ipfs-core/tree/master/SPEC/CONFIG.md) @@ -930,6 +918,78 @@ const { CID } = require('ipfs') import { CID } from 'ipfs' ``` +##### Glob source + +A utility to allow files on the file system to be easily added to IPFS. + +###### `globSource(path, [options])` + +- `path`: A path to a single file or directory to glob from +- `options`: Optional options +- `options.recursive`: If `path` is a directory, use option `{ recursive: true }` to add the directory and all its sub-directories. +- `options.ignore`: To exclude file globs from the directory, use option `{ ignore: ['ignore/this/folder/**', 'and/this/file'] }`. +- `options.hidden`: Hidden/dot files (files or folders starting with a `.`, for example, `.git/`) are not included by default. To add them, use the option `{ hidden: true }`. + +Returns an async iterable that yields `{ path, content }` objects suitable for passing to `ipfs.add`. + +###### Example + +```js +const IPFS = require('ipfs') +const { globSource } = IPFS + +const ipfs = await IPFS.create() + +for await (const file of ipfs.add(globSource('./docs', { recursive: true }))) { + console.log(file) +} + +/* +{ + path: 'docs/assets/anchor.js', + cid: CID('QmVHxRocoWgUChLEvfEyDuuD6qJ4PhdDL2dTLcpUy3dSC2'), + size: 15347 +} +{ + path: 'docs/assets/bass-addons.css', + cid: CID('QmPiLWKd6yseMWDTgHegb8T7wVS7zWGYgyvfj7dGNt2viQ'), + size: 232 +} +... +*/ +``` + +##### URL source + +A utility to allow content from the internet to be easily added to IPFS. + +###### `urlSource(url)` + +- `url`: A string URL or [`URL`](https://developer.mozilla.org/en-US/docs/Web/API/URL) instance to send HTTP GET request to + +Returns an async iterable that yields `{ path, content }` objects suitable for passing to `ipfs.add`. + +###### Example + +```js +const IPFS = require('ipfs') +const { urlSource } = IPFS + +const ipfs = await IPFS.create() + +for await (const file of ipfs.add(urlSource('https://ipfs.io/images/ipfs-logo.svg'))) { + console.log(file) +} + +/* +{ + path: 'ipfs-logo.svg', + cid: CID('QmTqZhR6f7jzdhLgPArDPnsbZpvvgxzCZycXK7ywkLxSyU'), + size: 3243 +} +*/ +``` + ## FAQ #### How to enable WebRTC support for js-ipfs in the Browser diff --git a/examples/browser-add-readable-stream/index.js b/examples/browser-add-readable-stream/index.js index 9f7d0c0f7c..7abfb83554 100644 --- a/examples/browser-add-readable-stream/index.js +++ b/examples/browser-add-readable-stream/index.js @@ -38,27 +38,29 @@ const createFiles = (directory) => { }] } -const streamFiles = (ipfs, directory, files) => new Promise((resolve, reject) => { +const streamFiles = async (ipfs, directory, files) => { // Create a stream to write files to - const stream = ipfs.addReadableStream() + const stream = new ReadableStream({ + start(controller) { + for (let i = 0; i < files.length; i++) { + // Add the files one by one + controller.enqueue(files[i]) + } + + // When we have no more files to add, close the stream + controller.close() + } + }) - stream.on('data', (data) => { + for await (const data of ipfs.add(stream)) { log(`Added ${data.path} hash: ${data.hash}`) // The last data event will contain the directory hash if (data.path === directory) { - resolve(data.hash) + return data.cid } - }) - - stream.on('error', reject) - - // Add the files one by one - files.forEach(file => stream.write(file)) - - // When we have no more files to add, close the stream - stream.end() -}) + } +} const log = (line) => { document.getElementById('output').appendChild(document.createTextNode(`${line}\r\n`)) diff --git a/examples/browser-browserify/public/index.html b/examples/browser-browserify/public/index.html index 041385a09d..cebc18fcd4 100644 --- a/examples/browser-browserify/public/index.html +++ b/examples/browser-browserify/public/index.html @@ -18,7 +18,7 @@