Skip to content
This repository has been archived by the owner on Aug 23, 2019. It is now read-only.

Revisit Swarm - multitransport + upgrades - https://github.com/diasdavid/node-ipfs-swarm/issues/8 #10

Merged
merged 18 commits into from
Sep 23, 2015
Merged

Revisit Swarm - multitransport + upgrades - https://github.com/diasdavid/node-ipfs-swarm/issues/8 #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
544e4a4
update README with new candidate interface
daviddias Sep 20, 2015
1833ded
making progress
daviddias Sep 21, 2015
73a6a4f
adding transports works
daviddias Sep 21, 2015
e1df0b9
add transport and close listener test
daviddias Sep 22, 2015
5e4cca5
dial a conn + test
daviddias Sep 22, 2015
8e8d8e9
dial on a protocol + test
daviddias Sep 22, 2015
59b00f6
use warmed up connection + test
daviddias Sep 22, 2015
416e107
quick fix for travis
daviddias Sep 22, 2015
0040be7
add spdy + test
daviddias Sep 23, 2015
168d01b
stream muxer for connection reuse test
daviddias Sep 23, 2015
0bcbe63
rm old code
daviddias Sep 23, 2015
2000827
add identify
daviddias Sep 23, 2015
fb37b4d
clear unused console logs
daviddias Sep 23, 2015
e6bcde4
change cov
daviddias Sep 23, 2015
92b499d
fix readme typos and missing links
daviddias Sep 23, 2015
5fe9444
rm old test file
daviddias Sep 23, 2015
5b7a605
comment undone tests
daviddias Sep 23, 2015
1ba8e80
rm laf
daviddias Sep 23, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 33 additions & 46 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,82 +1,69 @@
ipfs-swarm Node.js implementation
libp2p-swarm Node.js implementation
=================================

[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io) [![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/) [![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs) [![Build Status](https://img.shields.io/travis/diasdavid/node-ipfs-swarm/master.svg?style=flat-square)](https://travis-ci.org/diasdavid/node-ipfs-swarm)

> IPFS swarm implementation in Node.js
> libp2p swarm implementation in Node.js

# Description

ipfs-swarm is an abstraction for the network layer on IPFS. It offers an API to open streams between peers on a specific protocol.
libp2p-swarm is a connection abstraction that is able to leverage several transports and connection upgrades, such as congestion control, channel encryption, multiplexing several streams in one connection, and more. It does this by bringing protocol multiplexing to the application level (instead of the traditional Port level) using multicodec and multistream.

Ref spec (WIP) - https://github.com/diasdavid/specs/blob/protocol-spec/protocol/layers.md#network-layer
libp2p-swarm is used by libp2p but it can be also used as a standalone module.

# Usage

### Create a new Swarm
### Install and create a Swarm

```javascript
var Swarm = require('ipfs-swarm')
libp2p-swarm is available on npm and so, like any other npm module, just:

var s = new Swarm([port]) // `port` defalts to 4001
```bash
$ npm install libp2p-swarm --save
```

### Set the swarm to listen for incoming streams
And use it in your Node.js code as:

```javascript
s.listen([port], [callback]) // `port` defaults to 4001, `callback` gets called when the socket starts listening
```

### Close the listener/socket and every open stream that was multiplexed on it
```JavaScript
var Swarm = require('libp2p-swarm')

```javascript
s.closeListener()
var sw = new Swarm(peerInfoSelf)
```

### Register a protocol to be handled by an incoming stream

```javascript
s.registerHandler('/name/protocol/you/want/version', function (stream) {})
```
peerInfoSelf is a [PeerInfo](https://github.com/diasdavid/node-peer-info) object that represents the peer creating this swarm instance.

### Open a new connection
### Support a transport

Used when we want to make sure we can connect to a given peer, but do not intend to establish a stream with any of the services offered right away.
libp2p-swarm expects transports that implement [abstract-transport](https://github.com/diasdavid/abstract-transport). For example [libp2p-tcp](https://github.com/diasdavid/node-libp2p-tcp), a simple shim on top of the `net` module to make it work with swarm expectations.

```
s.openConnection(peerConnection, function (err) {})
```JavaScript
sw.addTransport(transport, [options, dialOptions, listenOptions])
```

### Add a connection upgrade

### Dial a new stream
A connection upgrade must be able to receive and return something that implements the [abstract-connection](https://github.com/diasdavid/abstract-connection) interface.

```JavaScript
sw.addUpgrade(connUpgrade, [options])
```
s.openStream(peerInfo, protocol, function (err, stream) {})
```

peerInfo must be a [`ipfs-peer`](https://www.npmjs.com/package/ipfs-peer) object, contaning both peer-id and multiaddrs.

## Events emitted
Upgrading a connection to use a stream muxer is still considered an upgrade, but a special case since once this connection is applied, the returned obj will implement the [abstract-stream-muxer](https://github.com/diasdavid/abstract-stream-muxer) interface.

```JavaScript
sw.addStreamMuxer(streamMuxer, [options])
```
.on('error')

.on('connection')
.on('connection-unknown') // used by Identify to start the Identify protocol from listener to dialer
```

## Identify protocol

The Identify protocol is an integral part to Swarm. It enables peers to share observedAddrs, identities and other possible address available. This enables us to do better NAT traversal.

To instantiate Identify:
### Dial to another peer

```JavaScript
sw.dial(PeerInfo, options, protocol)
sw.dial(PeerInfo, options)
```
var Identify = require('ipfs-swarm/identify')

var i = new Identify(swarmInstance, peerSelf)
```
dial uses the best transport (whatever works first, in the future we can have some criteria), and jump starts the connection until the point we have to negotiate the protocol. If a muxer is available, then drop the muxer onto that connection. Good to warm up connections or to check for connectivity. If we have already a muxer for that peerInfo, than do nothing.

`swarmInstance` must be an Instance of swarm and `peerSelf` must be a instance of `ipfs-peer` that represents the peer that instantiated this Identify
### Accept requests on a specific protocol

Identify emits a `peer-update` event each time it receives information from another peer.
```JavaScript
sw.handleProtocol(protocol, handlerFunction)
```
0 examples/c.js → examples/peerA.js
View file
Open in desktop
File renamed without changes.
14 changes: 14 additions & 0 deletions examples/peerB.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
var Swarm = require('./../src')

var Peer = require('peer-info')
var Id = require('peer-id')
var multiaddr = require('multiaddr')
var tcp = require('libp2p-tcp')

var mh = multiaddr('/ip4/127.0.0.1/tcp/8010')
var p = new Peer(Id.create(), [])
var sw = new Swarm(p)

sw.addTransport('tcp', tcp, { multiaddr: mh }, {}, {port: 8010}, function () {
console.log('transport added')
})
28 changes: 0 additions & 28 deletions examples/s.js
View file
Open in desktop

This file was deleted.

19 changes: 10 additions & 9 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
{
"name": "ipfs-swarm",
"name": "libp2p-swarm",
"version": "0.4.1",
"description": "IPFS swarm implementation in Node.js",
"main": "src/index.js",
"scripts": {
"test": "./node_modules/.bin/lab tests/*-test.js",
"coverage": "./node_modules/.bin/lab -t 100 tests/*-test.js",
"codestyle": "./node_modules/.bin/standard --format",
"lint": "./node_modules/.bin/standard",
"validate": "npm ls"
"coverage": "./node_modules/.bin/lab -t 88 tests/*-test.js",
"lint": "./node_modules/.bin/standard"
},
"repository": {
"type": "git",
Expand All @@ -24,15 +22,18 @@
},
"homepage": "https://github.com/diasdavid/node-ipfs-swarm",
"pre-commit": [
"codestyle",
"test"
"lint",
"test",
"coverage"
],
"engines": {
"node": "^4.0.0"
},
"devDependencies": {
"code": "^1.4.1",
"lab": "^5.13.0",
"libp2p-spdy": "^0.1.0",
"libp2p-tcp": "^0.1.1",
"precommit-hook": "^3.0.0",
"sinon": "^1.15.4",
"standard": "^4.5.2",
Expand All @@ -42,11 +43,11 @@
"async": "^1.3.0",
"ip-address": "^4.0.0",
"ipfs-logger": "^0.1.0",
"ipfs-peer": "^0.3.0",
"ipfs-peer-id": "^0.3.0",
"multiaddr": "^1.0.0",
"multiplex-stream-muxer": "^0.2.0",
"multistream-select": "^0.6.1",
"peer-id": "^0.3.3",
"peer-info": "^0.3.2",
"protocol-buffers-stream": "^1.2.0",
"spdy-stream-muxer": "^0.6.0"
}
Expand Down
141 changes: 79 additions & 62 deletions src/identify/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,93 +4,110 @@
*/

var Interactive = require('multistream-select').Interactive
var EventEmmiter = require('events').EventEmitter
var util = require('util')
var protobufs = require('protocol-buffers-stream')
var fs = require('fs')
var schema = fs.readFileSync(__dirname + '/identify.proto')
var v6 = require('ip-address').v6
var Id = require('ipfs-peer-id')
var Id = require('peer-id')
var multiaddr = require('multiaddr')

exports = module.exports = Identify
exports = module.exports = identify

util.inherits(Identify, EventEmmiter)
var protoId = '/ipfs/identify/1.0.0'

function Identify (swarm, peerSelf) {
var self = this
self.createProtoStream = protobufs(schema)
exports.protoId = protoId
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs module.exports?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check line 14 exports = module.exports = identify that is typically the pattern I go for :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Word.

var createProtoStream = protobufs(schema)

swarm.registerHandler('/ipfs/identify/1.0.0', function (stream) {
var ps = self.createProtoStream()
function identify (muxedConns, peerInfoSelf, socket, conn, muxer) {
var msi = new Interactive()
msi.handle(conn, function () {
msi.select(protoId, function (err, ds) {
if (err) {
return console.log(err) // TODO Treat error
}

ps.on('identify', function (msg) {
updateSelf(peerSelf, msg.observedAddr)
var ps = createProtoStream()

var peerId = Id.createFromPubKey(msg.publicKey)
ps.on('identify', function (msg) {
var peerId = Id.createFromPubKey(msg.publicKey)

updateSelf(peerInfoSelf, msg.observedAddr)

muxedConns[peerId.toB58String()] = {
muxer: muxer,
socket: socket
}

// TODO: Pass the new discovered info about the peer that contacted us
// to something like the Kademlia Router, so the peerInfo for this peer
// is fresh
// - before this was exectued through a event emitter
// self.emit('peer-update', {
// peerId: peerId,
// listenAddrs: msg.listenAddrs.map(function (mhb) {return multiaddr(mhb)})
// })
})

var socket = swarm.connections[peerId.toB58String()].socket
var mh = getMultiaddr(socket)

ps.identify({
protocolVersion: 'na',
agentVersion: 'na',
publicKey: peerSelf.id.pubKey,
listenAddrs: peerSelf.multiaddrs.map(function (mh) {return mh.buffer}),
publicKey: peerInfoSelf.id.pubKey,
listenAddrs: peerInfoSelf.multiaddrs.map(function (mh) {
return mh.buffer
}),
observedAddr: mh.buffer
})

self.emit('peer-update', {
peerId: peerId,
listenAddrs: msg.listenAddrs.map(function (mhb) {return multiaddr(mhb)})
})

ps.pipe(ds).pipe(ps)
ps.finalize()
})
ps.pipe(stream).pipe(ps)
})
}

exports.getHandlerFunction = function (peerInfoSelf, muxedConns) {
return function (conn) {
// wait for the other peer to identify itself
// update our multiaddr with observed addr list
// then get the socket from our list of muxedConns and send the reply back

var ps = createProtoStream()

ps.on('identify', function (msg) {
updateSelf(peerInfoSelf, msg.observedAddr)

swarm.on('connection-unknown', function (conn, socket) {
conn.dialStream(function (err, stream) {
if (err) { return console.log(err) }
var msi = new Interactive()
msi.handle(stream, function () {
msi.select('/ipfs/identify/1.0.0', function (err, ds) {
if (err) { return console.log(err) }

var ps = self.createProtoStream()

ps.on('identify', function (msg) {
var peerId = Id.createFromPubKey(msg.publicKey)

updateSelf(peerSelf, msg.observedAddr)

swarm.connections[peerId.toB58String()] = {
conn: conn,
socket: socket
}

self.emit('peer-update', {
peerId: peerId,
listenAddrs: msg.listenAddrs.map(function (mhb) {return multiaddr(mhb)})
})
})

var mh = getMultiaddr(socket)

ps.identify({
protocolVersion: 'na',
agentVersion: 'na',
publicKey: peerSelf.id.pubKey,
listenAddrs: peerSelf.multiaddrs.map(function (mh) {return mh.buffer}),
observedAddr: mh.buffer
})

ps.pipe(ds).pipe(ps)
ps.finalize()
})
var peerId = Id.createFromPubKey(msg.publicKey)

var socket = muxedConns[peerId.toB58String()].socket

var mh = getMultiaddr(socket)

ps.identify({
protocolVersion: 'na',
agentVersion: 'na',
publicKey: peerInfoSelf.id.pubKey,
listenAddrs: peerInfoSelf.multiaddrs.map(function (mh) {
return mh.buffer
}),
observedAddr: mh.buffer
})

// TODO: Pass the new discovered info about the peer that contacted us
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate comment. Delete lines 97-106.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll keep these though as it is a good note for me to remember. But you are right, it is a dup comment, thank you for catching that

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still have a dupe comment? Other than that, good to merge.

// to something like the Kademlia Router, so the peerInfo for this peer
// is fresh
// - before this was exectued through a event emitter
// self.emit('peer-update', {
// peerId: peerId,
// listenAddrs: msg.listenAddrs.map(function (mhb) {
// return multiaddr(mhb)
// })
// })

ps.finalize()
})
})
ps.pipe(conn).pipe(ps)
}
}

function getMultiaddr (socket) {
Expand Down
2 changes: 0 additions & 2 deletions src/stream-muxer.js
View file
Open in desktop

This file was deleted.

Loading