-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: rendezvous protocol initial implementation #6
Open
vasco-santos
wants to merge
38
commits into
master
Choose a base branch
from
feat/rendezvous-protocol-full-implementation
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 18 commits
Commits
Show all changes
38 commits
Select commit
Hold shift + click to select a range
b8bf044
feat: rendezvous protocol full implementation
vasco-santos d7290df
chore: interface-peer-discovery compliance
vasco-santos b080670
feat: garbage collector
vasco-santos ebb22d1
feat: cookie for discovery
vasco-santos 9765a95
chore: cleanup
vasco-santos b6edaf3
chore: update aegir
vasco-santos 0e304f9
chore: convert to seconds in the wire
vasco-santos b248924
chore: remove unregister comments for response
vasco-santos 47641f7
feat: use signed peer records to exchange multiaddrs
vasco-santos b668c8a
chore: tests
vasco-santos 7e3c541
chore: change readme
vasco-santos 1a1590d
chore: update deps
vasco-santos b357829
chore: remove peer discovery interface as we will be creating libp2p.…
vasco-santos 4abd363
chore: use uint8array instead of buffer
vasco-santos 7763df2
chore: update libp2p integration doc
vasco-santos 63d607b
chore: fix register ttl param return
vasco-santos 5f45c6f
chore: update docs
vasco-santos 640b64f
chore: remove enabled property from libp2p integration doc
vasco-santos 8f6e148
chore: apply suggestions from code review
vasco-santos 894ad2e
chore: separate server and client rendezvous
vasco-santos ee10d69
chore: add docker
vasco-santos 3798fbb
fix: changed default values and moved them into the server with prope…
vasco-santos cdf2f6b
chore: update docs and constants
vasco-santos 9fe0691
chore: add tests for protocol with direct connection to server
vasco-santos 52fa2bd
chore: DoS protection with max registrations
vasco-santos 06d53ac
chore: refactor client
vasco-santos d501d97
chore: add datastore and types
vasco-santos a2d5f83
chore: fix build
vasco-santos 83cd4b7
chore: run with mysql
vasco-santos 9b294f5
feat: gc
vasco-santos 9bf5bcb
chore: add gc tests
vasco-santos 7a569c7
chore: review docs and binary
vasco-santos c297156
chore: add datastore docs and model picture
vasco-santos e1cd224
chore: add library docs
vasco-santos 264ce2a
chore: add docker setup docks
vasco-santos 01ec7bc
chore: remove client code and move server into src
vasco-santos 5f025d3
chore: use connection pool
vasco-santos 2840251
fix: bin stdout addresses and ports correctly
vasco-santos File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
'use strict' | ||
|
||
const Libp2p = require('libp2p') | ||
const { MULTIADDRS_WEBSOCKETS } = require('./test/fixtures/browser') | ||
const Peers = require('./test/fixtures/peers') | ||
const PeerId = require('peer-id') | ||
const WebSockets = require('libp2p-websockets') | ||
const Muxer = require('libp2p-mplex') | ||
const { NOISE: Crypto } = require('libp2p-noise') | ||
|
||
let libp2p | ||
|
||
const before = async () => { | ||
// Use the last peer | ||
const peerId = await PeerId.createFromJSON(Peers[Peers.length - 1]) | ||
|
||
libp2p = new Libp2p({ | ||
addresses: { | ||
listen: [MULTIADDRS_WEBSOCKETS[0]] | ||
}, | ||
peerId, | ||
modules: { | ||
transport: [WebSockets], | ||
streamMuxer: [Muxer], | ||
connEncryption: [Crypto] | ||
}, | ||
config: { | ||
relay: { | ||
enabled: true, | ||
hop: { | ||
enabled: true, | ||
active: false | ||
} | ||
} | ||
} | ||
}) | ||
|
||
await libp2p.start() | ||
} | ||
|
||
const after = async () => { | ||
await libp2p.stop() | ||
} | ||
|
||
module.exports = { | ||
bundlesize: { maxSize: '100kB' }, | ||
hooks: { | ||
pre: before, | ||
post: after | ||
}, | ||
webpack: { | ||
node: { | ||
// this is needed until bcrypto stops using node buffers in browser code | ||
Buffer: true | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,31 +1,42 @@ | ||
sudo: false | ||
language: node_js | ||
cache: npm | ||
stages: | ||
- check | ||
- test | ||
- cov | ||
|
||
matrix: | ||
include: | ||
- node_js: 6 | ||
env: CXX=g++-4.8 | ||
- node_js: 8 | ||
env: CXX=g++-4.8 | ||
# - node_js: stable | ||
# env: CXX=g++-4.8 | ||
node_js: | ||
- '10' | ||
- '12' | ||
|
||
os: | ||
- linux | ||
- osx | ||
- windows | ||
|
||
script: | ||
- npm run lint | ||
- npm run test | ||
- npm run coverage | ||
script: npx nyc -s npm run test:node -- --bail | ||
after_success: npx nyc report --reporter=text-lcov > coverage.lcov && npx codecov | ||
|
||
jobs: | ||
include: | ||
- stage: check | ||
script: | ||
- npx aegir dep-check | ||
- npm run lint | ||
|
||
before_script: | ||
- export DISPLAY=:99.0 | ||
- sh -e /etc/init.d/xvfb start | ||
- stage: test | ||
name: chrome | ||
addons: | ||
chrome: stable | ||
script: | ||
- npx aegir test -t browser -t webworker | ||
|
||
after_success: | ||
- npm run coverage-publish | ||
- stage: test | ||
name: firefox | ||
addons: | ||
firefox: latest | ||
script: | ||
- npx aegir test -t browser -t webworker -- --browsers FirefoxHeadless | ||
|
||
addons: | ||
firefox: 'latest' | ||
apt: | ||
sources: | ||
- ubuntu-toolchain-r-test | ||
packages: | ||
- g++-4.8 | ||
notifications: | ||
email: false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# Rendezvous Protocol in js-libp2p | ||
|
||
The rendezvous protocol can be used in different contexts across libp2p. For using it, the libp2p network needs to have well known libp2p nodes acting as rendezvous servers. These nodes will have an extra role in the network. They will collect and maintain a list of registrations per rendezvous namespace. Other peers in the network will act as rendezvous clients and will register themselves on given namespaces by messaging a rendezvous server node. Taking into account these registrations, a rendezvous client is able to discover other peers in a given namespace by querying a server. A registration should have a `ttl`, in order to avoid having invalid registrations. | ||
|
||
## Usage | ||
|
||
`js-libp2p` supports the usage of the rendezvous protocol through its configuration. It allows the rendezvous protocol to be enabled, as well as its server mode. | ||
|
||
You can configure it through libp2p as follows: | ||
|
||
```js | ||
const Libp2p = require('libp2p') | ||
const Rendezvous = require('libp2p-rendezvous') | ||
|
||
const node = await Libp2p.create({ | ||
modules: { | ||
rendezvous: Rendezvous | ||
}, | ||
config: { | ||
rendezvous: { | ||
server: { | ||
enabled: false | ||
} | ||
} | ||
} | ||
}) | ||
``` | ||
|
||
While `js-libp2p` supports the rendezvous protocol out of the box through its discovery API, it also provides a rendezvous API that users can interact with. This API allows users to register new rendezvous namespaces, unregister from previously registered namespaces and to manually discover peers. | ||
|
||
## Libp2p Flow | ||
|
||
When a libp2p node with the rendezvous protocol enabled starts, it should start by connecting to a rendezvous server. The rendezvous server can be added to the bootstrap nodes or manually dialed. WHen a rendezvous server is connected, the node can ask for nodes in given namespaces. An example of a namespace could be a relay namespace, so that undiable nodes can register themselves as reachable through that relay. | ||
vasco-santos marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
When a libp2p node running the rendezvous protocol is stopping, it will unregister from all the namespaces previously registered. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,165 @@ | ||
# libp2p-rendezvous | ||
# js-libp2p-rendezvous | ||
|
||
A javascript implementation of the rendezvous protocol for libp2p | ||
[](http://protocol.ai) | ||
[](http://libp2p.io/) | ||
[](http://webchat.freenode.net/?channels=%23libp2p) | ||
[](https://discuss.libp2p.io) | ||
|
||
> Javascript implementation of the rendezvous protocol for libp2p | ||
|
||
## Overview | ||
|
||
Libp2p rendezvous is a lightweight mechanism for generalized peer discovery. It can be used for bootstrap purposes, real time peer discovery, application specific routing, and so on. Any node implementing the rendezvous protocol can act as a rendezvous point, allowing the discovery of relevant peers in a decentralized fashion. | ||
|
||
See https://github.com/libp2p/specs/tree/master/rendezvous for more details | ||
|
||
## Lead Maintainer | ||
|
||
[Vasco Santos](https://github.com/vasco-santos). | ||
|
||
See https://github.com/libp2p/specs/pull/44 for more details | ||
## Usage | ||
|
||
```js | ||
const Libp2p = require('libp2p') | ||
const Rendezvous = require('libp2p-rendezvous') | ||
|
||
const libp2p = await Libp2p.create({ | ||
// check on js-libp2p repo the options to provide | ||
}) | ||
const rendezvous = new Rendezvous({ libp2p }) // Set other options below | ||
|
||
await libp2p.start() | ||
rendezvous.start() | ||
|
||
// ... | ||
|
||
rendezvous.stop() | ||
await libp2p.stop() | ||
``` | ||
|
||
## API | ||
|
||
### constructor | ||
|
||
Creating an instance of Rendezvous. | ||
|
||
`const rendezvous = new Rendezvous({ libp2p })` | ||
|
||
#### Parameters | ||
|
||
| Name | Type | Description | | ||
|------|------|-------------| | ||
| params | `object` | rendezvous parameters | | ||
| params.libp2p | `Libp2p` | a libp2p node instance | | ||
| params.server | `object` | rendezvous server options | | ||
| params.server.enabled | `boolean` | rendezvous server enabled (default: `true`) | | ||
| params.server.gcInterval | `number` | rendezvous garbage collector interval (default: `3e5`) | | ||
|
||
### rendezvous.start | ||
|
||
Register the rendezvous protocol topology into libp2p. The rendezvous server will be started if enabled, as well as the service to keep self registrations available. | ||
|
||
`rendezvous.start()` | ||
|
||
### rendezvous.stop | ||
|
||
Unregister the rendezvous protocol and the streams with other peers will be closed. | ||
|
||
`rendezvous.stop()` | ||
|
||
### rendezvous.register | ||
|
||
Registers the peer in a given namespace. | ||
|
||
`rendezvous.register(namespace, [options])` | ||
|
||
#### Parameters | ||
|
||
| Name | Type | Description | | ||
|------|------|-------------| | ||
| namespace | `string` | namespace to register | | ||
| options | `object` | rendezvous registrations options | | ||
| options.ttl | `number` | registration ttl in ms (default: `7200e3` and minimum `120`) | | ||
|
||
#### Returns | ||
|
||
| Type | Description | | ||
|------|-------------| | ||
| `Promise<number>` | Remaining ttl value | | ||
|
||
#### Example | ||
|
||
```js | ||
// ... | ||
const ttl = await rendezvous.register(namespace) | ||
``` | ||
|
||
### rendezvous.unregister | ||
|
||
Unregisters the peer from a given namespace. | ||
|
||
`rendezvous.unregister(namespace)` | ||
|
||
#### Parameters | ||
|
||
| Name | Type | Description | | ||
|------|------|-------------| | ||
| namespace | `string` | namespace to unregister | | ||
|
||
#### Returns | ||
|
||
| Type | Description | | ||
|------|-------------| | ||
| `Promise<void>` | Operation resolved | | ||
|
||
#### Example | ||
|
||
```js | ||
// ... | ||
await rendezvous.register(namespace) | ||
await rendezvous.unregister(namespace) | ||
``` | ||
|
||
### rendezvous.discover | ||
|
||
Discovers peers registered under a given namespace. | ||
|
||
`rendezvous.discover(namespace, [limit])` | ||
|
||
#### Parameters | ||
|
||
| Name | Type | Description | | ||
|------|------|-------------| | ||
| namespace | `string` | namespace to discover | | ||
| limit | `number` | limit of peers to discover | | ||
|
||
#### Returns | ||
|
||
| Type | Description | | ||
|------|-------------| | ||
| `AsyncIterable<{ signedPeerRecord: Envelope, ns: string, ttl: number }>` | Async Iterable registrations | | ||
|
||
#### Example | ||
|
||
```js | ||
// ... | ||
await rendezvous.register(namespace) | ||
|
||
for await (const reg of rendezvous.discover(namespace)) { | ||
console.log(reg.signedPeerRecord, reg.ns, reg.ttl) | ||
} | ||
``` | ||
|
||
## Contribute | ||
|
||
Feel free to join in. All welcome. Open an [issue](https://github.com/libp2p/js-libp2p-pubsub-peer-discovery/issues)! | ||
|
||
This repository falls under the IPFS [Code of Conduct](https://github.com/ipfs/community/blob/master/code-of-conduct.md). | ||
|
||
[](https://github.com/ipfs/community/blob/master/contributing.md) | ||
|
||
## License | ||
|
||
MIT - Protocol Labs 2020 | ||
|
||
[multiaddr]: https://github.com/multiformats/js-multiaddr |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO: clarify the language here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fyi, this is mostly just a note for me.