Skip to content
This repository has been archived by the owner on Jul 21, 2023. It is now read-only.

fix: refactor query logic #237

Merged
merged 37 commits into from
Nov 18, 2021
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
12d39e0
fix: refactor query logic
achingbrain Oct 6, 2021
8c99da8
chore: remove libp2p from component types
achingbrain Oct 6, 2021
5f953d3
chore: query manager tests passing
achingbrain Oct 7, 2021
36aae55
chore: kad-dht tests passing
achingbrain Oct 8, 2021
d965111
fix: couple of tests left
achingbrain Oct 9, 2021
f4fca0b
chore: tests passing
achingbrain Oct 10, 2021
05b7005
chore: swap travis for gh actions
achingbrain Oct 10, 2021
27707e2
chore: test on node
achingbrain Oct 10, 2021
416e19d
chore: ignore coverage dir
achingbrain Oct 10, 2021
12204ac
chore: update parallel
achingbrain Oct 10, 2021
a234676
chore: make timeouts optional
achingbrain Oct 11, 2021
cc1e3d4
chore: speed up tests
achingbrain Oct 11, 2021
8e5fe2b
chore: allow sub-queries to time out without aborting the whole query
achingbrain Oct 11, 2021
6e287c8
chore: restore prepare script, get datastore from libp2p
achingbrain Oct 11, 2021
546db47
fix: bubbled up query errors should be non-fatal
achingbrain Oct 12, 2021
09eb418
chore: temp fix for fix node 14
achingbrain Oct 12, 2021
e351e02
feat: query self occasionally
achingbrain Oct 12, 2021
0d63aee
chore: linting
achingbrain Oct 12, 2021
43be700
feat: add query event handlers
achingbrain Oct 13, 2021
1c8a125
feat: events all the way down
achingbrain Oct 14, 2021
287a77a
chore: tests passing for events
achingbrain Oct 15, 2021
a29959a
chore: tests passing again
achingbrain Nov 8, 2021
c61ca6c
chore: linting
achingbrain Nov 8, 2021
ac53b24
chore: listen to connection events
achingbrain Nov 8, 2021
93e2dd1
chore: linting
achingbrain Nov 8, 2021
1d5f2f5
chore: do not expose dht messages
achingbrain Nov 8, 2021
ac8698b
chore: fields will always be set
achingbrain Nov 8, 2021
4af423b
chore: avoid leaking libp2p-record in types as it depends on crypto
achingbrain Nov 9, 2021
fffed89
fix: split DHTs into lan and wan
achingbrain Nov 15, 2021
2e96752
chore: rename as events
achingbrain Nov 15, 2021
36e198e
chore: let caller throw and only continue query if xor is closer
achingbrain Nov 16, 2021
6bdf4b4
chore: fix flaky test
achingbrain Nov 16, 2021
04e7aca
chore: add missing dep
achingbrain Nov 16, 2021
e8f882f
chore: better logging
achingbrain Nov 17, 2021
dc47b5c
chore: update implementation summary
achingbrain Nov 17, 2021
1cfea97
chore: address review comments
achingbrain Nov 18, 2021
1ead080
chore: slow ci is slow
achingbrain Nov 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: ci
on:
push:
branches:
- master
pull_request:
branches:
- master

jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- run: npm install
- run: npx aegir lint
- uses: gozala/typescript-error-reporter-action@v1.0.8
- run: npx aegir build
- run: npx aegir dep-check
- uses: ipfs/aegir/actions/bundle-size@master
name: size
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
test-node:
needs: check
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [windows-latest, ubuntu-latest, macos-latest]
node: [14, 16]
fail-fast: true
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node }}
- run: npm install
- run: npx nyc --reporter=lcov aegir test -t node -- --bail
- uses: codecov/codecov-action@v1
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,4 @@ test/test-data/go-ipfs-repo/LOG.old
# while testing npm5
package-lock.json
yarn.lock
.nyc_output
42 changes: 0 additions & 42 deletions .travis.yml

This file was deleted.

26 changes: 14 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# js-libp2p-kad-dht
# js-libp2p-kad-dht <!-- omit in toc -->

[![](https://img.shields.io/badge/made%20by-Protocol%20Labs-blue.svg?style=flat-square)](http://ipn.io)
[![](https://img.shields.io/badge/project-IPFS-blue.svg?style=flat-square)](http://ipfs.io/)
[![](https://img.shields.io/badge/freenode-%23ipfs-blue.svg?style=flat-square)](http://webchat.freenode.net/?channels=%23ipfs)
[![Discourse posts](https://img.shields.io/discourse/https/discuss.libp2p.io/posts.svg)](https://discuss.libp2p.io)
[![Build Status](https://travis-ci.org/libp2p/js-libp2p-kad-dht.svg?style=flat-square)](https://travis-ci.org/libp2p/js-libp2p-kad-dht)
[![Build status](https://github.com/libp2p/js-libp2p-kad-dht/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/libp2p/js-libp2p-kad-dht/actions/workflows/test.yml)
[![Coverage Status](https://coveralls.io/repos/github/libp2p/js-libp2p-kad-dht/badge.svg?branch=master)](https://coveralls.io/github/libp2p/js-libp2p-kad-dht?branch=master)
[![Dependency Status](https://david-dm.org/libp2p/js-libp2p-kad-dht.svg?style=flat-square)](https://david-dm.org/libp2p/js-libp2p-kad-dht)
[![Bundle Size](https://flat.badgen.net/bundlephobia/minzip/libp2p-kad-dht)](https://bundlephobia.com/result?p=libp2p-kad-dht)
Expand All @@ -15,19 +15,23 @@

> JavaScript implementation of the Kademlia DHT for libp2p, based on [go-libp2p-kad-dht](https://github.com/libp2p/go-libp2p-kad-dht).

## Lead Maintainer
## Lead Maintainer <!-- omit in toc -->

[Vasco Santos](https://github.com/vasco-santos).

## Table of Contents
## Table of Contents <!-- omit in toc -->

- [Install](#install)
- [npm](#npm)
- [Use in Node.js](#use-in-nodejs)
- [API](#api)
- [Custom secondary DHT in libp2p](#custom-secondary-dht-in-libp2p)
- [Peer Routing](#peer-routing)
- [Content Routing](#content-routing)
- [Peer Discovery](#peer-discovery)
- [Implementation Summary](#implementation-summary)
- [Contribute](#contribute)
- [License](#license)

## Install

### npm
Expand All @@ -39,7 +43,7 @@
### Use in Node.js

```js
const KadDHT = require('libp2p-kad-dht')
import { create } from 'libp2p-kad-dht'
```

## API
Expand All @@ -51,20 +55,18 @@ The libp2p-kad-dht module offers 3 APIs: Peer Routing, Content Routing and Peer
### Custom secondary DHT in libp2p

```js
import { create } from 'libp2p-kad-dht'

/**
* @param {Libp2p} libp2p
*/
function addDHT(libp2p) {
const customDHT = new KadDHT({
const customDHT = create({
libp2p,
dialer: libp2p.dialer,
peerId: libp2p.peerId,
peerStore: libp2p.peerStore,
registrar: libp2p.registrar,
protocolPrefix: '/custom'
})
customDHT.start()
customDHT.on('peer', libp2p._onDiscoveryPeer)

return customDHT
}
```
Expand Down
24 changes: 14 additions & 10 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
"leadMaintainer": "Vasco Santos <vasco.santos@moxy.studio>",
"main": "src/index.js",
"scripts": {
"lint": "aegir ts -p check && aegir lint",
"prepare": "npm run build",
"lint": "aegir ts -p check && aegir lint",
"build": "npm run build:proto && npm run build:proto-types && aegir build",
"build:proto": "pbjs -t static-module -w commonjs -r libp2p-dht-message --force-number --no-verify --no-delimited --no-create --no-beautify --no-defaults --lint eslint-disable -o src/message/dht.js ./src/message/dht.proto",
"build:proto-types": "pbts -o src/message/dht.d.ts src/message/dht.js",
Expand All @@ -18,7 +18,8 @@
"release-major": "aegir release --type major --docs -t node",
"coverage": "aegir coverage",
"coverage-publish": "aegir-coverage publish",
"sim": "node test/simulation/index.js"
"sim": "node test/simulation/index.js",
"dep-check": "aegir dep-check"
},
"files": [
"src",
Expand Down Expand Up @@ -47,47 +48,50 @@
"homepage": "https://github.com/libp2p/js-libp2p-kad-dht",
"types": "dist/src/index.d.ts",
"dependencies": {
"any-signal": "^2.1.2",
"datastore-core": "^6.0.7",
"debug": "^4.3.1",
"err-code": "^3.0.0",
"hashlru": "^2.3.0",
"heap": "~0.2.6",
"interface-datastore": "^6.0.2",
"it-all": "^1.0.5",
"it-drain": "^1.0.4",
"it-first": "^1.0.4",
"it-length": "^1.0.3",
"it-length-prefixed": "^5.0.2",
"it-map": "^1.0.5",
"it-merge": "^1.0.3",
"it-parallel": "^2.0.1",
"it-pipe": "^1.1.0",
"it-take": "^1.0.2",
"k-bucket": "^5.1.0",
"libp2p-crypto": "^0.19.5",
"libp2p-interfaces": "^1.0.0",
"libp2p-record": "^0.10.4",
"multiaddr": "^10.0.0",
"multiformats": "^9.4.5",
"native-abort-controller": "^1.0.4",
"p-defer": "^3.0.0",
"p-map": "^4.0.0",
"p-queue": "^6.6.2",
"p-timeout": "^4.1.0",
"peer-id": "^0.15.0",
"protobufjs": "^6.10.2",
"streaming-iterables": "^6.0.0",
"timeout-abort-controller": "^1.1.1",
"timeout-abort-controller": "^2.0.0",
"uint8arrays": "^3.0.0",
"varint": "^6.0.0"
},
"devDependencies": {
"@types/debug": "^4.1.7",
"aegir": "^35.0.1",
"async-iterator-all": "^1.0.0",
"crypto-browserify": "^3.12.0",
"datastore-level": "^7.0.1",
"delay": "^5.0.0",
"execa": "^5.1.1",
"it-pair": "^1.0.0",
"libp2p": "^0.32.3",
"libp2p": "^0.33.0",
"lodash.random": "^3.2.0",
"lodash.range": "^3.2.0",
"p-defer": "^3.0.0",
"p-each-series": "^2.1.0",
"p-map-series": "^2.1.0",
"p-retry": "^4.2.0",
"sinon": "^11.1.1",
"which": "^2.0.2"
Expand Down
2 changes: 2 additions & 0 deletions src/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,5 @@ exports.K = 20

// Alpha is the concurrency for asynchronous requests
exports.ALPHA = 3

exports.QUERY_SELF_INTERVAL = Number(minute)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems too frequent, can this be configured? Looks like spec default is 5mins https://github.com/libp2p/specs/tree/master/kad-dht#bootstrap-process.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's exposed as the querySelfInterval factory function option. Will increase the default to 5 mins

Loading