Peer and Content Routing (DHT) #142

dignifiedquire · 2017-04-04T02:27:53Z

WIP

daviddias · 2017-04-04T02:39:14Z

src/index.js

+          return callback(err)
+        }
+
+        this.wm.wantBlocks([cid])


move wantBlocks out, it can be done in parallel

daviddias · 2017-04-04T02:39:33Z

src/index.js

@@ -198,7 +204,14 @@ class Bitswap {
      }

      addListener()
-      this.wm.wantBlocks([cid])
+
+      this.network.findAndConnect(cid, CONSTANTS.maxProvidersPerRequest, (err) => {


We will need to avoid doing a DHT query for every block

daviddias · 2017-04-16T10:38:38Z

src/components/network/index.js

+      pull(pull.empty, conn)
+      callback()
+    })
+  }


Dialing without a protocol doesn't create a conn because it never opens a multiplex conn, it is what is called 'warming up a conn'.

This can be just: this.libp2p.dial(peerId, callback)

daviddias · 2017-04-16T10:41:36Z

Left:

update deps (use libp2p with DHT)
Write tests for get and getMany
Bubble up getMany to BlockService and Unixfs-Engine (separate PRs)

daviddias · 2017-04-16T12:16:53Z

This is ready for a round of review.

I feel that network.js is wrapping libp2p APIs more than needed. I understand that the origin is because this module was a map from go-ipfs-bitswap and so it carries all of its weight when it comes to APIs, we can trim it though :)

dignifiedquire

lgtm

daviddias · 2017-07-13T08:46:53Z

Quick realization. Bitswap needs to still work if no DHT is passed. Will update the PR with all the updated deps and make it work that way.

daviddias · 2017-07-15T12:23:03Z

To merge this PR, we need:

Tests that test it with and without a DHT, so make sure it works in both cases.
Make start and stop async operations (it's weird not to)

daviddias · 2017-07-15T21:09:47Z

test/bitswap.js

+
+      // setTimeout(() => nodes[0].bitswap.unwant(block.cid), 2000)
+    })
+  })


I noticed something strange with this test. It seems that bitswap gives up too easily in trying to find a block. See the logs, even before it sends out the want list, it just returns.

» DEBUG=bitswap:* npm run test:node > ipfs-bitswap@0.14.0 test:node /Users/koruza/code/js-ipfs-bitswap > aegir-test node [22:07:09] Starting 'test:node'... [22:07:09] Starting 'mocha'... bitswap without DHT ✓ connect 0 -> 1 && 1 -> 2 (62ms) Sat, 15 Jul 2017 21:07:10 GMT bitswap:error Failed to provide: DHT is not available Sat, 15 Jul 2017 21:07:10 GMT bitswap:wantmanager adding to wl --> I have concluded that I cant access the block Sat, 15 Jul 2017 21:07:10 GMT bitswap:wantmanager:queue sending message Sat, 15 Jul 2017 21:07:10 GMT bitswap:network sendMessage to QmcuySKjW2Xnsm9ETu2d8XAwxRgvqmYe26kv9akYEWcmuV BitswapMessage { full: false, wantlist: Map { '\u0012 M�\u001d��8�\u0019�\u0019��+I!\u001f1,<��\u0017]�t��' => BitswapMessageEntry { entry: [Object], cancel: false } }, blocks: Map {} } Sat, 15 Jul 2017 21:07:10 GMT bitswap:network incomming new bitswap connection: /ipfs/bitswap/1.1.0 Sat, 15 Jul 2017 21:07:10 GMT bitswap:network ending connection

@dignifiedquire thoughts?

I've detected this same problem in a couple of other tests now too.

daviddias · 2017-08-22T08:49:45Z

src/network.js

+  // Dial to the peer and try to use the most recent Bitswap
+  _dialPeer (peer, callback) {
+   // Attempt Bitswap 1.1.0
+    this.libp2p.dial(peer, BITSWAP110, (err, conn) => {


Removing the try/catch block here showed that our mocks were actually not correct. The try catch around these calls was making so that any error that would happen after a dial would end up here to be dealt from bitswap possibly causing a lot of other erros.

daviddias

Just a couple of tidbits, but most is ok :)

I feel happy that this integration is happening. So much work has been put into the project, so good to see it all coming together 🌟

daviddias · 2017-08-24T20:40:43Z

.gitignore

@@ -35,3 +35,5 @@ test/test-repo-for*
 docs

 test/test-repo/datastore
+
+yarn.lock


This could be added next to the package-lock.json

daviddias · 2017-08-24T20:46:01Z

README.md

    │   ├── entry.js
    │   ├── index.js
    │   └── message.proto.js
-    └── wantlist            # (Type) track wanted blocks
+    └── wantlist           # (Type) track wanted blocks
        ├── entry.js
        └── index.js
 ```


A though. I've been the one documenting pretty much the whole Bitswap. I do like to document things and make them easier for other people to use and contribute (its like teaching), however, it is kind of hard to when I'm mostly the one pushing the docs, graphs and writing other ideas down. It would be very helpful when the way that things are designed change (or created), to get the contributor to document as well.

This tree is missing the new notifications.js and I'm sure a lot of other thoughts that went into this. Could you try to capture all of them in writing to the best of your ability so that future selfs or contributors don't have to hit a wall? Thank you :)

@dignifiedquire never received an acknowledgment here. Could you share your feedback?

daviddias · 2017-08-24T20:48:32Z

src/decision-engine/index.js

        const nextBatch = batch.slice()
        batch = []
-        this._sendSafeBlocks(env.peer, nextBatch, cb)
+        this._sendSafeBlocks(env.peer, nextBatch, (err) => {


Just noticed now that we have this env thing. I'm curious to learn why have you adopted this strategy, doesn't seem to match your love for things like Flow where every argument gets a type and every function a signature. Here it is impossible for the reader to figure out the amount of variables that can go into this env object.

I am not sure to be honest, probably just wanted to make it work

no more env :)

daviddias · 2017-08-24T20:48:53Z

src/decision-engine/index.js

+            this._log('sendblock error: %s', err.message)
+          }
+          // not returning the error, so we send as much as we can
+          // as otherwise `eachSeries` would cancel


❤️ thank you for documenting :)

daviddias · 2017-08-24T20:50:53Z

src/index.js

  }

  // handle messages received through the network
  _receiveMessage (peerId, incoming, callback) {
    this.engine.messageReceived(peerId, incoming, (err) => {
      if (err) {
-        log('failed to receive message', incoming)
+        // TODO: Q: why do we just log and not return here?
+        this._log('failed to receive message', incoming)


Could you add that as a comment replacing my TODO: Q?

daviddias · 2017-08-24T20:54:00Z

src/index.js

+        if (err) {
+          this._log.error(err)
+        }
+      })


What about moving this constant to the network module itself, it isn't like the API is exposed and this call can be invoked with another value.

daviddias · 2017-08-24T20:54:51Z

src/network.js

+    this._running = false
+
+    // TODO: move this up to libp2p-swarm
+    // this.libp2p.swarm.setMaxListeners(CONSTANTS.maxListeners)


Could you open a issue for this on Swarm and delete this comment, please?

dignifiedquire · 2017-08-24T21:06:32Z

@diasdavid on the swarm issue, shouldn't we keep the call around until it is actually moved?

daviddias · 2017-08-24T21:10:39Z

@dignifiedquire its one line change in Swarm, open a PR :D

ipfs/js-ipfs-bitswap#142 (comment)

daviddias · 2017-08-24T21:12:25Z

There you go libp2p/js-libp2p-switch#226

dignifiedquire · 2017-08-24T21:20:43Z

@diasdavid I think I fixed all things you mentioned, except for more docs. I'll think about what to write tomorrow and open a separate PR for that.

daviddias · 2017-08-24T21:32:27Z

Thank you :)

Seems that CI is unhappy now though

https://travis-ci.org/ipfs/js-ipfs-bitswap/jobs/268143607#L2299

dignifiedquire · 2017-08-24T21:36:38Z

@diasdavid I don't even:

* feat: increase maxListeners to Infinity ipfs/js-ipfs-bitswap#142 (comment) * fix linting

daviddias · 2017-08-25T10:34:31Z

test/bitswap-mock-internals.js

@@ -109,7 +109,8 @@ describe('bitswap with mocks', () => {
      })
    })

-    it('multi peer', (done) => {
+    it('multi peer', function (done) {
+      this.timeout(10 * 1000)


no more global timeouts? I love it!

daviddias · 2017-08-25T10:52:56Z

test/bitswap-mock-internals.js

@@ -110,7 +110,7 @@ describe('bitswap with mocks', () => {
    })

    it('multi peer', function (done) {
-      this.timeout(10 * 1000)
+      this.timeout(40 * 1000)


next up is 41 :P

dignifiedquire · 2017-08-25T11:26:55Z

@diasdavid this is entirely ridiculous at this point, tests are timing out randomly on travis as soon as I increase timeout on one another starts failing. I don't think we can realistically leave timeouts low and continue using travis.. also at this point the work has nothing to do anymore with the actual content of the PR this has been an issue on master for some time already

refactor(tests): make tests independent without require global set up steps feat: move libp2p just to the network, take libp2p as a whole and not pieces docs: Update README Documentation

fix: getMany call feat: add id to loggers feat(engine): improve message sending test: skip webworker tests

daviddias · 2017-08-25T12:07:50Z

Ok, waiting for a last round of CI.

codecov · 2017-08-25T12:17:09Z

Codecov Report

❗ No coverage uploaded for pull request base (master@dc143e8). Click here to learn what that means.
The diff coverage is 93.1%.

@@            Coverage Diff            @@
##             master     #142   +/-   ##
=========================================
  Coverage          ?   91.16%           
=========================================
  Files             ?       14           
  Lines             ?      736           
  Branches          ?        0           
=========================================
  Hits              ?      671           
  Misses            ?       65           
  Partials          ?        0

Impacted Files	Coverage Δ
src/types/message/index.js	`93.47% <ø> (ø)`
src/notifications.js	`100% <100%> (ø)`
src/types/wantlist/index.js	`96.42% <100%> (ø)`
src/decision-engine/ledger.js	`100% <100%> (ø)`
src/utils.js	`100% <100%> (ø)`
src/decision-engine/index.js	`88.37% <85.71%> (ø)`
src/want-manager/msg-queue.js	`93.75% <85.71%> (ø)`
src/index.js	`82.01% <88.6%> (ø)`
src/want-manager/index.js	`94.91% <95%> (ø)`
src/network.js	`96.29% <96.29%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc143e8...17c464b. Read the comment docs.

* chore: update contributors * chore: release version v0.29.0 * fix: move emitters to last thing in the method (libp2p#218) * fix: move emitters to last thing in the method * fix: setImmediate everything * chore: update contributors * chore: release version v0.29.1 * fix: move 'pull-stream' from devDependencies to dependencies (libp2p#220) 'pull-stream' package is needed in dependencies because it is used in './src/limit-dialer/queue.js'. * chore: update deps * chore: update contributors * chore: release version v0.29.2 * feat: dial to PeerId and/or Multiaddr in addition to PeerInfo (libp2p#222) * chore: update deps * feat: support dial to peerId and/or multiaddr in adition to peerInfo * chore: update CI * chore: update contributors * chore: release version v0.30.0 * chore: no sauce * chore: update deps * chore: update contributors * chore: release version v0.31.0 * fix: use the right callback * chore: update deps * chore: update contributors * chore: release version v0.31.1 * feat: increase maxListeners to Infinity (libp2p#226) * feat: increase maxListeners to Infinity ipfs/js-ipfs-bitswap#142 (comment) * fix linting * chore: update deps * chore: update contributors * chore: release version v0.31.2 * feat: p2p addrs situation (libp2p#229) * chore: update gitignore and CI * chore: update deps * test: update tests to new p2p-webrtc-star multiaddr format * chore: update contributors * chore: release version v0.32.0 * chore: update deps * chore: update contributors * chore: release version v0.32.1 * fix: remove unused protocol-buffers dep (libp2p#230) * chore: update contributors * chore: release version v0.32.2 * chore: update deps * chore: update contributors * chore: release version v0.32.3 * chore: update deps * fix: increase dial timeout * chore: update contributors * chore: release version v0.32.4 * feat: Circuit Relay (libp2p#224) * chore: update deps * chore: update contributors * chore: release version v0.33.0 * fix: don't dial on relay if not enabled (libp2p#234) * chore: update deps * chore: fix package.json * chore: update contributors * chore: release version v0.33.1 * chore: update deps * fix: don't dial circuit if no transports available (libp2p#236) * chore: update contributors * chore: release version v0.33.2 * fix: circuit dialing * feat: fix circuit dialing * chore: upgrade deps * chore: update circle ci config * chore: adding missing dev dependency * fix: removing unused dependency * test: adding tests * fix: remove unused dep * chore: updating CI files (libp2p#238) * chore: update contributors * chore: release version v0.34.0 * chore: use latest SECIO API * chore: update deps * feat: use latest secio API * chore: update deps * chore: update contributors * chore: release version v0.35.0 * chore: update deps * chore: update contributors * chore: release version v0.35.1 * docs: update name references and API touches * chore: update name references * refactor: update name to switch, make it a class and rename start and stop methods * test: refactor tcp transport tests to avoid code duplication * test: reuse same test code for Websockets, remove code duplication * test: update aegir pre and post hooks * chore: use pre-push instead * test: update and deduplicate code on stream muxing tests * test: restructure test suits * test: refactor swarm-no-muxing tests * test: refactor circuit-relay tests * test: refactor browser tests too * style: fix linting * fix: enableCircuitRelay is async and therefore needs a callback * fix: transports.add does not need to be async at all * docs: fix badges * test: Linux does not like that we use multiple sockets with port 0 * test: fix test * chore: update contributors * chore: release version v0.36.0 * chore: update deps * chore: update contributors * chore: release version v0.36.1 * feat: use mplex, update CI * docs: typo * feat: observe traffic and expose statistics (libp2p#243) * chore: update deps * chore: update contributors * chore: release version v0.37.0 * fix: for when handler func is not defined * fix: for when peerinfo resolves to undefined * chore: update contributors * chore: release version v0.37.1 * chore: update deps * chore: update contributors * chore: release version v0.37.2 * fix: one more observer edge case * chore: update deps * chore: fix linting * test: fix transport tests before all step by increasing the timeout * chore: update contributors * chore: release version v0.37.3 * chore: update deps Chore which i think fixes this issue also https://github.com/libp2p/js-libp2p-switch/issues/235 * fix: revert version back to the current release fix for https://github.com/libp2p/js-libp2p-switch/pull/249/files#r178832198 * chore: update deps * chore: update deps * chore: update contributors * chore: update deps * test: timeout * chore: update contributors * chore: release version v0.39.0 * chore: update deps * chore: update contributors * chore: update deps * chore: update contributors * chore: release version v0.39.2 * feat: improve circuit err messages (libp2p#250) * feat: improve circuit err handling * feat: add test to to validate err when circuit not enabled * refactor: update files and add jsdocs to improve readability refactor: initial refactor of dial.js refactor: add more jsdocs to dial and clean up some code refactor: make get-peer-info more readable fix: jsdocs in dial docs: update some jsdocs refactor: make dial.js a bit easier to consume fix: fix linting docs: add more jsdocs and comments refactor: clean up dial methods and encryption order * test: add tests for get-peer-info * docs: remove answered todo comment answered at libp2p/js-libp2p-switch#252 (comment) * fix: dont create base conn when muxed exists * fix: tests and conflicts * chore: update deps * chore: update contributors * chore: release version v0.40.0 * test: fix require of multiplex * fix: libp2p#189 Prevent self-dial * test: add selfdial test * chore: add lead maintainer * chore: update contributors * chore: update contributors * chore: release version v0.40.1 * fix: return on call to nextMuxer When the call to multistream.Dialer.select is unsuccessful, call nextMuxer to try select the next one in the list but do not continue executing callback afterwards. License: MIT Signed-off-by: Alan Shaw <alan@tableflip.io> * fix: drop connection when stream ends unexpectedly Pull streams pass true in the error position when the sream ends. In https://github.com/multiformats/js-multistream-select/blob/5b19358b91850b528b3f93babd60d63ddcf56a99/src/select.js#L18-L21 ...we're getting lots of instances of pull-length-prefixed stream erroring early with `true` and it's passed back up to the dialer in https://github.com/libp2p/js-libp2p-switch/blob/fef2d11850379a4720bb9c736236a81a067dc901/src/dial.js#L238-L241 The `_createMuxedConnection` contains an assumption that any error that occurs when trying `_attemptMuxerUpgrade` is ok, and keeps the relveant baseConnecton in the cache. If the pull-stream has ended unexpectedly then keeping the connection arround starts causing the "already piped" errors when we try and use the it later. This PR adds a guard to avoid putting the connection back into the cache if the stream has ended. There is related work in an old PR to add a check for exactly this issue in pull-length-prefixed dignifiedquire/pull-length-prefixed#8 ...but it's still open, so this PR adds a check for true in the error position at the site where the "already piped" errors were appearing. Once the PR on pull-length-prefixed is merged this check can be removed. It's not ideal to have it in this code as it is far removed from the source, but it fixes the issue for now. Arguably anywhere that `msDialer.handle` is called should do the same check, but we're not seeing this error occur anywhere else so to keep this PR small, I've left it as the minimal changeset to fix the issue. Of note, we had to add '/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star' to the swarm config to trigger the "already piped" errors. There is a minimal test app here https://github.com/tableflip/js-ipfs-already-piped-error Manual testing shows ~50 streams fail in the first 2 mins of running a node, and then things stabalise with ~90 active muxed connections after that. Fixes libp2p#235 Fixes ipfs/js-ipfs#1366 See dignifiedquire/pull-length-prefixed#8 License: MIT Signed-off-by: Oli Evans <oli@tableflip.io> * fix: add utility methods to prevent already piped error * chore: update contributors * chore: release version v0.40.2 * fix: prevent undefined error during a mutual hangup * chore: update contributors * chore: release version v0.40.3 * feat: swap quick-lru by hashlru This removes the only dependency using generators in the ipfs/libp2p ecosystem. Next version of create-react-app will support ipfs out-of-box with this change. * chore: update contributors * chore: release version v0.40.4 * fix: stats - observer expects protocolTag * fix: re-enable stats tests in node * chore: Upgrade big.js to 5.1.2 * chore: Change require('big.js') to require('big.js').Big * chore: update contributors * chore: release version v0.40.5 * fix: no stats on multistream proto dial * fix: adjust test values * fix: handle error in protocol handshake * chore: update contributors * chore: release version v0.40.6 * chore: remove travis and circleci * Add private network support (libp2p#266) * feat: add support for private networks fix: update protector.protect usage chore: fix linting and update deps test: add secio to pnet tests docs: add private network info the readme chore: update pnet package version test: add skipped test back in and update it * fix: improve erroring around invalid peers docs: add some comments chore: update deps test: simplify identify test * chore: update contributors * chore: release version v0.40.7 * test: add sample network circuit relay tests (libp2p#275) * test: add sample network circuit relay tests * test: use ephemeral ports * chore: update deps chore: remove test pre-push chore: update test ports * chore: update contributors * chore: release version v0.40.8 * chore: update mplex and stats test numbers * feat: make switch a state machine (libp2p#278) * feat: add basic state machine functionality to switch * feat: make connections state machines * refactor: clean up logs * feat: add dialFSM to the switch * feat: add better support for closing connections * test: add tests for some uncovered lines * feat: add warning emitter for muxer upgrade failed * docs: update readme * chore: update contributors * chore: release version v0.41.0 * fix: ignore dial request when one is in progress (libp2p#283) * chore: update contributors * chore: release version v0.41.1 * fix: improve connection closing and error handling (libp2p#285) * fix: improve connection closing and error handling * test: improve identify test * chore: update deps * fix: only emit from connections if there is a listener * test: add more connection tests * chore: update libp2p-mplex * fix: dont dial an address that we have * fix: ensure circuit listens last on start * chore: update npm publish files * chore: update contributors * chore: release version v0.41.2 * fix: use retimer to avoid creating so many timers (libp2p#289) * use retimer to avoid scheduling so many timers * Fixed linting * fix: improve connection tracking and closing (libp2p#291) * chore: update deps * fix: check we have a proper transport before filtering addresses * fix: improve connection close on stop * fix: improve stat stopping * test: fix stats test * fix: improve tracking of open connections * chore: remove log * fix: stats stop in browser chore: fix linting and browser tests * fix: remove uneeded set peer info * fix: abort the base connection on close * fix: catch edge cases of dialTimeout calling back twice * fix: close all connections instead of checking peerbook peers * test: update dial fsm test waits * test: make parallel dial tests deterministic fix: improve logic around disconnecting fix: remove duplicate event handling logic * chore: fix lint * test: improve test reliability * chore: update contributors * chore: release version v0.41.3 * refactor: stat use for over forEach (libp2p#295) forEach is 10x slower than a regular for(;;) loop, and it should be avoided in hot code paths. * fix: avoid sync callback in async functions (libp2p#297) * fix: avoid sync callback in async functions * test: add error check * refactor: clean up async usage * chore: clean up * refactor: remove async waterfall usage on identify * chore: fix linting * chore: update contributors * chore: release version v0.41.4 * fix: peerBook undefined libp2p#299 * fix: reduce bundle size (libp2p#292) * fix: reduce bundle size * fix: use bignumber everywhere * chore: update deps * chore: update contributors * chore: release version v0.41.5 * fix: import async/setImmediate to avoid webpack errors (libp2p#303) * test: add pull-mplex to test suite (libp2p#305) * chore: use travis * chore: update dependencies * fix: dial in series until we have proper abort support (libp2p#306) refactor: simplify the circuit dial logic chore: remove travis windows cache refactor: clean up dial many error logic test: explicitly set correct address test(refactor): update order of echo logic and add after refactor: cleanup per feedback * chore: update contributors * chore: release version v0.41.6 * fix: peer disconnect event and improve logging performance (libp2p#309) * fix: only emit disconnects from muxed conns * fix: update disconnect logic * chore: clean up logging to prevent unneeded string formatting * chore: fix spelling * chore: update contributors * chore: release version v0.41.7 * feat: add basic dial queue to avoid many connections to peer (libp2p#310) BREAKING CHANGE: This adds a very basic dial queue peer peer. This will prevent multiple, simultaneous dial requests to the same peer from creating multiple connections. The requests will be queued per peer, and will leverage the same connection when possible. The breaking change here is that `.dial`, will no longer return a connection. js-libp2p, circuit relay, and kad-dht, which use `.dial` were not using the returned connection. So while this is a breaking change it should not break the existing libp2p stack. If custom applications are leveraging the returned connection, they will need to convert to only using the connection returned via the callback. * chore: dont log priviatized unless it actually happened * refactor: only get our addresses for filtering once * feat: update identify to include supported protocols (libp2p#311) * chore: update contributors * chore: release version v0.42.0 * fix: ensure dials always use the latest PeerInfo from the PeerBook (libp2p#312) * fix: ensure dials always use the latest PeerInfo from the PeerBook This fixes an issue where if dial is called with a new instance of PeerInfo, if it is the first dial to that peer, the queue was forever associated with that instance. This is currently the case when Circuit checks the HOP status of a potential relay. This ensures that whenever we dial, we are updating the peer book and using the latest PeerInfo in that dial request. * test: add test for get peer info * refactor: just use id with dialer queue * chore: update contributors * chore: release version v0.42.1 * fix: identify on dial (libp2p#313) * chore: update contributors * chore: release version v0.42.2 * feat: global dial queue (libp2p#314) * feat: add a general queue to limit all dials * fix: improve queue count logic and add better abort * feat: add a basic blacklist * fix: abort dial queue on error instead of stop * feat: add a crude priority lane * test: add test for blacklist error * fix: make blacklist and max dials configurable * refactor: blacklist after callback * test: improve testings around blacklisting * chore: update contributors * chore: release version v0.42.3 * fix: improve dial queue and parallel dials (libp2p#315) * feat: allow dialer queues to do many requests to a peer * fix: parallel dials and validate cancelled conns * feat: make dial timeout configurable * fix: allow already connected peers to dial immediately * refactor: add dial timeout to consts file * fix: keep better track of in progress queues * refactor: make dials race * chore: update contributors * chore: release version v0.42.4 * feat: limit the number of cold calls we can do (libp2p#316) * feat: limit the number of cold calls we can do * feat: add a backoff to blacklisting * refactor: make cold calls configurable * fix: make blacklist duration longer * fix: improve blacklisting * test: add some tests for queue * feat: add jitter to blacklist ttl * test: validate cold queue is removed * feat: purge old queues every hour * test: fix aegir post script node shutdown * fix: abort the cold call queue on manager abort * fix: improve queue cleanup and lower interval to 15 mins * fix: improve connection tracking (libp2p#318) * fix: centralize connection events and peer connects * fix: remove unneeded peerBook put * chore: update contributors * chore: release version v0.42.5 * fix: dont blacklist good peers (libp2p#319) * fix: revert to try each (libp2p#320) * chore: update contributors * chore: release version v0.42.6 * fix: missing queue (libp2p#323) * fix: improve stopping logic (libp2p#324) * chore: update contributors * chore: release version v0.42.7 * chore: add discourse badge (libp2p#327) * fix: dial self (libp2p#329) * feat: support a priority queue for dials (libp2p#325) * chore: update contributors * chore: release version v0.42.8 * fix: dont compare empty strings (libp2p#330) * chore: update contributors * chore: release version v0.42.9 * fix: resolve transport sort order in browsers (libp2p#333) * fix: resolve transport sort order in browsers * fix: update sort logic * fix: dont use peerinfo distinct (libp2p#334) * fix: dont use peerinfo distinct * refactor: remove unneeded code * refactor: clean up * refactor: fix feedback * chore: update contributors * chore: release version v0.42.10 * fix(stats): prevent 0ms timeDiff breaking movingAverage (libp2p#336) * stats - stat - prevent 0ms timeDiff breaking movingAverage * chore: remove commitlint * chore: update contributors * chore: release version v0.42.11 * fix: dont blindly add observed addresses to our list (libp2p#337) Until we can properly validate the observed address our peer tells us about, we shouldnt blindly add it to our address list. Until we have better NAT management we cant reliably validate that we're adding an appropriate address for ourselves. * fix: clear blacklist for peer when connection is established (libp2p#340) * chore: update contributors * chore: release version v0.42.12 * refactor: move switch into src/switch * refactor: cleanup switch and move tests into test dir

dignifiedquire added the status/in-progress In progress label Apr 4, 2017

dignifiedquire force-pushed the feat/dht branch from 88b97a8 to 02a55e2 Compare April 4, 2017 02:29

daviddias reviewed Apr 4, 2017

View reviewed changes

src/index.js Outdated

return callback(err)

}

this.wm.wantBlocks([cid])

Copy link

Member

daviddias Apr 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move wantBlocks out, it can be done in parallel

daviddias reviewed Apr 4, 2017

View reviewed changes

dignifiedquire mentioned this pull request Apr 4, 2017

[WIP] Actually implement this libp2p/js-libp2p-kad-dht#1

Merged

38 tasks

daviddias assigned daviddias and dignifiedquire Apr 5, 2017

daviddias requested changes Apr 16, 2017

View reviewed changes

daviddias unassigned dignifiedquire Apr 16, 2017

daviddias approved these changes Apr 16, 2017

View reviewed changes

dignifiedquire commented Apr 16, 2017

View reviewed changes

daviddias added status/ready Ready to be worked and removed status/in-progress In progress labels Jul 3, 2017

daviddias force-pushed the feat/dht branch 3 times, most recently from e992c01 to 42dda83 Compare July 13, 2017 14:36

daviddias changed the title ~~wip: dht integration~~ wip: Peer and Content Routing (DHT) Jul 13, 2017

daviddias force-pushed the feat/dht branch from 9f97fe0 to ff250de Compare July 15, 2017 21:07

daviddias reviewed Jul 15, 2017

View reviewed changes

daviddias changed the title ~~wip: Peer and Content Routing (DHT)~~ Peer and Content Routing (DHT) Jul 20, 2017

daviddias mentioned this pull request Jul 20, 2017

Awesome Endeavour: DHT Part II ipfs/js-ipfs#856

Merged

35 tasks

daviddias force-pushed the feat/dht branch from ff250de to 2df29c8 Compare July 27, 2017 05:54

feat: Peer and Content Routing Support (DHT) - first round

1dbf264

daviddias force-pushed the feat/dht branch 2 times, most recently from 42c3a62 to 84db9ce Compare August 22, 2017 08:44

daviddias reviewed Aug 22, 2017

View reviewed changes

daviddias requested changes Aug 24, 2017

View reviewed changes

daviddias added a commit to libp2p/js-libp2p-switch that referenced this pull request Aug 24, 2017

feat: increase maxListeners to Infinity

a3ec583

ipfs/js-ipfs-bitswap#142 (comment)

daviddias mentioned this pull request Aug 24, 2017

feat: increase maxListeners to Infinity libp2p/js-libp2p-switch#226

Merged

daviddias added a commit to libp2p/js-libp2p-switch that referenced this pull request Aug 25, 2017

feat: increase maxListeners to Infinity (#226)

77bf1ab

* feat: increase maxListeners to Infinity ipfs/js-ipfs-bitswap#142 (comment) * fix linting

daviddias reviewed Aug 25, 2017

View reviewed changes

daviddias and others added 2 commits August 25, 2017 13:04

feat: support for DHT and without DHT at the same time

84add4d

refactor(tests): make tests independent without require global set up steps feat: move libp2p just to the network, take libp2p as a whole and not pieces docs: Update README Documentation

feat: factor notifications out

f17ecfa

fix: getMany call feat: add id to loggers feat(engine): improve message sending test: skip webworker tests

daviddias force-pushed the feat/dht branch from 759fb97 to f17ecfa Compare August 25, 2017 12:04

chore: fix travis CI... common friedel

17c464b

daviddias approved these changes Aug 25, 2017

View reviewed changes

daviddias mentioned this pull request Aug 25, 2017

Missing webworker support #120

Closed

daviddias merged commit 3a6b8a1 into master Aug 25, 2017

daviddias deleted the feat/dht branch August 25, 2017 12:15

daviddias mentioned this pull request Aug 26, 2017

Bizarre discrepancies between running ipfs.files.cat on images, videos, and text ipfs/js-ipfs#959

Closed

This was referenced Sep 2, 2017

Create better documentation for bitswap + webpage for it #21

Open

Document how DHT is implemented libp2p/js-libp2p-kad-dht#5

Open

daviddias mentioned this pull request Sep 12, 2017

Notifications: More Generic events #151

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Peer and Content Routing (DHT) #142

Peer and Content Routing (DHT) #142

dignifiedquire commented Apr 4, 2017 •

edited by daviddias

Loading

daviddias Apr 4, 2017

daviddias Apr 4, 2017

daviddias Apr 16, 2017

daviddias commented Apr 16, 2017 •

edited

Loading

daviddias commented Apr 16, 2017

dignifiedquire left a comment

daviddias commented Jul 13, 2017

daviddias commented Jul 15, 2017 •

edited

Loading

daviddias Jul 15, 2017

daviddias Jul 15, 2017

daviddias Aug 22, 2017

daviddias Aug 22, 2017

daviddias left a comment •

edited

Loading

daviddias Aug 24, 2017

daviddias Aug 24, 2017

daviddias Sep 12, 2017

daviddias Aug 24, 2017

dignifiedquire Aug 24, 2017

dignifiedquire Aug 24, 2017

daviddias Aug 24, 2017

daviddias Aug 24, 2017

daviddias Aug 24, 2017

daviddias Aug 24, 2017

dignifiedquire commented Aug 24, 2017

daviddias commented Aug 24, 2017

daviddias commented Aug 24, 2017

dignifiedquire commented Aug 24, 2017

daviddias commented Aug 24, 2017 •

edited

Loading

dignifiedquire commented Aug 24, 2017

daviddias Aug 25, 2017

daviddias Aug 25, 2017

dignifiedquire commented Aug 25, 2017

daviddias commented Aug 25, 2017

codecov bot commented Aug 25, 2017 •

edited

Loading

Peer and Content Routing (DHT) #142

Peer and Content Routing (DHT) #142

Conversation

dignifiedquire commented Apr 4, 2017 • edited by daviddias Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daviddias commented Apr 16, 2017 • edited Loading

daviddias commented Apr 16, 2017

dignifiedquire left a comment

Choose a reason for hiding this comment

daviddias commented Jul 13, 2017

daviddias commented Jul 15, 2017 • edited Loading

To merge this PR, we need:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daviddias left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dignifiedquire commented Aug 24, 2017

daviddias commented Aug 24, 2017

daviddias commented Aug 24, 2017

dignifiedquire commented Aug 24, 2017

daviddias commented Aug 24, 2017 • edited Loading

dignifiedquire commented Aug 24, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dignifiedquire commented Aug 25, 2017

daviddias commented Aug 25, 2017

codecov bot commented Aug 25, 2017 • edited Loading

Codecov Report

dignifiedquire commented Apr 4, 2017 •

edited by daviddias

Loading

daviddias commented Apr 16, 2017 •

edited

Loading

daviddias commented Jul 15, 2017 •

edited

Loading

daviddias left a comment •

edited

Loading

daviddias commented Aug 24, 2017 •

edited

Loading

codecov bot commented Aug 25, 2017 •

edited

Loading