Hole Punching Protocol #711

vyzo · 2019-08-26T11:41:33Z

Implements the Hole Punching co-ordination protocol and upgrades dependencies to solve multi-stream simultaneous connect.

TODO

Deps.
Allow the Hole Punching Stream to open on a transient connection.
Unit Tests (once we are happy with the general direction we are taking here).

bigs · 2019-08-26T18:08:13Z

go.mod

 	github.com/libp2p/go-eventbus v0.1.0
 	github.com/libp2p/go-libp2p-autonat v0.1.0
 	github.com/libp2p/go-libp2p-blankhost v0.1.3
 	github.com/libp2p/go-libp2p-circuit v0.1.1
-	github.com/libp2p/go-libp2p-core v0.2.2
+	github.com/libp2p/go-libp2p-core v0.2.3-0.20190826112323-405451631c05


just noting for when we need to propagate releases

aarshkshah1992 · 2021-01-12T13:02:55Z

Ping @marten-seemann @Stebalien for review.

aarshkshah1992 · 2021-01-21T08:18:17Z

ping @vyzo for review .

p2p/protocol/holepunch/coordination.go

aarshkshah1992 · 2021-01-21T08:25:35Z

p2p/protocol/holepunch/coordination.go

+// attempts to make a direct connection with the remote peer of `relayConn` by co-ordinating a hole punch over
+// the given relay connection `relayConn`.
+func (hs *HolePunchService) holePunch(relayConn network.Conn) {
+	rp := relayConn.RemotePeer()


Same concern as https://github.com/libp2p/go-libp2p/pull/711/files#r561681671.

requilence · 2021-01-26T09:11:35Z

@aarshkshah1992 Hello! I'm looking forward to this PR being merged, as we are facing a lot of troubles with multiplexing with TCP transport.

BTW, I've just run the bidirectional connections test from this issue: libp2p/go-libp2p-noise#70, and it fails: on the 4th-8th iteration h1.Connect(...) returns nil err but the next-line h1.Network().Peers() returns 0 peers.
The same happens when running with NoSecurity option.

aarshkshah1992 · 2021-01-26T16:57:43Z

@requilence That definitely sounds like a bug. I'll take a look !

aarshkshah1992 · 2021-01-27T07:06:35Z

@requilence There was a race in that test wherein a peer could check for peers it is connected to after that connection has been closed by the remote peer depending on when it hit that code path.

I've fixed the test and you can follow this issue at #1041. Once this PR is merge, we can release simopen.

requilence · 2021-01-28T15:22:11Z

@aarshkshah1992 can I clarify? Do both of them share the same connection? For each node, it seems that the connection has been initiated on their side and can be safely closed when the operation on their side is finished.

Here is an example

node A opens a connection to node B and starts some long transfer via stream
at the same time node B opens a connection to node B and downloads a file via bitswap
node B finishes blocks exchange and closes the connection
will it lead to a reset of the stream initiated by node A?

vyzo

this is looking pretty good, a few comments.

Note: I can't green tick this one, as I originally opened the pr.

vyzo · 2021-02-17T17:50:56Z

config/config.go

+	if cfg.EnableHolePunching && !cfg.Relay {
+		return nil, errors.New("cannot enable hole punching; relay is not enabled")
+	}


this will need to use the v2 relay instead of the legacy relay.

vyzo · 2021-02-17T17:53:33Z

p2p/protocol/holepunch/coordination.go

+const (
+	protocol         = "/libp2p/holepunch/1.0.0"
+	maxMsgSize       = 4 * 1024 // 4K
+	holePunchTimeout = 2 * time.Minute


this may exceed the limited relay timeout... maybe we need to increase the default for the latter?

It's 1min currently.

vyzo · 2021-02-17T17:55:36Z

p2p/protocol/holepunch/coordination.go

+
+	// short-circuit hole punching if a direct dial works.
+	// attempt a direct connection ONLY if we have a public address for the remote peer
+	for _, a := range hs.host.Peerstore().Addrs(rp) {


this kind of assumes that identify has completed for the peer, which may not be the case here.

vyzo · 2021-02-17T17:56:08Z

p2p/protocol/holepunch/coordination.go

+	}
+
+	// hole punch
+	s, err := hs.host.NewStream(hs.ctx, rp, protocol)


WithUseTransient(ctx)

vyzo · 2021-02-17T17:57:22Z

p2p/protocol/holepunch/coordination.go

+	if msg.GetType() != holepunch_pb.HolePunch_CONNECT {
+		s.Reset()
+		return


let's log this, probably Debug.

p2p/protocol/holepunch/coordination.go

vyzo · 2021-02-17T18:01:36Z

p2p/protocol/holepunch/pb/holepunch.proto

+  // We plan to have a better address discovery and advertisement mechanism in the future.
+  // See https://github.com/libp2p/go-libp2p-autonat/pull/98
+  repeated bytes ObsAddrs = 2;
+}


newline....

vyzo · 2021-02-17T18:05:45Z

Note that this is getting quite messy in terms of commit history, we might have to squash it.

vyzo · 2021-02-17T18:07:10Z

We will want to integrate the circuit v2 pr here and change the libp2p constructor to use the new beast instead of the legacy one.

aarshkshah1992 · 2021-02-19T04:55:36Z

Closed in favour of #1057.

vyzo requested review from Stebalien and raulk August 26, 2019 11:41

vyzo mentioned this pull request Aug 26, 2019

NAT traversal: Implement support for simultaneous open in mulstistream #712

Closed

5 tasks

bigs reviewed Aug 26, 2019

View reviewed changes

vyzo force-pushed the feat/simopen branch from b047a56 to fde49e1 Compare September 22, 2019 09:20

aarshkshah1992 force-pushed the feat/simopen branch from fde49e1 to d06bee2 Compare January 12, 2021 12:57

aarshkshah1992 changed the title ~~Implement support for simultaneous open~~ [WIP ]Implement support for simultaneous open and hole punching Jan 15, 2021

This was referenced Jan 19, 2021

Close connections where streams haven't been opened since some time libp2p/go-libp2p-connmgr#77

Closed

Project Flare(decentralised Hole Punching) Phase1 Meta Issue #1039

Closed

aarshkshah1992 changed the title ~~[WIP ]Implement support for simultaneous open and hole punching~~ Implement support for simultaneous open and hole punching Jan 21, 2021

aarshkshah1992 requested review from willscott and bigs January 21, 2021 08:17

aarshkshah1992 self-assigned this Jan 21, 2021

aarshkshah1992 requested review from marten-seemann and removed request for bigs January 21, 2021 08:18

aarshkshah1992 reviewed Jan 21, 2021

View reviewed changes

p2p/protocol/holepunch/coordination.go Show resolved Hide resolved

aarshkshah1992 reviewed Jan 21, 2021

View reviewed changes

aarshkshah1992 added 2 commits January 27, 2021 17:26

emit NAT device type

78ce16b

unit tests

0b45239

aarshkshah1992 changed the title ~~Implement support for simultaneous open and hole punching~~ Hole Punching Protocol Jan 28, 2021

aarshkshah1992 added 2 commits January 28, 2021 15:22

emit event for NAT Device Type

3566bfd

updated deps

29d4987

aarshkshah1992 added 2 commits February 1, 2021 17:09

refactor tcp-udp NAT events

5484ccb

update go ver

1537769

aarshkshah1992 added 16 commits February 2, 2021 15:04

test scripts

2603920

default listen

e868d73

addrs

431a9a6

relay server

fa9994c

rela server

001aa52

relay server

14ef1ec

relay server

450d431

addrs

36ca4a7

testing

1bdb86c

server code

1d6d07f

server code

5ecac86

more changes

d9d7e6a

upated swarm

dcf822c

clean-up

2ee5344

cleaned up PR

46ef4d8

rebased to have NAT type event

749086f

aarshkshah1992 force-pushed the feat/simopen branch from dbd9c5e to 749086f Compare February 2, 2021 09:47

aarshkshah1992 added 6 commits February 2, 2021 16:48

better logging

37141ec

remove log

5bf6a8f

log connection as well

837edb0

remove chatty logs

35b7b8a

fix go-multiaddr

9482582

fix logging

eddb65c

aarshkshah1992 mentioned this pull request Feb 4, 2021

Other signaling strategies libp2p/go-libp2p-webrtc-direct#14

Open

vyzo commented Feb 17, 2021

View reviewed changes

aarshkshah1992 mentioned this pull request Feb 19, 2021

Signalling for Hole Punching #1057

Closed

2 tasks

aarshkshah1992 closed this Feb 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Hole Punching Protocol #711

Hole Punching Protocol #711

vyzo commented Aug 26, 2019 •

edited by aarshkshah1992

Loading

bigs Aug 26, 2019

aarshkshah1992 commented Jan 12, 2021

aarshkshah1992 commented Jan 21, 2021

aarshkshah1992 Jan 21, 2021

requilence commented Jan 26, 2021 •

edited

Loading

aarshkshah1992 commented Jan 26, 2021

aarshkshah1992 commented Jan 27, 2021

requilence commented Jan 28, 2021 •

edited

Loading

vyzo left a comment

vyzo Feb 17, 2021

vyzo Feb 17, 2021

vyzo Feb 17, 2021

vyzo Feb 17, 2021

vyzo Feb 17, 2021

aarshkshah1992 Feb 18, 2021

vyzo Feb 17, 2021

aarshkshah1992 Feb 18, 2021

vyzo Feb 17, 2021

aarshkshah1992 Feb 18, 2021

vyzo commented Feb 17, 2021

vyzo commented Feb 17, 2021

aarshkshah1992 commented Feb 19, 2021

Hole Punching Protocol #711

Hole Punching Protocol #711

Conversation

vyzo commented Aug 26, 2019 • edited by aarshkshah1992 Loading

TODO

Choose a reason for hiding this comment

aarshkshah1992 commented Jan 12, 2021

aarshkshah1992 commented Jan 21, 2021

Choose a reason for hiding this comment

requilence commented Jan 26, 2021 • edited Loading

aarshkshah1992 commented Jan 26, 2021

aarshkshah1992 commented Jan 27, 2021

requilence commented Jan 28, 2021 • edited Loading

vyzo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vyzo commented Feb 17, 2021

vyzo commented Feb 17, 2021

aarshkshah1992 commented Feb 19, 2021

vyzo commented Aug 26, 2019 •

edited by aarshkshah1992

Loading

requilence commented Jan 26, 2021 •

edited

Loading

requilence commented Jan 28, 2021 •

edited

Loading