Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should have NAT Traversal #57

Closed
whyrusleeping opened this issue Sep 12, 2014 · 28 comments
Closed

Should have NAT Traversal #57

whyrusleeping opened this issue Sep 12, 2014 · 28 comments
Labels
help wanted Seeking public contribution on this issue kind/enhancement A net-new feature or improvement to an existing feature topic/nat Topic nat

Comments

@whyrusleeping
Copy link
Member

Peers behind nats need a way for other peers in the network to connect to them. It would be nice to also have webrtc, but until a go library for that exists, this will have to do.

@whyrusleeping whyrusleeping added kind/enhancement A net-new feature or improvement to an existing feature help wanted Seeking public contribution on this issue labels Sep 12, 2014
@jbenet jbenet changed the title STUN/TURN for peer connectivity Should have NAT Traversal Nov 6, 2014
@jbenet
Copy link
Member

jbenet commented Nov 6, 2014

The good news is I found a very promising approach for nat traversal, which takes its ICE implementation directly from the chromium src (like i want to do in jbenet/random-ideas#13). It's https://github.com/getlantern/natty with go bindings at https://github.com/getlantern/go-natty.

I can get OSX processes to connect to each other using their https://github.com/getlantern/waddell signaling server, and the https://github.com/getlantern/nattywad client, using a very crappy test script: https://gist.github.com/jbenet/632d9459414b996e9457 i wrote (but when I went to test linux I got getlantern/nattywad#1).

Pros:

  • same ICE as in chromium
  • signaling server in Go, so we can embed it into our nodes
  • of all the approaches I've seen so far, this is the most promising long term.

Cons

  • seems to be only udp? (some nats stop UDP, so can always try: RFC: Fake TCP with UDP jbenet/random-ideas#2)
  • embeds a binary (natty) bigger than all of built ipfs. not a problem now, but will make implementations for embedded devices harder.

@jbenet
Copy link
Member

jbenet commented Nov 6, 2014

Got farther with natty: https://github.com/jbenet/go-netcatnat/

@jbenet
Copy link
Member

jbenet commented Nov 8, 2014

Ok the roadmap for NAT traversal is:

1. For now

  1. add a signaling Service to ipfs nodes, to do what waddell does.
  2. use the bootstrap nodes as the only signalers (fine for now)
  3. (maybe) if a node discovers it is behind a nat, it can report its address as requiring nat traversal[1]

[1] this is a convoluted example, and there's probably simpler ways, but maybe:

<multiaddr of signaler used>/ipfsstun/<peer.ID>/ipfs/<peer.ID>`
// or something nicer than ipfsstun
// the peer.ID is repeated because encapsulation. ipfsstun could use arbitrary identifiers

This says "connect to <multiaddr of signaler used> and ask for connection to /ipfsstun/<peer.ID>, it will give you addresses, attempt to connect to those. behind them is an /ipfs/<peer.ID>.

2. Down the road

  1. signaling integrated with routing
  2. e.g. in kademlia, nodes in each others' routing table have open connections + can help signal.
  3. in better routing systems, should be able to deliver a message through the network even if node not directly accessible, so can do signaling that way.
  4. also look into pwnat

@jbenet
Copy link
Member

jbenet commented Nov 8, 2014

I reiterate here that we cannot rely on others' systems for nat traversal. It's an important enough piece and should not rely on public stun servers, as our traffic needs to not depend on pieces of infrastructure that might {fail, rate limit, block ipfs} unexpectedly. ipfs must sustain itself.

@whyrusleeping
Copy link
Member Author

Where do you put UDP on our priority list?

@jbenet
Copy link
Member

jbenet commented Nov 8, 2014

@whyrusleeping ah right, natty is currently for udp only, so i guess we're switching now. We can find the simplest reliability protocol on top of udp, and aim for uTP or SCTP, and then QUIC once its implementations settle.

@jbenet
Copy link
Member

jbenet commented Nov 11, 2014

Update: now that we have UDT (#58), I can move ahead on this. My next thing will be adding Natty and UDT. these are two C++ libs, that will add significant size (and noise) to the binary. These are stopgaps to get the right functionality, in the end native Go impls, or Cgo compiled things will probably be less problematic to use.

@whyrusleeping
Copy link
Member Author

this makes me kinda sad, since now we cant just distribute binaries...

@jbenet
Copy link
Member

jbenet commented Nov 11, 2014

@whyrusleeping oh we can. cross compilation :)

@whyrusleeping
Copy link
Member Author

ehhh, not quite what i was hoping for though, with pure go, you can build once for each system and go. Once you mix in cgo and other linking stuff, you have to worry more about the specific system and their version of libc

@jbenet
Copy link
Member

jbenet commented Nov 11, 2014

@whyrusleeping yeah, it's painful. Pure Go impl of {UTP, UDT, SCTP, and QUIC} in the horizon. in due time. For now, I much prefer my ipfs node able to talk yours, (almost) no matter where it is.

@whyrusleeping
Copy link
Member Author

Oh yeah, im much more happy about UDP and NAT working than i am sad about not being pure Go

@jbenet jbenet added this to the NAT Traversal milestone Nov 17, 2014
@jbenet
Copy link
Member

jbenet commented Jan 15, 2015

Alpha: NAT or Relay

NAT traversal is non trivial, the roadmap looks like this:

  • relay service for ICE comm -- our p2p/protocols/relay
  • allow ipfs nodes to find suitable relays -- i.e. nodes directly connected to the node we wish to reach (implemented with (*IpfsDht).FindPeersConnectedToPeer)
  • use something to perform ICE (current candidate: natty, which gives us only UDPAddr pairs)
  • use reliable transport over UDP.

The blocker is reliable transport. Our options so far have been:

  • utp - needs to be implemented robustly. h2so5/utp is not yet working, could go-wrap libutp.
  • udt - go-wrapped, needs some ARM assembly
  • sctp - needs go-wrapping, should not have assembly.
  • quic - needs lib-izing, and go-wrapping.

Of these, quic is my favorite option for the future. It is a truly phenmenal transport, with tons of things done very, very well. However, though the dragon is ready to hatch from its egg, it must still be unearthed from the enormous Code Caverns, of the Chromium Mountains, a quest not to be taken lightly.

Reliable transport a really big deal for us, and hard to get right:

  1. Buggy transports will cause us endless pain and suffering. They already have: I spent many hours debugging dht and bitswap problems only to find that utp was the problem all along.
  2. "Congestion Control" and "Flow Control" are both very hard. It took the TCP, SCTP, UTP, and QUIC teams many months to perfect designs and implementations.
  3. Low {memory, cpu} footprints, etc are hard to achieve well, while also achieving low bandwidth and latency overheads.

@whyrusleeping pre-empting your call for "Lets Implement UTP!", we should not roll our own right now-- it will take us weeks to get reasonable robustness, speed, flow control, memory consumption, etc. We are good hackers, but it's taken the broader community years to make the few implementations there are. this is not to be underestimated. I totally want to have native go implementations of these transports (see my related emails to go-nuts mailing list), but we cannot afford the implementation cost right now.

another option TCP/UDP tunnel

And, frankly, I'm seriously considering doing the following horrible hack that might actually work robustly and be much quicker to pull off that

  • use the UDPAddr pairs from natty, BUT!
  • setup a TCP/UDP tunnel with our own NAT:
    A-TCPConn <---> IPConn <---> UDPConn <--IPConn--> TCPConn-B

This is a mad science(tm) hack that gets around the awful realities of the network and the awful realities of software: simple things outside of specific rails is too fucking hard to do.

Looking at http://golang.org/pkg/net/#IPConn -- it should be possible, my concern is making sure DialIP doesn't interfere with other traffic:

Resources:

Relay past NAT -- easy cop-out for now

The easiest thing to do for now is to just relay all traffic. Would get us around NAT, but would certainly increase load of nodes.

We'll have to relay some traffic, as not all NAT is hole punching. Sometimes we do have to use relaying. This could actually be done with Bitswap in mind, and be governed by the standard bitswap decision engine: relay for those nodes who are useful to us.

In lieu of using natty and hole-punching, we could start with the big guns (which will work for everything and we have to do anyway) and optimize later. My Knuth VM is telling me to optimize later.

@whyrusleeping
Copy link
Member Author

I think a combination of relays, and NAT hole punching (NAT-PMP, uPnP) could be our best option (at least short term). I found a PMP library: https://github.com/jackpal/go-nat-pmp, and there was a few promising uPnP libraries around, which should get us through a decent number of networks, and the rest can be relayed.

@btc
Copy link
Contributor

btc commented Jan 15, 2015

we could start with the big guns

the big guns being relay?

@jbenet
Copy link
Member

jbenet commented Jan 16, 2015

the big guns being relay?

yeah

@jbenet jbenet added the topic/nat Topic nat label Mar 28, 2015
@jbenet jbenet removed this from the NAT Traversal milestone Mar 29, 2015
@tv42
Copy link
Contributor

tv42 commented Apr 5, 2015

Since I don't think I've seen this mentioned yet, and it's good overview:

https://tools.ietf.org/html/rfc5128
State of Peer-to-Peer (P2P) Communication across Network Address Translators (NATs)

@drwasho
Copy link

drwasho commented Nov 17, 2015

Any update on where you're at on this issue?

@whyrusleeping
Copy link
Member Author

closing this for now, basic NAT traversal has been implemented as well as utp in dev0.4.0. Further discussion on relays and more advanced traversal should start a new issue

@jbenet jbenet removed the icebox label Jan 1, 2016
@drwasho
Copy link

drwasho commented Jan 4, 2016

Awesome, well done guys!

@myleshorton
Copy link

Hey @whyrusleeping and @jbenet can you describe the NAT traversal implementation in 0.4.0 or link to more details? Exciting stuff.

@makew0rld
Copy link
Contributor

@myleshorton @whyrusleeping @jbenet Yes, I am interested in how it works for the purposes of my own decentralized application. My application would require IPFS anyway, and I am wondering if I can just piggyback on the NAT traversal that IPFS uses, and then just specify a different a different port?

@makew0rld
Copy link
Contributor

I just came back to this link again. Does anyone have an answer?

@lidel
Copy link
Member

lidel commented Apr 19, 2018

@Cole128 check https://libp2p.io, the networking layer of IPFS that, among other things, provides solutions for NAT Traversal.
If you need support, ask at http://discuss.ipfs.io (it has bigger reach than a github issue)

@makew0rld
Copy link
Contributor

@lidel libp2p led me here which seems to say it is only kind of implemented now.

@Stebalien
Copy link
Member

@Cole128 there are two implementations of libp2p, one in js and one in go (and a work in progress one in rust). The issue you linked was for js. NAT traversal works pretty well in go (although we're still missing a few pieces).

If you just want to piggyback on ipfs's NAT traversal, you may be interested in https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#ipfs-p2p

There's some improved documentation in #4894 but it's blocked on actually making this feature behave according to that documentation... (i.e., how one would expect).

If you're still having trouble with NAT traversal, you may also be interested in the circuit relay feature: https://github.com/ipfs/go-ipfs/blob/master/docs/experimental-features.md#circuit-relay

@makew0rld
Copy link
Contributor

@Stebalien thanks a lot, I'll check all those links out.

@aschmahmann aschmahmann mentioned this issue Sep 22, 2020
72 tasks
@aschmahmann aschmahmann mentioned this issue Dec 1, 2021
80 tasks
laurentsenta pushed a commit to laurentsenta/kubo that referenced this issue Feb 25, 2022
use key size constraints defined in libp2p
laurentsenta pushed a commit to laurentsenta/kubo that referenced this issue Feb 25, 2022
…size

use key size constraints defined in libp2p
laurentsenta pushed a commit to laurentsenta/kubo that referenced this issue Mar 4, 2022
…size

use key size constraints defined in libp2p
laurentsenta pushed a commit to laurentsenta/kubo that referenced this issue Mar 4, 2022
…size

use key size constraints defined in libp2p
@ajnavarro ajnavarro mentioned this issue Aug 24, 2022
72 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Seeking public contribution on this issue kind/enhancement A net-new feature or improvement to an existing feature topic/nat Topic nat
Projects
None yet
Development

No branches or pull requests

10 participants