Make NAT traversal more user-friendly

[JoshOrndorff]

During office hours today, we (re) realized that getting nodes to connect from behind routers is a bit of a pain for a user who is just looking to join a network.

One option is to use upnp. I had a good experience with that at rchain (their upnp code)

@folsen mentioned there are other solutions too.

I don't have strong feelings about which solution we use, or that it has to happen fast, but it would be one fewer hurdle for beginner and intermediate users.

[tomaka]

What do you mean by "is a bit of a pain"? It should just work, unless your bootstrap nodes are behind routers.

[bkchr]

I think that was his problem, he probably wanted to create a network with his office hour participants. And all of them are probably behind some NAT.

[JoshOrndorff]

he probably wanted to create a network with his office hour participants. And all of them are probably behind some NAT.

Yes, that is exactly what I meant.

[Demi-Marie]

uPnP is great, but it is not always enabled, especially in corporate environments. Being able to use UDP hole punching, and falling back to using a public (Parity-provided?) STUN or TURN server, would be a much better option.

[tomaka]

To give some context:

• We have looked into UPnP and PCP a long time ago, but we had some API issues that have now been solved. However nobody has taken the task to write an implementation again.
• Libp2p has its equivalents to STUN and TURN, called identify and relay. This makes it possible to use other libp2p nodes as STUN and TURN servers, instead of a centralized server. Rust-libp2p only supports the former. Adding the latter is a work-in-progress.
• Connections are made over TCP at the moment, so we can't use UDP hole punching. There's a work in progress spec for TCP hole punching in libp2p, so that similarly to STUN/TURN we can use libp2p nodes themselves instead of centralized servers: relay/DCUtR: Add Direct Connection Upgrade through Relay protocol libp2p/specs#173

[Demi-Marie]

To give some context:

• We have looked into UPnP and PCP a long time ago, but we had some API issues that have now been solved. However nobody has taken the task to write an implementation again.
• Libp2p has its equivalents to STUN and TURN, called identify and relay. This makes it possible to use other libp2p nodes as STUN and TURN servers, instead of a centralized server. Rust-libp2p only supports the former. Adding the latter is a work-in-progress.
• Connections are made over TCP at the moment, so we can't use UDP hole punching. There's a work in progress spec for TCP hole punching in libp2p, so that similarly to STUN/TURN we can use libp2p nodes themselves instead of centralized servers: libp2p/specs#173

How difficult would it be to switch to QUIC? That runs over UDP and just seems to be overall better.

[tomaka]

How difficult would it be to switch to QUIC?

Attempts have been made.

The difficulty is that at the moment QUIC kind of assumes TLSv3, but we don't want to depend on certificate authorities.
The libp2p specs contain an alternative to TLSv3 using self-signed certificates that are signed with the network key, but implementing that is not a trivial task.

[Demi-Marie]

How difficult would it be to switch to QUIC?

Attempts have been made.

The difficulty is that at the moment QUIC kind of assumes TLSv3, but we don't want to depend on certificate authorities.

Can we transmit something else in place of the certificates? My understanding is that TLS does not imply X.509, so we should be able to transmit anything we want in place of the certs.

[tomaka]

QUIC doesn't imply TLS either.
It's not a design issue, just an engineering issue. The current Rust QUIC and TLS libraries haven't been written to be used for anything else than regular QUIC and TLS.

[JoshOrndorff]

FWIW, It's 5 years later, and I still can't easily start a blockchain network at a meetup or an online workshop because of this issue. I always need a bootnode on some centralized aws-like thing for the users to connect too.

It isn't that hard to set up now that I have experience, but it really undermines the whole "decentralized" point.

[bkchr]

Yes :) I mean it is a nice to have and I get where you are coming from. If we follow this route, we could speak about discovering nodes via bluetooth, mesh networks and whatever. However, Substrate is currently not build with this in mind.

For local area networks we have MDNS that works quite okayish.

[aohan237]

Yes :) I mean it is a nice to have and I get where you are coming from. If we follow this route, we could speak about discovering nodes via bluetooth, mesh networks and whatever. However, Substrate is currently not build with this in mind.

For local area networks we have MDNS that works quite okayish.

i think every node is equal to find and connect is the "decentralized" core, otherwise the whole web3 is built on a lie