w3f grant application: nat traversal

[dvc94ch]

I'll be submitting a w3f grant application for adding nat traversal support to rust-libp2p. I understand that parity is busy with other priorities and that we'll have to do it ourselves. I wanted to get feedback on the proposal before submitting it and see if there is anyone on the rust-libp2p team that can help with reviewing PR's to get these features implemented.

1. Transports are assumed by libp2p to have distinct listening and dialing ports. This is an issue when trying to add the quic transport or when using tcp ports with SO_PORTREUSE.

To mitigate this one solution would be to have some notion of compatibility between multiaddr's, and listening on a multiaddr is only allowed if it's incompatible with other listening addresses. This means that when dialing there is a single compatible listening address that can be used for dialing or none, in which case a transient outgoing port is used.

proposed change to the api would be adding a listener trait that has a dial method. listen_on returns a listener that can be used for dialing. the default dial method of the transport is kept for when no compatible listener was found to dial on or for cases where you want to explicitly dial using a transient port, in a dial back protocol for example.

2. Implement the libp2p relay protocol.

This is more or less self descriptive. There is a branch by @tomaka that made a start, but even after updating to the latest libp2p it seems to be not functional, there are also no tests and no examples. One issue mentioned by @tomaka is the relay having no way of injecting relayed connections into the swarm.

[DemiMarie]

Some (non-libp2p-specific) notes:

• The only reliable method of NAT traversal is a relay server. The unreliability of alternatives cannot be overemphasized. Protocols like STUN that avoid needing to relay all traffic are fantastic, but they are purely optimizations and must not be relied upon. For instance, they won’t work for devices that are tethered to an Android phone, or behind a Linux conntrack-based stateful firewall.
• Because of stateful firewalls, a relay server might be needed even when no NAT is in use. This makes automatic NAT detection unreliable. Users MUST be able to force NAT traversal even when autodetection indicates it is not necessary.
• I strongly suggest using coturn and the TURN protocol for relaying. It is fast, widely deployed, and simple. Relay servers need to handle massive amounts of data and connections, so the data path must be as simple as possible. For example, the relay server should not need to do any encryption or decryption for data connections.
• While a relay server can be integrated into a parent application, this should be a convenience feature. A good relay protocol is so simple that techniques like high-performance network stacks like AF_XDP, netmap, DPDK, and Snabb can achieve speedups of over an order of magnitude. 10Gb/s/core on a modern x86_64 server is not at all an unreasonable expectation if the NIC is fast enough, but you won’t get anywhere close with standard network stacks. These techniques are invariably non-portable and require elevated privileges, so relay servers should generally be dedicated bare-metal servers.
• A default relay server MUST be provided. While others SHOULD be encouraged to provide their own relay servers, this is a significant administrative overhead, especially if one wants good performance (see above). Furthermore, cloud providers like AWS or GCE are not good places for relay servers, as data egress costs will be crushing at large scale.
• Relay servers are not highly security-sensitive. They should not have access to any plaintext sensitive data.

[DemiMarie]

To elaborate on what I wrote above: The data plane of a UDP relay server is really more like networking equipment than it is an ordinary server. It must carry very large amounts of data, but the processing it performs is probably less complex than that of an IPv6 router. As such, it needs tight integration with the network stack to get good performance. The control plane is far more complex, but it doesn’t need to be anywhere near as fast. TCP and (especially) TLS over TCP are significantly more complex, and should probably be avoided whenever possible.

[dvc94ch]

Makes sense, but I'm not sure how that fits with libp2p, as coturn uses tls and doesn't have a concept of peerid's. Maybe it makes sense to start with a simple relay in libp2p and then add an external data plane later that isn't managed by libp2p as a performance improvement? I guess what you are referring to as the control plane would remain the same.

On detecting if nat traversal using a third party that has a public IP dialing back and reporting if it failed, I don't think it's relevant if the failure is really because of nat or a firewall.

[DemiMarie]

I am suggesting that we piggyback on an existing relay protocol (TURN) for data. libp2p will still need to handle IP address discovery itself, but it won’t need to handle the full relay data flow. Of course, it would be awesome if libp2p could handle the data, but I suspect that low-level networking that requires root privileges is out of scope for libp2p.

That said, libp2p will need to be involved in the control plane. TURN requires authentication, and with good reason: if it did not, it would be trivial to abuse a TURN server to hide the source of a DDoS attack. However, TURN does not have any way to confirm that a peer wants to receive the traffic that will be sent to it. Clients must not be given credentials that allow them to send data to X until X has confirmed that it wishes to receive that data. coturn exposes a REST API for management, which libp2p can use to control it.

[dvc94ch]

TURN requires authentication, and with good reason: if it did not, it would be trivial to abuse a TURN server to hide the source of a DDoS attack.

Can you please elaborate? I don't know if this is a big deal as long as there is no amplification. If it's a public relay, having to sign up to create an account probably won't be a large slowdown [0].

• [0] https://cloudsek.com/how-to-bypass-captchas-easily-using-python-and-other-methods/

[DemiMarie]

TURN requires authentication, and with good reason: if it did not, it would be trivial to abuse a TURN server to hide the source of a DDoS attack.

Can you please elaborate? I don't know if this is a big deal as long as there is no amplification. If it's a public relay, having to sign up to create an account probably won't be a large slowdown 0.

My understanding is that a VoIP server, such as Synapse, typically issues authentication tokens to its clients. The clients then use the tokens to authenticate to the TURN server. I could be wrong, though.

[DemiMarie]

So I looked at TURN and it appears that it can be used to send packets to any IP address. A better relay protocol would be at least as simple as TURN, but would require that an IP address ask to receive packets before anyone is allowed to transmit packets to them.

[dvc94ch]

Closing in favor of #1134