Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracking Issue] transports/quic: Add QUIC Transport #2883

Closed
13 of 15 tasks
elenaf9 opened this issue Sep 9, 2022 · 23 comments
Closed
13 of 15 tasks

[Tracking Issue] transports/quic: Add QUIC Transport #2883

elenaf9 opened this issue Sep 9, 2022 · 23 comments

Comments

@elenaf9
Copy link
Contributor

elenaf9 commented Sep 9, 2022

Summary

Implement the QUIC transport for rust-libp2p.

Done criteria

  • Quic can be used as a transport in the Swarm for communication between nodes that can directly reach each other
  • TLS handshake is compliant with the libp2p/specs TLS Handshake and unit tested
  • Transport is unit & integration tested
  • Benchmarked with mxinden/libp2p-perf
  • Tested again go-libp2p with Testground
  • Public API is documented

Status Quo

Alpha implementation based on quinn crate. See #3454.

Previous attempts / History

Tasks

Long-term (not required for first implementation):

@mxinden
Copy link
Member

mxinden commented Sep 11, 2022

Thanks for creating this @elenaf9!

Whenever you make a significant change to the original issue description, would you mind posting a comment like #2052 (comment)? That way, everyone subscribed to this issue gets a notification.

@elenaf9
Copy link
Contributor Author

elenaf9 commented Sep 16, 2022

Changelog

Handle address change if a connection migrates

  • Moved to Long-term. For now we can simply disable connection migration.

@elenaf9
Copy link
Contributor Author

elenaf9 commented Sep 22, 2022

Changelog
QuicMuxer refactoring (elenaf9#6) was replaced and merged into #2289 with kpp#23:

  • Updated the status-quo description
  • Marked corresponding task as done.

@elenaf9
Copy link
Contributor Author

elenaf9 commented Sep 27, 2022

Changelog

Evaluate whether we should switch to use UdpSocket from quinn_udp instead of the rust std implementation

  • For the first implementation we will stick with UdpSocket from rust std. Integrating quinn_udp::UdpSocket requires additional complexity, and we don't strictly need it, especially if we plan on long-term switching to quinn.

Handle / use ECN bits

  • Moved to Long-term since the UdpSocket from std does not support this.

Fix local_address when processing an UDP datagram for a new incoming connection on a socket that is bound to a wildcard address

  • Moved to Long-term. To fix this, we need to obtain the destination-ip of the first udp packet on the connection, which we can not access through the normal UdpSocket API. quinn_udp supports it for linux only. Long-term we can contribute support for other platforms as well (related issue: Provide local and remote socket addresses from a Connecting quinn-rs/quinn#508). For now the local_address will have an unspecified IP in this case.

@mxinden mxinden mentioned this issue Oct 5, 2022
@mxinden mxinden changed the title [Tracking Issue] QUIC Transport [Tracking Issue] transports/quic: Add QUIC Transport Oct 6, 2022
@marten-seemann
Copy link

Should Stateless Reset handling / generation be added to the Long Term list?

@elenaf9
Copy link
Contributor Author

elenaf9 commented Oct 23, 2022

Changelog:

@thomaseizinger
Copy link
Contributor

Changelog:

Added link to kpp#27 (comment) so we don't forget to refactor the TLS-config handling.

@elenaf9
Copy link
Contributor Author

elenaf9 commented Nov 14, 2022

Changelog:

@geotro
Copy link

geotro commented Nov 17, 2022

I hope this is the right place to ask.

Is quic supported on relay-v2, and if not, what are the future plans regarding this?

@elenaf9
Copy link
Contributor Author

elenaf9 commented Nov 17, 2022

@geotro yes it is supported. You can combine them in the same manner as it is done here with tcp and quic:

let quic_transport = quic::async_std::Transport::new(config);
let tcp_transport = tcp::async_io::Transport::new(tcp::Config::default())
.upgrade(upgrade::Version::V1)
.authenticate(
noise::NoiseConfig::xx(
noise::Keypair::<noise::X25519Spec>::new()
.into_authentic(&keypair)
.unwrap(),
)
.into_authenticated(),
)
.multiplex(yamux::YamuxConfig::default());
let transport = OrTransport::new(quic_transport, tcp_transport)
.map(|either_output, _| match either_output {
EitherOutput::First((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)),
EitherOutput::Second((peer_id, muxer)) => (peer_id, StreamMuxerBox::new(muxer)),
})
.boxed();

and either replace the tcp_transport with your relay transport, or if you'd like to support tcp as well you have to combine (OrTransport) tcp and relay and then do the upgrade.

Does this help?

@elenaf9
Copy link
Contributor Author

elenaf9 commented Nov 17, 2022

Changelog

@geotro
Copy link

geotro commented Nov 19, 2022

@elenaf9 it helps a bit, but I was hoping that once QUIC is implemented to be able to do UDP, as well as DCUTR.

Why are you referring to TCP in the context of QUIC?

I understand DCUTR with QUIC is not yet supported so I'll have to wait until that works before I retry it.

@elenaf9
Copy link
Contributor Author

elenaf9 commented Nov 21, 2022

Why are you referring to TCP in the context of QUIC?

I was referring to TCP to point out how you could support both TCP and QUIC together with the relay-v2 protocol. But TCP is not required, you may as well just combine QUIC and relay-v2.

@geotro
Copy link

geotro commented Nov 21, 2022

Ah OK, thank you for clarifying this!

My use case requires DCUTR so I'm going to have to wait until this is available together with QUIC before I can begin testing/using it. Very much looking forward to this as I hope it's going to greatly improve the success rates of establishing a direct connection!

dariusc93 added a commit to dariusc93/rust-ipfs that referenced this issue Nov 25, 2022
Notes:
- This is only compatible with v1 so likelihood of connecting to go-libp2p and go-ipfs/kubo peers via quic will unlikely to work until draft-29 is supported in rust-libp2p (see libp2p/rust-libp2p#3133)
- DCuTR will not work with quic-v1 at this time so unless port mapping/UPNP is used (coming later), we wont be able to utilize quic-v1 with hole punching, but should work fine with relay (see libp2p/rust-libp2p#2883)

Closes #10
@Frando
Copy link

Frando commented Dec 19, 2022

Is there an issue or note somewhere what is missing for a working setup with QUIC + DCUTR + relay v2? Looking what pieces are missing there and would love any pointers.

@mxinden
Copy link
Member

mxinden commented Dec 29, 2022

Is there an issue or note somewhere what is missing for a working setup with QUIC + DCUTR + relay v2?

As far as I remember, the only thing missing is the implementation of <quic::GenTransport as Transport>::dial_as_listener.

fn dial_as_listener(
&mut self,
addr: Multiaddr,
) -> Result<Self::Dial, TransportError<Self::Error>> {
// TODO: As the listener of a QUIC hole punch, we need to send a random UDP packet to the
// `addr`. See DCUtR specification below.
//
// https://github.com/libp2p/specs/blob/master/relay/DCUtR.md#the-protocol
Err(TransportError::MultiaddrNotSupported(addr))
}

Note that https://github.com/libp2p/punchr/blob/main/rust-client/ is already using QUIC for hole punching, though only on the Transport::dial and not on the Transport::dial_as_listener side.

@Frando contributions here are welcome.

@mxinden
Copy link
Member

mxinden commented May 15, 2023

Posting some in-progress research on the various Rust QUIC implementations out there.

Verdict

Thus far my preference is still quinn via #3454.

With #3454, quinn is no longer exposed on the public API surface. Thus we can easily switch out the underlying QUIC implementation in the future in case we ever want to move off of quinn.

Next step: Report our performance measurements to quinn maintainers to push quinn-rs/quinn#1219 forward.

Comparison

  integration effort comments WebTransport Pacing Managed UDP socket GSO GRO random bytes for hole-punching TLS license
quinn low, see PR 3454 Uses unbounded channels Work-in-progress Pacing implementation Yes, through quinn crate Added with GSO GRO Yes. One can gain access to the socket via Endpoint::new rustls and ring Apache and MIT
quiche unknown Large cooperation focused on performance (Cloudflare) Work-in-progress out of tree community owned Pacing implementation No, see example on manual management Doesn't manage socket, thus only example Yes, since UDP socket isn't managed boring ssl BSD 2-Clause
s2n unknown Large cooperation focused on performance (AWS) \n use unbounded channel to accept new connections. Needs to be investigated whether that is a problem. Can't find any yes Yes, see client and server yes Not sure how to access the socket on a `Server`. s2n-tls Apache
msquic unknown API for Rust, published on crates.io              
neqo unknown not published on crates.io              

//CC @marten-seemann

@marten-seemann
Copy link

Thanks for compiling this list @mxinden! This looks pretty comprehensive.

Some thoughts:

  • Not sure if GSO and pacing warrant their own column, they’re just part of a long list of things a QUIC implementation can (optionally) do to speed things up. Others would be DPLPMTUD and ECN. They might serve as a proxy variable to see how advanced a QUIC stack is though.
  • I expect WebTransport to be supported by more implementations once the draft makes more progress at the IETF. Not a lot of implementations are willing to pay the cost of shooting at a moving target, unless they have a very clear use case for the protocol (as quic-go obviously has, and Chrome’s implementation as well). This is even more so true after the adoption of the RELIABLE_RESET_STREAM draft at the last IETF meeting.
  • Choosing Quinn (for now) doesn’t seem like a terrible choice, especially since the work is mostly driven by the community. Making the package API agnostic to the underlying QUIC implementation seems like a wise choice, enabling more experimentation in the future.

@kpp
Copy link
Contributor

kpp commented May 22, 2023

Column random bytes for hole-punching
quinn - Yes. One can gain access to the socket via Endpoint::new

The way it is implemented in #3964 is through channels sending quinn_proto::Transmit. However the go impl uses multiple sockets with SO_REUSEPORT. So in quinn wrapper we can go both ways not sure which one is better.

mergify bot pushed a commit that referenced this issue Jun 13, 2023
Implement `Transport::dial_as_listener` for QUIC as specified by the [DCUtR spec](https://github.com/libp2p/specs/blob/master/relay/DCUtR.md).

To facilitate hole punching in QUIC, one side needs to send random UDP packets to establish a mapping in the routing table of the NAT device. If successful, our listener will emit a new inbound connection. This connection needs to then be sent to the dialing task. We achieve this by storing a `HashMap` of hole punch attempts indexed by the remote's `SocketAddr`. A matching incoming connection is then sent via a oneshot channel to the dialing task which continues with upgrading the connection.

Related #2883.

Pull-Request: #3964.
@mxinden
Copy link
Member

mxinden commented Jul 31, 2023

Status Update

#3454 merged and thus our libp2p-quic implementation depends on upstream quinn directly. Thank you @kpp for the work. In case no major blockers are discovered in libp2p-quic v0.9.0-alpha we can release it as v0.9.0 soonish.

@thomaseizinger
Copy link
Contributor

thomaseizinger commented Jul 31, 2023

Status Update

#3454 merged and thus our libp2p-quic implementation depends on upstream quinn directly. Thank you @kpp for the work. In case no major blockers are discovered in libp2p-quic v0.9.0-alpha we can release it as v0.9.0 soonish.

It appears that the above description of this issue is rather outdated, do you mind updating it?

@mxinden
Copy link
Member

mxinden commented Aug 24, 2023

Status Update

We just released libp2p v0.52.3 which contains a stable release of libp2p-quic v0.9.2. That is the first stable libp2p-quic release in rust-libp2p and thus a big step for the project 🎉

@mxinden
Copy link
Member

mxinden commented Sep 4, 2023

I am closing here as the major milestone, adding QUIC to rust-libp2p, is achieved. Smaller follow-up improvements are tracked in individual issues.

Thank you to the many many many people involved in this effort. ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants