feat: use BytesMut for Transmit content

[dignifiedquire]

This is a potential solution for my current problem. The issue I am running into, is that I have to split up the Transmits that are sent out, and send them over a channel. but because I only get access to a &[Transmit], I have to copy the content buffer.

The solution here switches the Vec<u8> simply for BytesMut, which I can then easily send over a channel (and store for potential delayed submission) without copying.

One additional change that for convenience I would add, is to derive Clone for Transmit, as now this is an actual cheap operation.

[dignifiedquire]

error: package regex v1.8.1 cannot be built because it requires rustc 1.60.0 or newer, while the currently active rustc version is 1.59.0

CI failures seem unrelated

[djc]

Would you mind adding a commit that bumps the MSRV to 1.60?

[dignifiedquire]

Would you mind adding a commit that bumps the MSRV to 1.60?

done

[dignifiedquire]

error: package time v0.3.20 cannot be built because it requires rustc 1.63.0 or newer, while the currently active rustc version is 1.60.0

@djc looks some more dependencies are increasing MSRV

[Ralith]

I have to split up the Transmits that are sent out

Can you elaborate on this requirement a bit? quinn-proto is intended to prepare bytes optimally for consumption by the underlying UDP stack.

One additional change that for convenience I would add, is to derive Clone for Transmit, as now this is an actual cheap operation.

BytesMut is specifically unique, so I don't think this change makes cloning any cheaper.

[dignifiedquire]

BytesMut is specifically unique, so I don't think this change makes cloning any cheaper.

oh no, you are right, that allocates, okay I guess I will have to change this to store Bytes then to actually solve my issue.

[dignifiedquire]

Can you elaborate on this requirement a bit?

I have three different types of connections in the background, 1 UDP socket for ip4, 1UDP socket for ip6 and a TCP based relay. Packets are sent dynamically to one of these three, depending on the best available route.

[Ralith]

okay I guess I will have to change this to store Bytes then

I thought you only needed to split them up, not duplicate them?

Packets are sent dynamically to one of these three, depending on the best available route.

Why does this require splitting anything?

[dignifiedquire]

okay I guess I will have to change this to store Bytes then

I thought you only needed to split them up, not duplicate them?

Packets are sent dynamically to one of these three, depending on the best available route.

Why does this require splitting anything?

Sorry I wasn't clear enough. By "Splitting up" I meant that I would need to potentially do different things with each Transmit.

The current operations I have to do concretely are

• Send Transmits over a channel.
• Change the destination address of a Transmit.
• Group Transmits from a single call

[Ralith]

None of those operations obviously involve splitting or cloning the payload. Can you elaborate?

[dignifiedquire]

The signature of poll_send only gives access to &[Transmit], which results in

• Send on a channel: requires 'static lifetime or owned, so I end up needing to clone
• updating the destination: this is not mutable reference, so in order to do so, I need to clone each element and change the destination field
• grouping: This works out if I have consecutive groups, but if they are not I again have to clone, as I otherwise would end up with &[&Transmit]s instead of &[Transmit], which I can not pass to poll_send of the underlying UdpSocket from quinn_udp.

[Ralith]

Ah, I had assumed you were using quinn-proto, not implementing AsyncUdpSocket; this makes more sense then, thanks for clarifying!

[Ralith]

MSRV update is in main now, so a rebase should clean up CI.

[dignifiedquire]

@Ralith any other thoughts on this? would love to get this in

[Ralith]

The implementation here looks good. I'm hesitating because I can't help but wonder if your use case would be better addressed by maintaining separate connections and switching them at the application layer, in which case we could avoid this (admittedly slight) increase in API complexity. Running QUIC over TCP in particular is going to be rather inefficient, and it would be perfectly reasonable to have parallel QUIC connections over IPv4/IPv6; this is basically happy eyeballs.

[dignifiedquire]

I can't help but wonder if your use case would be better addressed by maintaining separate connections and switching them at the application layer,

I have been wondering that as well, but the biggest obstacle to that that I see currently is that I need to be able to do parallel sends of packets, to reduce latency in connection establishment. Basically I am using the relay transport for the first few packets, while hole punching is running, and switch over as soon as I can, and if I am unsure if my udp addresses are any good I double send packets over both the relay and the udp until I get some kind of confirmation. I don't quite see how I can achieve the same robustness with switching connections at the application layer.

[Ralith]

I'm guessing "the relay" here means the TCP connection?

It sounds like you have two separate concerns: latency and robustness. Robustness is easy to achieve with application-layer switching: always send data only on a connection that's been successfully established, sorting by priority (presumably favoring QUIC once available).

It sounds like you're also invested in avoiding a superfluous round-trip between hole punching success and QUIC connection establishment. Could you use the actual initial QUIC handshake packet for the hole punching, so that success implies connection establishment?

[dignifiedquire]

That might be possible, but currently we are following an existing design that has proven to work for a similar use case (tailscale to be specific). So while we might be able to optimize things to that at later stage, we will need to make it work like this for the moment.

[Ralith]

Fair enough. I don't see really see any drawbacks to Bytes over Vec given that Bytes is already in our API, so we might as well support that.

[djc]

@dignifiedquire can you give this a rebase? Should be fairly straightforward.

[dignifiedquire]

done, should be fixed

[dignifiedquire]

okay, MSRV is upset again :( you want me to bump to 1.65?

[djc]

I'm working on a fix for the MSRV issue in #1558.

[djc]

Going to merge this, since the MSRV issue will be fixed separately (and FreeBSD CI passes).