feat: Add QUIC Address Discovery to iroh

[ramfox]

Description

There were two main things to "solve" on the iroh side:

1. gross circular dependency between magicsock and the quinn endpoint
2. needing to handle QAD packets in a special way, like we do with STUN packets that get sent from net-report

The first was accomplished by figuring out how to make MagicSock::spawn be the location that builds the quinn::Endpoint. This is what was changed:

• implemented AsyncUdpSocket on MagicSock itself, rather than magicsock::Handle
• magicsock::Handle now contains the quinn::Endpoint
• we now pass down a server_config as a MagicSock::Option
• in MagicSock::spawn:
  • after building the MagicSock we now build the quinn::Endpoint using Arc<MagicSock> as the AsyncUdpSocket
  • then we clone the quinn::Endpoint and passed it to the magicsock::Actor (which is independent of MagicSock)
  • give the quinn::Endpoint to the magicsock::Handle, which is the actual struct that we use to interact with the magicsocket
• the iroh::Endpoint now interacts with the quinn::Endpoint using msock.endpoint() in all places

The second was accomplished by keeping a list of special "QAD addresses" on the NodeMap (NodeMap::qad_addrs), and using specific methods to add_qad_addrs, get_qad_addrs_for_send and get_qad_addrs_for_recv to deal with the fickle way that the quinn::Endpoint expects SocketAddrs to behave when it dials and when it receives packets:

• before we do a net-report, we first attempt to resolve the RelayUrls in the RelayMap to get the SocketAddrs that we expect we will dial when we do a QAD probe
• ~we add those addresses to the NodeMap ~
• on the "send" side of the AsyncUdpSocket, after we do our normal checks for QuicMappedAddrs, we then check to see if the QuicMappedAddr is actually just a normal socket addr that we expect to use for QAD. If so, we just send the packets using the get_qad_addr_for_send address
• on the "recv" side of the AsyncUdpSocket, after we check to see if the recv'd address can be mapped to a QuicMappedAddr & therefore can be received using our normal process, we then check to see if the address is one that we expect for QAD. If so, we make sure to associate the packet with the get_qad_addr_for_recv address and pass it along

The most "unreliable" bits of this are due to dns. Before running a net report, we now have to do dns discovery for all the RelayUrls. I've capped this at 300 ms but it means that until we cache the dns responses then we have this delay before starting net-report. I've also done a bit of a cheat: when we initially start the magicsock::Actor, I've added a call to resolve_qad_addrs that has a timeout of 10 ms, just to send out the dns packets and hopefully get a jump on caching the responses.

The second was accomplished by creating a new IpMappedAddrs struct that keeps track of IpMappedAddrs to the actual SocketAddr that it represents. This is akin to how we deal with QuicMappedAddr (which is now called NodeIdMappedAddr. netreport now takes an optional IpMappedAddrs, and when it resolves a RelayUrl, it adds the resolved IP addr to the IpMappedAddrs, and uses the mapped address when sending on QUIC. In iroh, we check to see if the destination or source address is an IpMappedAddr or a NodeIdMappedAddr. If so, it maps the addresses correctly before returning the transmit to the endpoint or sending the transmit over UDP.

Most interesting bit

So...if someone adds an IP address to IpMappedAddrs, we can allow folks to use the Endpoint as a sort of normal quinn::Endpoint, bypassing our special iroh holepunching sauce.

depends on #3032

Change checklist

• Self-review.
• Documentation updates following the style guide, if relevant.
• Tests if relevant.
• All breaking changes documented.

[github-actions]

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/3049/docs/iroh/

Last updated: 2025-01-17T21:00:32Z

[flub]

The second was accomplished by keeping a list of special "QAD addresses" on the NodeMap

The downside of this approach is that you need to lock the NodeMap once again on every single try_send and poll_recv call. That's not great.

What if we made a separate mapping outside of the NodeMap for this? We can make a new type IpMappedAddr (and rename QuicMappedAddr to NodeIdMappedAddr while we're at it). It would be a mirror of the current QuicMappedAddr but with a different ADDR_SUBNET.

So we use a new subnet in our existing IPv6 Unique Local Addr space. This subnet works just like the existing subnet otherwise. But is used to map real IP addresses to other fake ones.

This allows the code that needs to check whether something is a IpMappedAddr or a QuicMappedAddr (or NodeIdMappedAddr) to simply check the IPv6 prefix with no need to get any locks until it is know which one you have.

[flub]

The way of connecting the quinn::Endpoint into the MagicSock is nice!

[flub]

So my biggest question is about why IpMappedAddr maps to a SocketAddr rather than an Ipv6Addr. But then I also have that question about NodeIdMappedAddr which I don't think should be solved in this PR. And since the IpMappedAddr works as is, I'm happy to defer that as well if you'd prefer.

Otherwise this looks really good, though I guess we agreed that the IpMappedAddr would move into net-report for now.

[flub]

You need to check that the entire prefix matches, [0..8], no? Just the subnet is not enough.

With some luck you can compute the full prefix in a const fn, but I'm not sure if that's possible. You can always just write a full const PREFIX: [u8; 8] = [0xfd, 21, 7, 10, 81, 11, 0, 1]; next to the existing constants.

[ramfox]

I adjusted this. I just checked each section of the slice, similar to how we fill the slice when we generate the addrs. Can you please look at this again and let me know if it's acceptable?

[flub]

I guess I have the same comments here as on the IpMappedAddr. But I do appreciate that this already had a dummy port in it. So let's not change that. I'll see if I can remove it reasonably in a followup PR and so I'm happy to leave this as-is for now.

[ramfox]

made the change already and here was my logic for why it was okay to do:

We never in the code checked the port before, that was something I added with the try_from, so I'm comfortable taking it "away". Now we have a method NodeIdMappedAddr::socket_addr that returns the addr w/ the MAPPED_ADDR_PORT for when we need to use the socket address.