feat!: add iroh-relay crate (#2873)

## Description

Adds a new crate `iroh-relay`. It's structured as follows:
- protos: protocols the server is able to handle, this includes the
relay protocol, stun, and partially disco.
- server: the server code, as it was already in the og mod
- client: the client code, as it was already in the og mod
- server binary (bin `iroh-relay`) as it was previously in iroh-net's
bin.

This moves the code around in the least disruptive possible way to give
us a starting point over which to improve this. Check the notes and open
questions for future work

## Breaking Changes

- `iroh_net::relay` is removed. `RelayUrl`, `RelayMode`, `RelayNode` and
`RelayMap` are moved to the top (`iroh_net`). All other members of this
module are now moved to the new crate `iroh-relay`

## Notes & open questions

- Disco includes a transaction id defined in terms of stun's transaction
id. Every use of this is now directly from `stun_rs` to make it clear
this is not defined in terms of the relay's stun server. The relay can't
read the contents of disco packages so I consider this the right thing
to do, even if the types are actually the same.
- The relay needs to identify Disco packages to queue them separately.
Disco is not yet its own crate because we do not know its future. Based
on this I chose to duplicate the code that does the identification. But
just this signals we should start the work on deciding what disco looks
like in a post ts world to de-duplicate this code and use it from a
single source. The code as it stands is highly unlikely to change;
however, any change to the protocol's wrapper would now need to be
changed in two places, which is ofc not ideal.
- The relay's "protocol" includes quite a lot of non protocol code, that
probably belongs more to the server. I was really tempted to try to move
this code to better places but in reality this is not the time. The
purpose of this PR is among many others, to allow for future work like
this.
- What used to be called the relay module had a lot of relay-related
code without a clear objective or offering. This includes for example,
all the relay map related code. The relay as a protocol, server, and
client, has absolutely no need for these types. Relay topology is
entirely a `iroh`/`iroh-net` topic. On the same page is the
infra-related code. Nothing in the protocol, server or client is
different depending on the infra values. This is entirely a
`iroh`/`iroh-net` topic. Based on these arguments, these types,
functions, etc that were part of the relay module have actually stayed
in `iroh-net` instead.
- I did a couple passes of `udeps`, but it's messy. Optional
dependencies are their own feature so they aren't flagged as unused.
There are (in general, not in the relay) dependencies that differ in
targets, and the features combinations also affect this as well. So the
deps that remain are a "reasonable best effort" in accordance with our
CI checks. In reality, - and in particular when checking feature
combinations with `fc` - there's a lot of work left to do on that front.
- The crate's version is 0.29 because there's no point in setting the
version to 0.28, which will never be released, to then later be changed
to 0.29. This might be unexpected but it's the option that reduces
senseless future work.
- dns code is duplicated. This is unfortunate but since it's only used
in tests to configure the client there's really not much to do here.
Most of what this does are fixes we should try to upstream

## Change checklist

- [x] Self-review.
- [x] Documentation updates following the [style
guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text),
if relevant.
- [ ] ~~Tests if relevant.~~
- [x] All breaking changes documented.

Author: divagant-martian

.github/workflows/ci.yml

@@ -190,7 +190,7 @@ jobs:
         # uses: obi1kenobi/cargo-semver-checks-action@v2
         uses: n0-computer/cargo-semver-checks-action@feat-baseline
         with:
-          package: iroh, iroh-base, iroh-cli, iroh-dns-server, iroh-metrics, iroh-net, iroh-net-bench, iroh-router, netwatch, portmapper
+          package: iroh, iroh-base, iroh-cli, iroh-dns-server, iroh-metrics, iroh-net, iroh-net-bench, iroh-router, netwatch, portmapper, iroh-relay
           baseline-rev: ${{ env.HEAD_COMMIT_SHA }}
           use-cache: false
 

.github/workflows/tests.yaml

@@ -23,7 +23,7 @@ env:
   RUSTFLAGS: -Dwarnings
   RUSTDOCFLAGS: -Dwarnings
   SCCACHE_CACHE_SIZE: "50G"
-  CRATES_LIST: "iroh,iroh-metrics,iroh-net,iroh-net-bench,iroh-test,iroh-cli,iroh-dns-server,iroh-router,netwatch,portmapper"
+  CRATES_LIST: "iroh,iroh-metrics,iroh-net,iroh-net-bench,iroh-test,iroh-cli,iroh-dns-server,iroh-router,netwatch,portmapper,iroh-relay"
   IROH_FORCE_STAGING_RELAYS: "1"
 
 jobs:

Cargo.lock

@@ -8,6 +8,7 @@ members = [
   "iroh-test",
   "iroh-net/bench",
   "iroh-cli",
+  "iroh-relay",
   "iroh-router",
   "net-tools/netwatch",
   "net-tools/portmapper",

Cargo.toml

@@ -33,7 +33,7 @@ use iroh::{
         },
         Iroh,
     },
-    net::{key::PublicKey, relay::RelayUrl, NodeAddr},
+    net::{key::PublicKey, NodeAddr, RelayUrl},
 };
 use tokio::io::AsyncWriteExt;
 

iroh-cli/src/commands/blobs.rs

@@ -38,9 +38,8 @@ use iroh::{
         key::{PublicKey, SecretKey},
         metrics::MagicsockMetrics,
         netcheck,
-        relay::{RelayMap, RelayMode, RelayUrl},
         ticket::NodeTicket,
-        Endpoint, NodeAddr, NodeId,
+        Endpoint, NodeAddr, NodeId, RelayMap, RelayMode, RelayUrl,
     },
     util::{path::IrohPaths, progress::ProgressWriter},
 };

iroh-cli/src/commands/doctor.rs

@@ -12,8 +12,7 @@ use iroh::{
     client::Iroh,
     net::{
         endpoint::{DirectAddrInfo, RemoteInfo},
-        relay::RelayUrl,
-        NodeAddr, NodeId,
+        NodeAddr, NodeId, RelayUrl,
     },
 };
 

iroh-cli/src/commands/net.rs

@@ -11,7 +11,7 @@ use anyhow::Result;
 use colored::Colorize;
 use indicatif::{ProgressBar, ProgressDrawTarget, ProgressStyle};
 use iroh::{
-    net::relay::{RelayMap, RelayMode},
+    net::{RelayMap, RelayMode},
     node::{Node, RpcStatus, DEFAULT_RPC_ADDR},
 };
 use tracing::{info_span, trace, Instrument};

iroh-cli/src/commands/start.rs

@@ -13,7 +13,7 @@ use anyhow::{anyhow, bail, Context, Result};
 use iroh::{
     client::Iroh,
     docs::{AuthorId, NamespaceId},
-    net::relay::{RelayMap, RelayNode},
+    net::{RelayMap, RelayNode},
     node::GcPolicy,
 };
 use parking_lot::RwLock;

iroh-cli/src/config.rs

@@ -19,9 +19,11 @@ workspace = true
 cc = "=1.1.31" # enforce cc version, because of https://github.com/rust-lang/cc-rs/issues/1278
 
 anyhow = { version = "1" }
+axum = { version = "0.7.4", optional = true }
 backoff = "0.4.0"
 base64 = "0.22.1"
 bytes = "1.7"
+clap = { version = "4", features = ["derive"], optional = true }
 der = { version = "0.7", features = ["alloc", "derive"] }
 derive_more = { version = "1.0.0", features = ["debug", "display", "from", "try_into", "deref"] }
 futures-buffered = "0.2.8"
@@ -40,6 +42,7 @@ hyper = { version = "1", features = ["server", "client", "http1"] }
 hyper-util = "0.1.1"
 igd-next = { version = "0.15.1", features = ["aio_tokio"] }
 iroh-base = { version = "0.28.0", features = ["key"] }
+iroh-relay = { version = "0.28", path = "../iroh-relay" }
 libc = "0.2.139"
 netdev = "0.30.0"
 netwatch = { version = "0.1.0", path = "../net-tools/netwatch" }
@@ -55,6 +58,7 @@ quinn-proto = { package = "iroh-quinn-proto", version = "0.12.0" }
 quinn-udp = { package = "iroh-quinn-udp", version = "0.5.5" }
 rand = "0.8"
 rcgen = "0.12"
+regex = { version = "1.7.1", optional = true }
 reqwest = { version = "0.12", default-features = false, features = ["rustls-tls"] }
 ring = "0.17"
 rustls = { version = "0.23", default-features = false, features = ["ring"] }
@@ -72,6 +76,7 @@ tokio-tungstenite = "0.21"
 tokio-tungstenite-wasm = "0.3"
 tokio-util = { version = "0.7.12", features = ["io-util", "io", "codec", "rt"] }
 tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["env-filter"], optional = true }
 tungstenite = "0.21"
 url = { version = "2.4", features = ["serde"] }
 watchable = "1.1.2"
@@ -80,15 +85,6 @@ webpki-roots = "0.26"
 x509-parser = "0.16"
 z32 = "1.0.3"
 
-# iroh-relay
-axum = { version = "0.7.4", optional = true }
-clap = { version = "4", features = ["derive"], optional = true }
-regex = { version = "1.7.1", optional = true }
-rustls-pemfile = { version = "2.1", optional = true }
-toml = { version = "0.8", optional = true }
-tracing-subscriber = { version = "0.3", features = ["env-filter"], optional = true }
-tokio-rustls-acme = { version = "0.4", optional = true }
-
 # metrics
 iroh-metrics = { version = "0.28.0", default-features = false }
 strum = { version = "0.26.2", features = ["derive"] }
@@ -114,14 +110,12 @@ axum = { version = "0.7.4" }
 clap = { version = "4", features = ["derive"] }
 criterion = "0.5.1"
 crypto_box = { version = "0.9.1", features = ["serde", "chacha20"] }
-ntest = "0.9"
 pretty_assertions = "1.4"
-proptest = "1.2.0"
 rand_chacha = "0.3.1"
 tokio = { version = "1", features = ["io-util", "sync", "rt", "net", "fs", "macros", "time", "test-util"] }
 tracing-subscriber = { version = "0.3", features = ["env-filter"] }
 iroh-test = "0.28.0"
-iroh-net = { path = ".", features = ["iroh-relay"] }
+iroh-net = { path = "." }
 serde_json = "1.0.107"
 testresult = "0.4.0"
 mainline = "2.0.1"
@@ -135,24 +129,11 @@ duct = "0.13.6"
 
 [features]
 default = ["metrics", "discovery-pkarr-dht"]
-iroh-relay = [
-    "dep:tokio-rustls-acme",
-    "dep:axum",
-    "dep:clap",
-    "dep:toml",
-    "dep:rustls-pemfile",
-    "dep:regex",
-    "dep:tracing-subscriber"
-]
 metrics = ["iroh-metrics/metrics"]
-test-utils = ["iroh-relay"]
+test-utils = ["iroh-relay/test-utils", "iroh-relay/server", "dep:axum"]
 discovery-local-network = ["dep:swarm-discovery"]
 discovery-pkarr-dht = ["pkarr/dht", "dep:genawaiter"]
 
-[[bin]]
-name = "iroh-relay"
-required-features = ["iroh-relay"]
-
 [package.metadata.docs.rs]
 all-features = true
 rustdoc-args = ["--cfg", "iroh_docsrs"]

iroh-net/Cargo.toml

@@ -8,8 +8,7 @@ use bytes::Bytes;
 use futures_lite::StreamExt as _;
 use iroh_net::{
     endpoint::{Connection, ConnectionError, RecvStream, SendStream, TransportConfig},
-    relay::{RelayMap, RelayMode, RelayUrl},
-    Endpoint, NodeAddr,
+    Endpoint, NodeAddr, RelayMap, RelayMode, RelayUrl,
 };
 use tracing::{trace, warn};
 

iroh-net/bench/src/iroh.rs

@@ -10,11 +10,7 @@ use std::net::SocketAddr;
 use anyhow::Context;
 use clap::Parser;
 use futures_lite::StreamExt;
-use iroh_net::{
-    key::SecretKey,
-    relay::{RelayMode, RelayUrl},
-    Endpoint, NodeAddr,
-};
+use iroh_net::{key::SecretKey, Endpoint, NodeAddr, RelayMode, RelayUrl};
 use tracing::info;
 
 // An example ALPN that we are using to communicate over the `Endpoint`

iroh-net/examples/connect-unreliable.rs

@@ -10,11 +10,7 @@ use std::net::SocketAddr;
 use anyhow::Context;
 use clap::Parser;
 use futures_lite::StreamExt;
-use iroh_net::{
-    key::SecretKey,
-    relay::{RelayMode, RelayUrl},
-    Endpoint, NodeAddr,
-};
+use iroh_net::{key::SecretKey, Endpoint, NodeAddr, RelayMode, RelayUrl};
 use tracing::info;
 
 // An example ALPN that we are using to communicate over the `Endpoint`

iroh-net/examples/connect.rs

@@ -5,7 +5,7 @@
 //!     $ cargo run --example listen-unreliable
 use anyhow::Context;
 use futures_lite::StreamExt;
-use iroh_net::{key::SecretKey, relay::RelayMode, Endpoint};
+use iroh_net::{key::SecretKey, Endpoint, RelayMode};
 use tracing::{info, warn};
 
 // An example ALPN that we are using to communicate over the `Endpoint`

iroh-net/examples/listen-unreliable.rs

@@ -7,7 +7,7 @@ use std::time::Duration;
 
 use anyhow::Context;
 use futures_lite::StreamExt;
-use iroh_net::{endpoint::ConnectionError, key::SecretKey, relay::RelayMode, Endpoint};
+use iroh_net::{endpoint::ConnectionError, key::SecretKey, Endpoint, RelayMode};
 use tracing::{debug, info, warn};
 
 // An example ALPN that we are using to communicate over the `Endpoint`

iroh-net/examples/listen.rs

@@ -2,7 +2,7 @@
 
 use url::Url;
 
-use crate::relay::{RelayMap, RelayNode};
+use crate::{RelayMap, RelayNode};
 
 /// The default STUN port used by the Relay server.
 ///
@@ -164,30 +164,4 @@ pub(crate) mod timeouts {
 
     /// Default Pinger timeout
     pub(crate) const DEFAULT_PINGER_TIMEOUT: Duration = Duration::from_secs(5);
-
-    /// Timeouts specifically used in the iroh-relay
-    pub(crate) mod relay {
-        use super::*;
-
-        /// Timeout used by the relay client while connecting to the relay server,
-        /// using `TcpStream::connect`
-        pub(crate) const DIAL_NODE_TIMEOUT: Duration = Duration::from_millis(1500);
-        /// Timeout for expecting a pong from the relay server
-        pub(crate) const PING_TIMEOUT: Duration = Duration::from_secs(5);
-        /// Timeout for the entire relay connection, which includes dns, dialing
-        /// the server, upgrading the connection, and completing the handshake
-        pub(crate) const CONNECT_TIMEOUT: Duration = Duration::from_secs(10);
-        /// Timeout for our async dns resolver
-        pub(crate) const DNS_TIMEOUT: Duration = Duration::from_secs(1);
-
-        /// Maximum time the client will wait to receive on the connection, since
-        /// the last message. Longer than this time and the client will consider
-        /// the connection dead.
-        pub(crate) const CLIENT_RECV_TIMEOUT: Duration = Duration::from_secs(120);
-
-        /// Maximum time the server will attempt to get a successful write to the connection.
-        #[cfg(feature = "iroh-relay")]
-        #[cfg_attr(iroh_docsrs, doc(cfg(feature = "iroh-relay")))]
-        pub(crate) const SERVER_WRITE_TIMEOUT: Duration = Duration::from_secs(2);
-    }
 }