feat(client): allow connecting to IPv6 link-local addresses #1937

ComputerDruid · 2019-09-11T20:14:33Z

No description provided.

ComputerDruid · 2019-09-11T20:15:09Z

Note: requires hyperium/http#343 to work (patch in Cargo.toml)

erickt · 2019-09-13T17:19:13Z

src/client/connect/dns.rs

@@ -22,7 +22,7 @@ use crate::common::{Future, Never, Pin, Poll, Unpin, task};
 /// Resolve a hostname to a set of IP addresses.
 pub trait Resolve: Unpin {
    /// The set of IP addresses to try to connect to.
-    type Addrs: Iterator<Item=IpAddr>;
+    type Addrs: Iterator<Item=SocketAddr>;


I think this is modifying a public trait, so it'd be really nice if this could be considered for 0.13 inclusion, otherwise we might need to wait for the next minor version to not break users.

src/client/connect/dns.rs

ComputerDruid · 2019-09-26T20:29:01Z

Addressed the review comments, so this is ready for review again.

Uploaded as a separate commit, I can of course squash/rebase if desired.

seanmonstar · 2019-10-07T20:11:27Z

I'm not up-to-speed on IPv6 zone IDs. It identifies a specific port? Or, why is this changing Resolve to return a SocketAddr instead of an IpAddr?

ComputerDruid · 2019-10-08T20:00:49Z

It returns a SocketAddr because it needs SocketAddrV6.scope_id to be set correctly. https://doc.rust-lang.org/std/net/struct.SocketAddrV6.html#method.scope_id , which is used to determine which network interface to use to make the connection. Since generally all interfaces have fe80::/64 link-local addresses, and link-local addresses are non-routable, there needs to be some disambiguation.

The only standard way I found that can turn a zone_id (on Linux, that's the interface name) into a scope_id (on Linux, the interface index) is getaddrinfo.

Some relevant info I found in man pages while I was digging: http://man7.org/linux/man-pages/man7/ipv6.7.html

sin6_scope_id is an ID depending on the scope of the address. It is new in Linux 2.4. Linux supports it only for link-local addresses, in that case sin6_scope_id contains the interface index (see netdevice(7))

The interface index can also be found in /proc/net/if_inet6 (where it's the second column). I don't use that in this CL since I'd rather use standard APIs, and getaddrinfo seems to do the trick.

erickt · 2019-10-15T21:29:02Z

src/client/connect/dns.rs

+        );
+
+        assert_eq!(addr.map(|a| format!("{:?}", a)), None);
+    }


We'll need a test to make sure clients don't pass the scope to servers.

Do you mean a test in this PR that hyper remembers to strip it somewhere?

Yeah, at least in the HOST header (not sure if we need to do this elsewhere). Specifically in the HOST header (but anywhere else we might be sending the URL to a remote server). Otherwise, this could potentially open up some attack surfaces. I think rfc7404 speaks more about this.

Sorry, wrong rfc. The right one is rfc6874, which states in section 4:

An HTTP client, proxy, or other intermediary MUST remove any ZoneID attached to an outgoing URI, as it has only local significance at the sending host.

seanmonstar · 2019-10-21T18:13:04Z

It returns a SocketAddr because it needs SocketAddrV6.scope_id to be set correctly. https://doc.rust-lang.org/std/net/struct.SocketAddrV6.html#method.scope_id , which is used to determine which network interface to use to make the connection.

I see, thanks for explaining! So, does this mean IPv6 addresses essentially allow encoding HttpConnector::set_local_addresses in a per-destination way?

seanmonstar · 2019-12-06T18:04:41Z

Since this has a breaking change, it'd need to be merged before 0.13 is released (target is early next week). I still don't have my head around it entirely, but it sounds like this extra info cannot exist in an IpAddr, only a SocketAddr?

a0145 · 2020-01-17T17:07:14Z

Is this change still on the table? Without it, it doesn't seem possible to use a link-local address with hyper with more than one link-local nic.

I still don't have my head around it entirely, but it sounds like this extra info cannot exist in an IpAddr, only a SocketAddr?

Per the rust implementation of SocketAddrV6, the scope id is only contained in there. It's not necessarily IP info but "connection setup" info which describes which interface to use for the connection, which is necessary when multiple link-local interfaces are present.

I see, thanks for explaining! So, does this mean IPv6 addresses essentially allow encoding HttpConnector::set_local_addresses in a per-destination way?

If there were an equivalent HttpConnector::set_local_interface(iface_id), technically that is what this notation does per destination. By specifying the zone/interface, it's possible to select which interface to use for the connection which is necessary to disambiguate where you want to connect to.

Ref: https://tools.ietf.org/html/rfc4007#section-11
Ref: https://en.wikipedia.org/wiki/Link-local_address#IPv6

seanmonstar · 2020-01-29T18:24:48Z

Is this change still on the table?

I'm still open to the change, it was mentioned by contributors they wanted to add tests. Also, at this point, it'd be a breaking change to the "resolver" API, so either a way to provide it without breakage is needed, or this needs to wait till the next major version.

If there were an equivalent HttpConnector::set_local_interface(iface_id), technically that is what this notation does per destination.

Thanks, I understand now. Sounds useful!

The DNS resolver part of `HttpConnector` used to require resolving to `IpAddr`s, and this changes it so that they resolve to `SocketAddr`s. The main benefit here is allowing for resolvers to set the IPv6 zone ID when resolving, but it also just more closely matches `std::net::ToSocketAddrs`. Closes #1937 BREAKING CHANGE: Custom resolvers used with `HttpConnector` must change to resolving to an iterator of `SocketAddr`s instead of `IpAddr`s.

seanmonstar · 2020-11-26T01:35:46Z

So, as part of the breaking changes in 0.14, I'm pulling part of this change in #2346: the resolvers should return SocketAddrs instead of IpAddrs. I didn't want to pull in percent_decoding to decode a tiny part of the authority, but I believe a custom resolver can handle that if needed.

The DNS resolver part of `HttpConnector` used to require resolving to `IpAddr`s, and this changes it so that they resolve to `SocketAddr`s. The main benefit here is allowing for resolvers to set the IPv6 zone ID when resolving, but it also just more closely matches `std::net::ToSocketAddrs`. Closes #1937 BREAKING CHANGE: Custom resolvers used with `HttpConnector` must change to resolving to an iterator of `SocketAddr`s instead of `IpAddr`s.

…#2346) The DNS resolver part of `HttpConnector` used to require resolving to `IpAddr`s, and this changes it so that they resolve to `SocketAddr`s. The main benefit here is allowing for resolvers to set the IPv6 zone ID when resolving, but it also just more closely matches `std::net::ToSocketAddrs`. Closes hyperium#1937 BREAKING CHANGE: Custom resolvers used with `HttpConnector` must change to resolving to an iterator of `SocketAddr`s instead of `IpAddr`s.

feat(client): allow connecting to IPv6 link-local addresses

f10f80b

ComputerDruid mentioned this pull request Sep 11, 2019

Allow % in IPv6 addresses to support zone identifiers. hyperium/http#343

Merged

erickt reviewed Sep 13, 2019

View reviewed changes

fix review comments

f8d96c5

erickt reviewed Oct 15, 2019

View reviewed changes

seanmonstar modified the milestone: 0.13 Dec 6, 2019

seanmonstar added the B-breaking-change Blocked: this is an "API breaking change". label Dec 10, 2019

LucioFranco mentioned this pull request May 19, 2020

Add IPv6-related options to the http sink vectordotdev/vector#2508

Open

seanmonstar added this to the 0.14 milestone Oct 9, 2020

seanmonstar mentioned this pull request Nov 26, 2020

feat(client): change DNS Resolver to resolve to SocketAddrs #2346

Merged

seanmonstar closed this in #2346 Dec 3, 2020

rcgoodfellow mentioned this pull request Oct 13, 2021

fix(client): remove ipv6 square brackets before resolving #2664

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(client): allow connecting to IPv6 link-local addresses #1937

feat(client): allow connecting to IPv6 link-local addresses #1937

ComputerDruid commented Sep 11, 2019

ComputerDruid commented Sep 11, 2019

erickt Sep 13, 2019

ComputerDruid commented Sep 26, 2019

seanmonstar commented Oct 7, 2019

ComputerDruid commented Oct 8, 2019

erickt Oct 15, 2019

seanmonstar Oct 21, 2019

erickt Oct 21, 2019

erickt Oct 22, 2019

seanmonstar commented Oct 21, 2019

seanmonstar commented Dec 6, 2019

a0145 commented Jan 17, 2020

seanmonstar commented Jan 29, 2020

seanmonstar commented Nov 26, 2020

feat(client): allow connecting to IPv6 link-local addresses #1937

feat(client): allow connecting to IPv6 link-local addresses #1937

Conversation

ComputerDruid commented Sep 11, 2019

ComputerDruid commented Sep 11, 2019

erickt Sep 13, 2019

Choose a reason for hiding this comment

ComputerDruid commented Sep 26, 2019

seanmonstar commented Oct 7, 2019

ComputerDruid commented Oct 8, 2019

erickt Oct 15, 2019

Choose a reason for hiding this comment

seanmonstar Oct 21, 2019

Choose a reason for hiding this comment

erickt Oct 21, 2019

Choose a reason for hiding this comment

erickt Oct 22, 2019

Choose a reason for hiding this comment

seanmonstar commented Oct 21, 2019

seanmonstar commented Dec 6, 2019

a0145 commented Jan 17, 2020

seanmonstar commented Jan 29, 2020

seanmonstar commented Nov 26, 2020