What should our generic low-level hostname resolution interface look like?

[njsmith]

There are really two questions here:

• What should the API that users call for hostname resolution look like?

• If someone wants to register a custom resolver implementation, then should that be possible and how should it work?

Currently we just expose the standard (RFC-defined!) getaddrinfo/getnameinfo API, and we allow custom resolvers as long as they implement that interface.

Twisted makes some slightly different choices. Their IHostnameResolver interface:

• streams back responses incrementally rather than all at once (in trio terms: it returns an async iterable instead of an iterable),

• doesn't support named service lookup (like getaddrinfo("127.0.0.1", "http"), which looks up http in /etc/services to figure out that this means port 80 – they just say you have to always pass a port number)

• doesn't offer an AI_CANONNAME flag, or any of the other AI_* flags

• doesn't provide any API for reverse resolution, AFAICT (i.e. it's only getaddrinfo, not getnameinfo)

Rationale for the first point is here, and for some others is here.

I think the big question here is whether we want to deviate from getaddrinfo by streaming responses back. The problem with this is that one of the things getaddrinfo does is sort the responses, so a full proper standards-compliant version can't possibly stream responses. Of course, maybe you have a specific use case where you don't care about this, and you want to use your more efficient lookup code that streams out responses as they come in.

But ... even this doesn't really work, because it turns out that the main consumer of these things -- happy eyeballs -- is supposed to then sort the response again (see RFC 6555 section 5.4), so it needs all the DNS responses before it starts. Or technically I guess it only needs at least 1 IPv4 and 1 IPv6 response, and it doesn't need them until after the first connection fails/times-out, but this doesn't make much difference given that the way DNS works, you get all your IPv4 and IPv6 responses in 1 packet apiece. Basically, if you know that your local policy is (say) prefer-IPv6, and the IPv6 DNS response arrives before the IPv4 DNS response, then that does let you start the first connection attempt a few milliseconds earlier, or would if anyone implemented this... meh. I don't think this is very compelling.

And the other changes are basically just minor simplifications for simplification's sake. I don't think they're useful enough on their own to deviate from the getaddrinfo spec.

So basically, I think we should provide a getaddrinfo implementation, because it's well understood and works and in most cases it actually does exactly what you want. And then if you want different semantics, well... you can of course define your own abstraction, that switches between calling trio.socket.getaddrinfo or something else. But I don't think the reasons above are compelling enough for trio to provide that abstraction; we simply have no use for it, so we'd probably get it wrong. And you really need a well-defined set of semantics for everyone to write code against; it's fine if someone wants to use c-ares to make up some sort of sorta-kinda-not-really getaddrinfo semantics that's good enough for their use case, but that should be restricted to their code then, not be something they inject globally into all the other code in the process.

I guess in twisted the main advantage of having IHostnameResolver is that it lets you use resolvers like c-ares or twisted.names along with tricky code like their happy eyeballs implementation. (Which ignores section 5.4.) But in trio even reimplementing happy eyeballs isn't that hard...

[njsmith]

I think this is resolved now – we aren't going to stop providing getaddrinfo, we have a dedicated API for tests to mock it if they need to, there's no compelling reason to add another interface on top, and if a reason shows up in the future we can always do it then.