http/3 support in reverse_proxy module

[dPeS]

Hi Guys,

I was tweaking with reverse_proxy settings a lot and was unable to proxy http/3 traffic - does RP supports H3?

Cheers,
Daniel

[mholt]

No, only an http/3 server is supported for now. What's your use case for http3 to a proxy backend?

[WeidiDeng]

quic-go (the http3 library caddy is using) supports http3 transport, but as matt said, what's the use case?

[lxhao61]

Reverse proxy to an HTTPS endpoint:

reverse_proxy https://example.com {
	header_up Host {upstream_hostport}
}

Would like to support reverse proxying to HTTP/3 endpoints.

[francislavoie]

@lxhao61 but why? What's the benefit for you?

[mholt]

Dangit Francis, you did it again! :) Right as I'm writing my comment 🤣

Anyway, why is HTTP/3 needed here?

[lxhao61]

@lxhao61 but why? What's the benefit for you?

Such as naiveproxy (forwardproxy) application. If you don't want to build a website that supports HTTP/3 (example1.com) on this machine; then reverse proxy to a website that supports HTTP/3(example2.com), see the following:

{
	order forward_proxy before reverse_proxy
}
:443, example1.com {
	forward_proxy {
		basic_auth user pass
		hide_ip
		hide_via
		probe_resistance
	}
	reverse_proxy https://example2.com {
		header_up Host {upstream_hostport}
	}
}

[WeidiDeng]

@lxhao61 There's no reason to do that, firewall between you and your server will always see the traffics is not http3 even if reverse proxy supports http3 upstream. Enable h3 on caddy instead.

As far as I know, naïve proxy has to explicitly use quic scheme to use h3.

[lxhao61]

@lxhao61 There's no reason to do that, firewall between you and your server will always see the traffics is not http3 even if reverse proxy supports http3 upstream. Enable h3 on caddy instead.

As far as I know, naïve proxy has to explicitly use quic scheme to use h3.

caddy starts HTTP/3 by default since v2.6.0, and no additional configuration is required.

[bdr99]

I'd like to share my particular use case and why I think it would be helpful if reverse_proxy supported HTTP/3 for communication with upstream backends.

I run a double-layer Caddy setup with two Caddy instances chained together. One instance runs on a home server in my private network, and the other runs in a public VPS. My home network has the HTTP ports closed, and the public VPS has them open. The public Caddy instance on the VPS routes requests to the private Caddy instance running on my home server via a Tailscale VPN tunnel. This way, my public IP is not visible to users of the sites I'm hosting with Caddy. It also allows me to host private sites that should only be accessible on my local network, by adding them to only the private Caddy instance and not the public one, and doing some DNS trickery to route requests originating from my local network directly to the private instance, bypassing the public one.

So, with this setup, requests to my sites are routed through two Caddy instances. With Caddy 2.6, the traffic between the client device and the public Caddy instance can use HTTP/3, but the traffic between the public Caddy instance and the private one cannot, since reverse_proxy does not support HTTP/3. It would be nice it did, since that way, I would be able gain the benefits of HTTP/3 for this additional hop in the chain.

[mholt]

Alrighty. Well, we can get around to it sooner or later. It's not a high priority atm but we'd welcome a proposal on how to implement this and a PR to review, if someone wanted this sooner!

[bdr99]

@mholt Thanks! I agree it certainly shouldn't be a high priority. At this point it's just a nice to have.

[francislavoie]

@huddhudd that's not a helpful comment. Please elaborate. We need to understand people's usecases for needing this. Why do you need it? What would it let you do that you couldn't do before?

[huddhudd]

Accelerate my local apps.

App ->（caddyserver ） 127.0.0.1:88 -> （ Cdn ） Http3

Because my local application does not support Quic, and there are a large number of multi-threaded segmentation requests

Http Header
Range: bytes=0-1023

So the speed is very slow.

[burgerlander]

I'd like to share my particular use case and why I think it would be helpful if reverse_proxy supported HTTP/3 for communication with upstream backends.

I run a double-layer Caddy setup with two Caddy instances chained together. One instance runs on a home server in my private network, and the other runs in a public VPS. My home network has the HTTP ports closed, and the public VPS has them open. The public Caddy instance on the VPS routes requests to the private Caddy instance running on my home server via a Tailscale VPN tunnel. This way, my public IP is not visible to users of the sites I'm hosting with Caddy. It also allows me to host private sites that should only be accessible on my local network, by adding them to only the private Caddy instance and not the public one, and doing some DNS trickery to route requests originating from my local network directly to the private instance, bypassing the public one.

So, with this setup, requests to my sites are routed through two Caddy instances. With Caddy 2.6, the traffic between the client device and the public Caddy instance can use HTTP/3, but the traffic between the public Caddy instance and the private one cannot, since reverse_proxy does not support HTTP/3. It would be nice it did, since that way, I would be able gain the benefits of HTTP/3 for this additional hop in the chain.

Chiming in to say that my use case is identical to this one except I'm using a plain wireguard tunnel to chain together both the exposed server and the reverse tunnel client, rather than tailscale. Both servers are located in two different countries, so any latency improvement, no matter how minor, would definitely be welcome.

[youguys]

Hey guys:

In my case:

UI---->local web server(caddy)----(http/3)--->remote web server(caddy)------(http/1)app

Only udp port was enable at remote web server, so i need to use http3(udp) as a tunnel
Maybe i should replace "local web server" to "Traefik" which support http3 entrypoint

entryPoints: name: http3: {}

[joshughes]

Many cloud providers are beginning to support http/3 on their load balancers. There are many cases where http/3 connections to upstreams could be beneficial such as less reliable networks. It would be an interesting feature for caddy to support http3 as a target protocol for upstream servers. Even more interesting if this could be negotiated like browser decide to upgrade to http/3 use the alt-svc header on an upstream to perform this upgrade.

There are some complications like firewall rules and other things where TCP 443 may be open but UDP 443 may not. But in any case being able to explicitly tell caddy to try http3 on a reverse proxy upstream with a configuration flag would be pretty interesting, useful in many cases, and a differentiator as this currently does not exist in anything that could be found currently.

POC of a http 1.1 to http3 upstream can be seen at https://github.com/AeroNotix/go-quic-proxy

An interesting use case that is maybe specific is IOT devices on less reliable networks. This is where HTTP3 shines. Many times on these devices you will be running software that may or may not have libraries for HTTP3 or may not have mature HTTP3 options. Being able to run a Caddy HTTP3 revers proxy on these devices to give extra reliability and performance on these types of devices would be a huge win.

[mholt]

Thanks -- as mentioned above, this isn't a high priority for me right now. If someone would like to sponsor this work, then I can prioritize it; or if someone would like to propose how this would be implemented and then submit a PR that would also help move this along.

[huddhudd]

I also have use cases, haven't there been any progress yet

[LDprg]

Just want to mention that there is a fork of NPM supporting http3. So maybe caddy should add support too.

[masx200]

Many of Cloudflare's websites have been blocked.

any update?
I found that the https TLS connection of the existing website was attacked by a man-in-the-middle, and the connection was disconnected, and the connection was successful only by using quic "xxxxxxxxxxx.xxxxxxxxxxxxxxxxx.xxxxxxxxx.workers.dev"!

我发现有的网站的https的tls连接被中间人攻击了,连接被断开了,只有使用quic才能连接成功!比如说"xxxxxxxxxxx.xxxxxxxxxxxxxxxxx.xxxxxxxxx.workers.dev".

At present, browsers Firefox, Chrome, and Edge already support HTTP3, and there is also a query of HTTPS type through DNS, and you can find out the server's support for HTTP3.

目前浏览器firefox,chrome,edge都已经支持http3了,还有通过dns查询https类型,可以查到服务器对于http3的支持情况.

I found that Chrome was able to open Cloudflare's workers.dev website, but caddy couldn't access workers.dev.

[mholt]

Latest update is still #5086 (comment)

If someone would like to sponsor this work, I'd be happy to prioritize it sooner. 👍

[demarcush]

When using caddy as a reverse proxy for something like dns-over-https, having support for h3 makes software like dnscrypt-proxy and others do a DNS-over-HTTP/3 query.

[mholt]

@joshughes

Even more interesting if this could be negotiated like browser decide to upgrade to http/3 use the alt-svc header on an upstream to perform this upgrade.

But why would the browser care or why should the browser be able to control the connection to the backend? That should be squarely on the proxy to decide IMO.

Also, just to clarify, for everyone suggesting use cases: you are talking about the specific need for a terminated, altered HTTP/3 connection from the proxy to the backend? i.e. NOT a UDP proxy, which routes the HTTP/3 connection directly from the client to the backend without terminating it at the proxy?

If you need unterminated HTTP/3 to go transparently from a client to a backend, you'll want Project Conncept (a layer 4 plugin for Caddy).

I just want to clarify first that what is needed is for a terminated HTTP/3 connection between proxy and backend... 🤔

[mholt]

@dPeS @demarcush @masx200 @LDprg @burgerlander @huddhudd @bdr99 @lxhao61 (phew, that's a lot of you) I have a PR at #6312 if you'd like to try out HTTP/3 to the backend. I just did a quick test and I have no idea if it'll meet your various needs and use cases, but please give it a shot. (I am skeptical this is actually what you all need, but maybe we'll be pleasantly surprised and it'll just work ...?)

[joshughes]

@mholt my case is pretty specific but Caddy is running on an IOT device out in the wild where it will experience vairous degrees of bad wifi or poor connection. If I can use HTTP3 to talk to my servers I can get better overall performance in communication from my IOT device to the cloud. There are various things going in in the Caddy proxy on the device which are helpful and we have written our own little golang http3 proxy with go-quic that then Caddy talks to. It would be nice to be able to remove that which is why I commented on this issue when I came across it.

Thanks for taking a look at it, I will take your PR for a spin and provide any feedback I can.

Also we did several benchmarks and certain see good benifit from using HTTP3 from our IOT device in our challenging environments.