Codespaces port forwarding adds duplicated Connection: keep-alive header breaking Websockets connection upgrade

[zioproto]

Select Topic Area

Bug

Body

I am running Uvicorn in GitHub Codespaces. I am forwarding a port with visibility public and port protocol HTTP.
My first Websocket connection works, but after a few seconds the Websocket endpoint starts returning HTTP 404.

I captured traffic at the Uvicorn server with tcpdump and I figured out that when I receive a HTTP 404 the HTTP request has a duplicated Connection header.

This is a problem and looking at the Uvicorn implementation it will make the Websockets connection fail to upgrade randomly depending on which one of the duplicated Connection header is read first.

https://github.com/encode/uvicorn/blob/1cb58c707ae7203f3b24e714947e7ab025b20895/uvicorn/protocols/http/h11_impl.py#L173-L183

According to RFC7230 there should not be duplicated Connection headers.

A sender MUST NOT generate multiple header fields with the same field
name in a message unless either the entire field value for that
header field is defined as a comma-separated list [i.e., #(values)]
or the header field is a well-known exception (as noted below).

I suspect GitHub codespaces reverse proxy is adding this header to the request, but it should actually not doing it.

Please could anyone confirm the proxy adding the duplicated Connection: Keep-Alive header is the root cause of the problem ? Or I should propose a PR to unicorn to deal with these duplicated headers ?

Thank you

Related discussions:
https://github.com/orgs/community/discussions/41915
https://github.com/orgs/community/discussions/3908

[zioproto]

I managed to patch Uvicorn to fix this problem:
encode/uvicorn#2002

But I am not sure if it is Uvicorn that should be patched or the GitHub codespaces reverse proxy that sends a malformed HTTP request.

[joshspicer]

Thanks for the feedback and the detailed report! I'll share this back with the port forwarding team.

[joshspicer]

We have an experimental implementation of public port forwarding running side by side with the default. This implementation is more slim and might avoid adding those headers you're seeing. You can try it out by starting with your app's port-forwarding URL, which looks something like:

https://joshspicer-glorious-fortnight-5vv55gj6j4jf79gp-8000.preview.app.github.dev/

And remove the preview sub-domain, so for example:

https://joshspicer-glorious-fortnight-5vv55gj6j4jf79gp-8000.app.github.dev/

If you see a trust warning on the first connecton similar to this, you're in the right place:

Let me know how that works in your scenario/let me know if those instructions were not clear :)

[zioproto]

I confirm using the forwarding domain app.github.dev without the preview fixes the issue. Thank you

[BasalGanglia]

This works for me personally now, but if I try to share this with anyone else, they get 404... In Postman, I can get it working if manually set up the Github_token into the header, in incognito mode my browser goes to GitHub login page. Is there some way to get it working without authentication, as it kind of ruins the point of public visibility if only I can access it anyway?

[jan-david-fischbach]

Any plans to fix the headers with the default url?

[jan-david-fischbach]

@joshspicer

[joshspicer]

Yes, the experimental "trick" I shared above (removing the preview segment of the sub-domain) is a new implementation of port forwarding service and is planned to be the default/only option in the near future! The team is still working on a general rollout.

The URL we inject into the environment for GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN will always point to the "current" port forwarding domain.

[AndriiTsok]

@joshspicer I am wondering if there is any ETA on the new port forwarding? :) There are some other issues with the current preview. version that seems to be fixed in the new version. Thank you!