Preserve header casing

[tomchristie]

So, it's looking like h11 is about to get support for preserving header casing, which is pretty fantastic.

Turns out we need to do some work on our side as well in order to take advantage of that, as the test case in this PR demonstrates...

def echo_raw_headers(request: httpx.Request) -> httpx.Response:
    data = [
        (name.decode("ascii"), value.decode("ascii"))
        for name, value in request.headers.raw
    ]
    return httpx.Response(200, json=data)


def test_raw_client_header():
    """
    Set a header in the Client.
    """
    url = "http://example.org/echo_headers"
    headers = {"Example-Header": "example-value"}

    client = httpx.Client(transport=MockTransport(echo_raw_headers), headers=headers)
    response = client.get(url)

    assert response.status_code == 200
    assert response.json() == [
        ["Host", "example.org"],
        ["Accept", "*/*"],
        ["Accept-Encoding", "gzip, deflate, br"],
        ["Connection", "keep-alive"],
        ["User-Agent", f"python-httpx/{httpx.__version__}"],
        ["Example-Header", "example-value"],
    ]

In the test case we're mirroring back whatever headers the transport layer is passed, so that we can check that we're receiving them with the correct casing at that point.

Without this pull request, the headers are all lowercased. After this pull request, the header casing is preserved, although Headers is still a case-insensitive dict.

In order to support this I've made some internal changes to the Headers class, while trying not to otherwise alter the behaviour.

We now store header information as a three-tuple of (raw_key, lowercase_key, value).

We always compare against and expose lowercase_key, so that Headers is always case-insensitive.

The only provisos to that are:

• We store the raw key that's used when setting header values.
• The headers.raw interface does expose headers a byte pairs, including the full original casing on the key.

I'm aware that we've got a bunch of refinements on Headers that we could make, in the same way that we've recently polished up the edges of the URL interface, but I'd like us to treat those separately to this PR.

[florimondmanca]

Awesome news!

How come tests are passing already? Has there been a release to h11 with support for preserved header casing already?

[tomchristie]

@florimondmanca Because the test is at the transport level.

It's making a request to the mock transport, which is echo'ing back what's passed to .request(method, url, headers, stream, ext).

If we actually made the request onto the wire, with httpcore, then right now we'd see it still getting lowercased, by h11.

But, not for long! ✨

[florimondmanca]

Got it! Smart :-)

[dhofstetter]

Hello everybody,
this changes will break functionality related to asgi. The asgi spec requires headers to be lower cased (see ASGI Spec). Changes made in this Pull Request break this requirement. So there are changes in 'ASGITransport' necessary (or revert this changes).

A longer description what the problems are with this changes can be found here.

BR Daniel