Python3: Accommodate both highcase and lowcase for response header key

[ziransun]

No description provided.

[stephenmcgruer]

So I understand properly; this has always been an 'issue' (spec-compliant issue, perhaps), with geckodriver. Back in py2, we didn't notice because HTTPResponse was normalizing the cases. In py3, HTTPResponse no longer normalizes the case so this fails for geckodriver?

I wonder if we should be normalizing headers in HTTPResponse then... cc @jgraham @Hexcles

[Hexcles]

Sorry I missed this. Python 2 seems to normalize (lower) the keys when parsing and storing the HTTP message (I haven't checked the source code, but just an observation based on #22858 (comment)), while Python 3 in fact does case-insensitive lookup while preserving the casing, which arguably is a better approach:

https://github.com/python/cpython/blob/7bcc6456ad4704da9b287c8045768fa53961adc5/Lib/email/message.py#L462-L468
https://github.com/python/cpython/blob/7bcc6456ad4704da9b287c8045768fa53961adc5/Lib/email/message.py#L423-L424

It was really puzzling to me why #22858 was needed in the first place.

Turns out, we actually coerced headers into a dict...

wpt/tools/webdriver/webdriver/transport.py

Line 43 in 654d5f8

headers = dict(http_response.getheaders())

We might want to stop doing that, or at least use this class from wptserve instead

wpt/tools/wptserve/wptserve/request.py

Line 373 in 42a4b17

class RequestHeaders(dict):

[stephenmcgruer]

Original discussion appears to be here: #14074

[stephenmcgruer]

Robert and I chatted about this, and we think making Response.from_http use a class like RequestHeaders would be best, as it both solves the case sensitivity problem as well as supports multiple headers (which the current code doesn't do).

The downside is that it would introduce a dependency from webdriver to wptserve, or otherwise require code duplication. At this point I am leaning towards code duplication, though it would be nice to perhaps introduce a WPT 'utils' library that everything can depend on? (Generally speaking wpt is quite messy atm, with multiple 'subprojects' that are rarely standalone subprojects anymore.)

cc @jgraham and @gsnedders

[stephenmcgruer]

Actually, looks like RequestHeaders assumes a fair amount about its input headers that isn't true for the usage in transport.py, where the headers are just a list of tuples. So the init code is slightly different at least.

[stephenmcgruer]

Playing with this in #25680

[stephenmcgruer]

Ping @jgraham @Hexcles for their opinion again, but to keep things moving I'm going to approve this PR.

EDIT: I originally said 'with a nit', but I don't have a nit. So, err, approved with no nit. :D