Allow recording the raw HTTP/1 headers sent and received

[dis-archive]

This allows feeding them into a WARC file (https://en.wikipedia.org/wiki/WARC_(file_format)). The rest of the request and response is already available as it's either the explicitly set body or the received response body.

This includes the final \r\n\r\n between the headers and the response (so that it can be distinguished from bare \n\n).

Needed for a similar request in reqwest: seanmonstar/reqwest#1229.

[dis-archive]

I'm already using this successfully through a patched reqwest to generate WARC files.

Remaining questions: Is this the correct API? One could hide the actual extension types this is stored in and instead provide extension methods for the response to get raw request/response header bytes. This wouldn't tie hyper to the exact types, but they could be merged/organized differently later on.

[seanmonstar]

We had something kinda like this for a little while for the C API, but then removed it. There's #3417 that re-introduces it, though it needed to target master.

It looks like this does a little more: you distinguish between request and response. Why not use use a single type? I think it's because you're trying to capture what is encoded too. FWIW, that could only be done for the client, the server wouldn't have a way to give that to the user.

Depending on the purpose, a logging connection type could also do this without any help from hyper, such as reqwest's connection_verbose() option.

[dis-archive]

Why not use use a single type? I think it's because you're trying to capture what is encoded too.

Yes, that's why. I want to capture both what headers are sent and what headers are received, in a byte-perfect fashion.

Depending on the purpose, a logging connection type could also do this without any help from hyper, such as reqwest's connection_verbose() option.

The purpose is recording requests I make into WARC files, i.e. permanently storing requests I make and responses I get through reqwest. I could try parsing that out of connection_verbose() logs, but that would be more brittle than I'd like.

[dis-archive]

I fixed the rustfmt, can I get another CI run?

[dis-archive]

I fixed the errors I saw in CI, previously I hadn't tested the nightly feature flag locally. Hopefully, everything works now.

I'd like to request another CI run.

It won't help for this PR, because the setting is not applied retroactively, but there's a setting to allow CI runs also for new contributors, it's in repository settings → "Actions" → "Approval for running fork pull request workflows from contributors " → "Require approval for first-time contributors who are new to GitHub (Only users who are both new on GitHub and who have never had a commit or pull request merged into this repository will require approval to run workflows.)".