HTTP client: Support persistent connections (aka HTTP/1.1 Keep-Alive)

[clue]

We should support persistent connections (aka HTTP/1.1 Keep-Alive) also for the client side. This suggests major performance improvements, especially when sending a lot of requests to the same remote host via HTTPS. I've implemented keep-alive support for the server side via #405, so we should apply similar logic also for the client side:

Once a complete response is received, we should put the now idle connection into a list of idle connections. If a new outgoing request is to be sent, check if one idle connection to this destination already exists. After a very short grace period (defaults to a millisecond or so, possibly configurable), close the connection if no new outgoing request is sent.

This way, consumers of this library do not have to take care of this feature. Also, by using a very short grace period, the loop does not keep running noticeably longer than without this feature. This doesn't matter for long running scripts, but it's important for short running scripts to not appear to be "blocked".

The implementation sounds easy enough, but unfortunately requires a major refactoring first to give the Sender class more control over reusing connections. I've started looking into this a while ago and have created some working prototypes, but full spec compliance turns out to be surprisingly complex. In particular, we may have to look into retrying requests over a new connection if the server closes the connection between requests.

We welcome contributions, reach out if you want to support this project 👍

Originally posted by @clue in #39 (comment)

[WyriHaximus]

Had a look at this as well recently, and I think the best way forward is to try and split up each step because otherwise, the diff will be massive and hard to follow.

[clue]

Small progress update: I've started working on this again a couple of weeks ago and have made some good progress already: I'll file some PRs to refactor some of the existing logic the next couple of days to break up some of the existing coupling and prepare getting more control over connection handling. I'm hoping to sort out connection reuse the next weeks once this is merged.

[WyriHaximus]

Awesome, great to hear that!