S3 + HTTP/2: "duplicate" header

[crepererum]

Describe the bug
The bug occurs when S3 signatures are used in combination with HTTP/2. This (IIRC) is currently not possible with AWS S3, but some other vendors or custom implementation may run into this.

The combination produces the special :authority: HTTP/2 pseudo-header as well as the Host header. Some server or middleware implementation don't like that, e.g. Nginx will return a "bad request" and emit the log (here the host was localhost:9999:

client sent duplicate host header: "host: localhost:9999", previous value: "host: localhost:9999" while reading client request headers, client: 127.0.0.1, server: , host: "localhost:9999"

Note that the header is not really duplicate. It's likely that Nginx internally renames :authority to Host and then trips over it. Also see https://trac.nginx.org/nginx/ticket/2268 .

To Reproduce

use object_store::{
    aws::AmazonS3Builder,
    ClientOptions,
};

let store = AmazonS3Builder::new()
    .with_client_options(
        ClientOptions::new()
        .with_http2_only()
     )
    .with_region(...)
    .with_access_key_id(...)
    .with_secret_access_key(...)
    .build()
    .unwrap();

store.get(...).await.unwrap();

Expected behavior

Additional context
I think the culprit is this bit here:

arrow-rs/object_store/src/aws/credential.rs

Lines 157 to 159 in 97ae9d7

	let host = &request.url()[url::Position::BeforeHost..url::Position::AfterPort];
	let host_val = HeaderValue::from_str(host).unwrap();
	request.headers_mut().insert("host", host_val);

Of course one could argue that reqwest or h2 should de-duplicate the headers properly.

[Xuanwo]

Maybe related to seanmonstar/reqwest#1809