HTTP protocol : store response headers verbatim in metadata

[jnioche]

WARCRecordFormat uses the value of the metadata key _response.headers_ to include the HTTP headers in the WARC representation. The WARCTypeValue would then be 'response' instead of resource.

Similarly we'll need to store the request for [https://github.com/DigitalPebble/sc-warc/issues/1]

[jnioche]

Storing the request headers is not easily doable with httpclient as the user agent info is not accessible from the httpget object.

[anjackson]

In case it's useful in the future, I think this is how Heritrix does it. i.e. it wraps the input and output streams at the socket level and records what happens so it can be picked apart afterwards.