Remove dashes to underscores normalization from http header attribute keys

[trask]

Fixes #304

Changes

Removes the dashes to underscores normalization from http header attribute keys.

Merge requirement checklist

• CONTRIBUTING.md guidelines followed.
• CHANGELOG.md updated for non-trivial changes.
• schema-next.yaml updated with changes to existing conventions.

[arminru]

The same change also needs to be applied to response headers to stay consistent:

http.response.header.<key>
HTTP response headers, <key> being the normalized HTTP Header name (lowercase, with - characters replaced by _), the value being the header values. [5]
http.response.header.content_type=["application/json"]; http.response.header.my_custom_header=["abc", "def"]

[pyohannes]

The same change also needs to be applied to response headers to stay consistent:

Definitely, and in case we decide to do this for HTTP, we probably should also do it for RPC. Likely not in this PR, but at least note it in issues:
https://github.com/open-telemetry/semantic-conventions/blob/main/docs/rpc/connect-rpc.md?plain=1#L23
https://github.com/open-telemetry/semantic-conventions/blob/main/docs/rpc/grpc.md?plain=1#L22

[Oberon00]

Likely not in this PR

I would say, please do it also in this PR. This should stay consistent. I'd rather leave HTTP unchanged than change only HTTP and leave RPC in an inconsistent state if we then in the rpc follow-up figure out the reasons for the normalization.

RPC handling adapted to match HTTP in 16bc32a

[jsuereth]

Based on our discussion in Semconv, I'm against this change.

I think we should either:

• Not normalize at all, so conflicts are always obvious, this doesn't actually solve the underlying issue.
• Normalize for consistency, including _. The notion of multiple headers and conflicts is something we have to resolve no mater what if we do any normalization of the raw data received.

[AlexanderWert]

Based on our discussion in Semconv, I'm against this change.

I think we should either:

• Not normalize at all, so conflicts are always obvious, this doesn't actually solve the underlying issue.
• Normalize for consistency, including _. The notion of multiple headers and conflicts is something we have to resolve no mater what if we do any normalization of the raw data received.

+1 (not blocking it from my side though, if we end up with lowercasing only)

I'm leaning towards:

• Not normalize at all, so conflicts are always obvious, this doesn't actually solve the underlying issue.

Lowercasing would be for the sake of cardinality, right? But, all the attributes being touched in this PR are not part of metric dimensions, so cardinality should not be an issue.

[arminru]

Approving following the reasoning @trask provided in #369 (comment)

[trask]

Lowercasing would be for the sake of cardinality, right?

Lowercasing is only for the keys (not the values, so not related to cardinality). If we don't lowercase the keys, then we end up with different keys, e.g. http.request.header.content-type and http.request.header.Content-Type.

[trask]

I think we should either:

• Not normalize at all, so conflicts are always obvious

I don't believe Content-Type and content-type are truly conflicts, so I don't think we need to make that difference obvious.

[pyohannes]

Lowercasing would be for the sake of cardinality, right?

Lowercasing is only for the keys (not the values, so not related to cardinality). If we don't lowercase the keys, then we end up with different keys, e.g. http.request.header.content-type and http.request.header.Content-Type.

👍 Agreeing with @trask, lowercasing makes sense here because we're "converting" from a case-insensitive to a case-sensitive format.

It doesn't cause conflicts (because the origin format is case-insensitive), and I think it drastically improves the user experience. If I don't have that normalization and I'd want to filter for a certain content type, I'd theoretically need to look for all http.request.header.[cC][oO][nN][tT][eE][nN][tT]-[tT][yY][pP][eE] keys.

[Flarna]

Most HTTP frameworks do the one or the other sort of normalizing of incoming headers because this eases the usage. This effects casing and concatenation/arrayfying. Simply because the spec is clear that casing doesn't matter and also it doesn't matter if headers which are allowed multiple times (e.g. tracestate) are sent as a single concat line or as array.

Similar on sending side HTTP frameworks might decide on such normalization.

In my opinion the main use of tracing is not to debug low level, framework specific low level details like the actual casing on the wire. It's more to monitor the HTTP request from functional point of view.
Allowing easy search/indexing/.. in backend by normalizing the input data without loosing functional details seems to be the better choice here.

If there is a use case to track also the low level bits and details (assuming the framework in use allows to extract this) I would add a separate set of attributes like http.request.raw_headers and http.response.raw_headers.