Clear fewer headers on 1xx/204/304 responses

[andersk]

This function is called on more than just 304 responses; it’s important to permit the Allow header on 204 responses. Also, the relevant RFCs have changed significantly.

Fixes #2726.

[bdarnell]

We should definitely permit the Allow header for 204 responses, but the name _clear_headers_for_304 is no longer literally accurate so I don't know if this is the change we want to make.

The primary purpose of this method is to clear the Content- headers from responses with no body, especially the Content-Type header that is added by set_default_headers. It's called for 1xx, 204, and 304 responses. I think we still want to do at least this much.

The RFCs have changed significantly since this function was written. It cites the definition of entity headers from RFC 2616 §7.1, which appears to have been replaced by a much shorter list of representation metadata headers in RFC 7231 §3.1 (Allow is no longer on the list). RFC 2616 §10.3.5 said that 304 responses MUST NOT contain other entity headers, while RFC 7232 §4.1 makes it a SHOULD NOT (and Last-Modified is given as an example of a header that's sometimes useful).

So this is what I'd do: rename _clear_headers_for_304 to _clear_representation_headers, change it to only clear Content-Type, Content-Language, and Content-Encoding, and keep calling it for all 1xx responses, 204, and 304. Make sense?

[andersk]

Sure. Updated as such.

[bdarnell]

Looks good, now the test test_static_304_if_modified_since just needs to be fixed up.

[SuminAndrew]

... shoud not contain representation metadata headers...

[andersk]

Thanks, fixed, now with passing CI.

[bdarnell]

Thanks!