-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix sometimes returns response body to HEAD requests #3985
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this. High level comment to get started.
include/envoy/http/filter.h
Outdated
@@ -203,7 +203,8 @@ class StreamDecoderFilterCallbacks : public virtual StreamFilterCallbacks { | |||
* response headers. | |||
*/ | |||
virtual void sendLocalReply(Code response_code, const std::string& body_text, | |||
std::function<void(HeaderMap& headers)> modify_headers) PURE; | |||
std::function<void(HeaderMap& headers)> modify_headers, | |||
bool is_head_request) PURE; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
High level comment: Does this need to be part of the API? Would it be better if the connection manager kept track of this and did the right thing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right, I will refactor
78fa8e1
to
8e84cf8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for working on this. In general the change looks correct, but it needs tests for all the different permutations you did this for:
- AsyncClient
- conn_manager_impl (early response and filter response)
@@ -112,6 +112,10 @@ void AsyncStreamImpl::encodeTrailers(HeaderMapPtr&& trailers) { | |||
} | |||
|
|||
void AsyncStreamImpl::sendHeaders(HeaderMap& headers, bool end_stream) { | |||
if (headers.Method() && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Method should always be set so I don't think we need to check
@@ -481,6 +481,11 @@ const Network::Connection* ConnectionManagerImpl::ActiveStream::connection() { | |||
|
|||
void ConnectionManagerImpl::ActiveStream::decodeHeaders(HeaderMapPtr&& headers, bool end_stream) { | |||
request_headers_ = std::move(headers); | |||
if (request_headers_->Method() && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Method can't be null
@@ -188,6 +188,10 @@ class ConnectionManagerImpl : Logger::Loggable<Logger::Id::http>, | |||
// called here may change the content type, so we must check it before the call. | |||
FilterHeadersStatus decodeHeaders(HeaderMap& headers, bool end_stream) { | |||
is_grpc_request_ = Grpc::Common::hasGrpcContentType(headers); | |||
if (headers.Method() && |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Method can't be null
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
HttpConnectionManagerImplTest.*
have many test, there is no set method header, I'm going to fix it
@@ -396,6 +402,7 @@ class ConnectionManagerImpl : Logger::Loggable<Logger::Id::http>, | |||
// By default, we will assume there are no 100-Continue headers. If encode100ContinueHeaders | |||
// is ever called, this is set to true so commonContinue resumes processing the 100-Continue. | |||
bool has_continue_headers_{}; | |||
bool is_head_request_{false}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need this at both levels for early response before we have filters?
@@ -75,6 +75,7 @@ Http::FilterHeadersStatus FaultFilter::decodeHeaders(Http::HeaderMap& headers, b | |||
// faults. In other words, runtime is supported only when faults are | |||
// configured at the filter level. | |||
fault_settings_ = config_->settings(); | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert unrelated change
@@ -45,7 +45,6 @@ Http::FilterHeadersStatus GrpcWebFilter::decodeHeaders(Http::HeaderMap& headers, | |||
return Http::FilterHeadersStatus::Continue; | |||
} | |||
is_grpc_web_request_ = true; | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert unrelated change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, forgot to check before submission
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you have a few whitespace removals you can revert
@@ -1,5 +1,6 @@ | |||
#include "extensions/filters/http/jwt_authn/filter.h" | |||
|
|||
#include "common/http/headers.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert all changes in file
@@ -43,6 +43,7 @@ class Filter : public Http::StreamDecoderFilter, | |||
State state_ = Init; | |||
// Mark if request has been stopped. | |||
bool stopped_ = false; | |||
bool is_head_request_{false}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert
@@ -1,5 +1,6 @@ | |||
#include "extensions/filters/http/rbac/rbac_filter.h" | |||
|
|||
#include "common/http/headers.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
revert
a7c8755
to
598212d
Compare
@mattklein123 conn_manager_impl call sendLocalReply is head response, the body is empty, the other HTTP filter test conn_manager_impl didn't go, I don't know how to test, some HTTP filter integration testing, walked conn_manager_impl, I already inside the test case,The AsyncClient::sendLocalReply I didn't find a place to call Can you give me some suggestions? |
For AsyncClient, sendLocalReply() can be called in certain error cases where the router does a direct response. For example in this test (https://github.com/envoyproxy/envoy/blob/master/test/common/http/async_client_impl_test.cc#L466) I think if you change that to a HEAD request you will hit the code you modified.
It would be nice to have a conn_manager_impl_test for this, though nice job adding the integration tests, so I'm not going to block on it, but if you feel like it would be great. Check out https://github.com/envoyproxy/envoy/blob/master/test/common/http/conn_manager_impl_test.cc#L2880 and the tests around it. You should be able to setup a test like that, and do a local reply from within various filter callbacks which should drive the various flows. Thank you! |
@mattklein123 AsyncClient and conn_manager_impl added some unittests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is awesome, thanks. One small nit. @alyssawilk if you have a sec can you take a quick skim to see if you have any comments?
test/common/http/utility_test.cc
Outdated
Utility::sendLocalReply(false, callbacks, is_reset, Http::Code::PayloadTooLarge, "large", false); | ||
} | ||
|
||
TEST(HttpUtility, SendLocalReplayHeaderReply) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
typo "Replay". I think maybe you mean "SendLocalReplyHeadRequest" ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, spelling mistakes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great overall. As always I'm a big fan of integration tests - I think we could msotly copy-paste IdleTimeoutIntegrationTest.PerStreamIdleTimeoutAfterDownstreamHeaders for and easy end to end regression test.
source/common/http/utility.h
Outdated
@@ -131,9 +131,10 @@ Http1Settings parseHttp1Settings(const envoy::api::v2::core::Http1ProtocolOption | |||
* @param response_code supplies the HTTP response code. | |||
* @param body_text supplies the optional body text which is sent using the text/plain content | |||
* type. | |||
* @param is_head_request tells if this is a HEAD request |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if this is a response to a HEAD request?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I fix it
There are 2 integration tests in the change (JWT filter and RBAC filter) |
Sorry, I could have said "we could also copy-paste" as I'd mildly prefer having an integration test which does not depend on filters. The one I suggested validates the behavior in the http connection manager for all 4 protocol combinations and permutations, which I believe has value. |
Ah yes, that would be good, agreed. |
@mattklein123 I met some problems when doing the integration test, has not yet been solved, can you help a look.
|
If you stick the whole test log with -l trace to pastebin I'll take a look. If there's no disconnect it's possible Envoy thinks it has handled the full request/response pair. Given startRequest I'd think it should be in "handling request" when the idle timeout comes and so end the connection. Given we (unfortunately) allow GET with body I assume we allow HEAD with body, but if I can see full logs I might be able to help you puzzle out what's going on. |
@alyssawilk test.log, Connection should be closed, I didn't see related connection through the netstat command information, through the log view, seems to be caused by connection was closed early. I modify Modify the diff content is as follows:
Thank you very much can you help me to look at this problem |
@mattklein123 @alyssawilk I agree with your view, I re-adjusted the call point of |
@@ -43,6 +43,23 @@ class IdleTimeoutIntegrationTest : public HttpProtocolIntegrationTest { | |||
return response; | |||
} | |||
|
|||
IntegrationStreamDecoderPtr setupPerStreamIdleTimeoutHeadRequestTest() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also sorry, I should have thought of this before, but is this the same as setupPerStreamIdleTimeoutTest() apart from the method? If so, what do you think of changing setupPerStreamIdleTimeoutTest() -> setupPerStreamIdleTimeoutTest(cont char* method = "GET") to reduce similar code?
// If this is an incomplete request (end_stream=false), then connection_.onEncodeComplete(); | ||
// will not be called in StreamEncoderImpl::encodeHeaders, head request state will not be passed | ||
// to the corresponding response so here need to call this method | ||
if (!end_stream) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: needs braces around this if statement.
I'm still not a fan of calling onEncodeComplete
here as the encode is not necessarily complete. Can we just pass the state directly? This is what I was referring to previously. Can the the RequestStreamEncoderImpl hold a reference to the ClientConnection such that we can transfer this state directly? I think that might be cleaner?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You actually think that the name of onEncodeComplete will have some ambiguity. In fact, there is no complete encode.
Can I directly dynamic_cast from ConnectionImpl to ClientConnectionImpl and pass the state directly?
// to the corresponding response so here need to call this method | ||
if (!end_stream) { | ||
auto c = dynamic_cast<ClientConnectionImpl*>(&connection_); | ||
if (c) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should always be true. If we are going to do the dynamic cast we can just assume it is valid, though I'm not a huge fan of dynamic_cast for something like this. The other option is to store ClientConncetion
in RequestStreamEncoderImpl
(and server connection in ResponseStreamEncoderImpl
and then have a base class virtual function connection()
which returns the connection base class. @alyssawilk any opinions here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd suggest that just like we have a virtual onEncodeComplete that the client/server connections implement, we could have a pure virtual onEncodeHeaders. We could handle passing on head status in the ClientConnectionImpl and the ServerConnectionImpl could just be a no-op for now.
I think that would be consistent with the style of the rest of the code base, and make it clear what part of the request/response pipeline is being handled.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good - almost there!
void ClientConnectionImpl::onEncodeComplete() { | ||
// Transfer head request state into the pending response before we reuse the encoder. | ||
pending_responses_.back().head_request_ = request_encoder_->headRequest(); | ||
void ClientConnectionImpl::onEncodeComplete() {} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mild preference for moving this to the .h file
pending_responses_.back().head_request_ = request_encoder_->headRequest(); | ||
void ClientConnectionImpl::onEncodeComplete() {} | ||
|
||
void ClientConnectionImpl::onEncodeHeader(const HeaderMapImpl& headers) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
onEncodeHeaders for consistency with encodeHeaders?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I think there was a misunderstanding on this one.
I was asking the name go from onEncodeHeader -> onEncodeHeaders (add an "s")
We don't need the same arguments, just Header with an S :-)
If you can remove the extra argument and change the name that would be great!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM once this comment is addressed :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I have never seen this comment (onEncodeHeader -> onEncodeHeaders (add an "s"), so I have not paid attention to this PR in recent days.
@@ -123,6 +122,11 @@ class ConnectionImpl : public virtual Connection, protected Logger::Loggable<Log | |||
*/ | |||
virtual void onEncodeComplete() PURE; | |||
|
|||
/** | |||
* Called when header are start encoding. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Called when headers are encoded
@@ -303,6 +307,7 @@ class ServerConnectionImpl : public ServerConnection, public ConnectionImpl { | |||
|
|||
// ConnectionImpl | |||
void onEncodeComplete() override; | |||
void onEncodeHeader(const HeaderMapImpl& headers) override { UNREFERENCED_PARAMETER(headers); } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can simplify this as
void onEncodeHeader(const HeaderMapImpl&) override { }
@@ -45,7 +45,6 @@ Http::FilterHeadersStatus GrpcWebFilter::decodeHeaders(Http::HeaderMap& headers, | |||
return Http::FilterHeadersStatus::Continue; | |||
} | |||
is_grpc_web_request_ = true; | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you have a few whitespace removals you can revert
@alyssawilk @mattklein123 Thank you for some review suggestions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally looks great to me. Thank you! Will defer to @alyssawilk for further review/merge.
@alyssawilk If this PR wants to merge, what else do I need to do? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool, almost set. Please do one more master merge to pick up clang-6* and this should be good to go.
https://groups.google.com/forum/#!topic/envoy-dev/Y15Dd3MQ7Wc
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
Signed-off-by: tianqian.zyf <tianqian.zyf@alibaba-inc.com>
* origin/master: (34 commits) fuzz: fixes oss-fuzz: 9895 (envoyproxy#4189) docs: added developer docs for pprof/tcmalloc testing (envoyproxy#4194) fix sometimes returns response body to HEAD requests (envoyproxy#3985) admin: fix config dump order (envoyproxy#4197) thrift: allow translation between downstream and upstream protocols (envoyproxy#4136) Use local_info.node() instead of bootstrap.node() whenever possible (envoyproxy#4120) upstream: allow extension protocol options to be used with http filters (envoyproxy#4165) [thrift_proxy] Replace local HeaderMap with Http::HeaderMap[Impl] (envoyproxy#4169) docs: improve gRPC-JSON filter doc (envoyproxy#4184) stats: refactoring MetricImpl without strings (envoyproxy#4190) fuzz: coverage profile generation instructions. (envoyproxy#4193) upstream: rebuild cluster when health check config is changed (envoyproxy#4075) build: use clang-6.0. (envoyproxy#4168) thrift_proxy: introduce header transport (envoyproxy#4082) tcp: allow connection pool callers to store protocol state (envoyproxy#4131) configs: match latest API changes (envoyproxy#4185) Fix wrong mock function name. (envoyproxy#4187) Bump yaml-cpp so it builds with Visual Studio 15.8 (envoyproxy#4182) Converting envoy configs to V2 (envoyproxy#2957) Add timestamp to HealthCheckEvent definition (envoyproxy#4119) ... Signed-off-by: Snow Pettersen <snowp@squareup.com>
Signed-off-by: tianqian.zyf tianqian.zyf@alibaba-inc.com
Description: fix sometimes returns response body to HEAD requests
Risk Level: low
Testing: unittests and integration test
Docs Changes:
Release Notes:
[Optional Fixes #Issue] #3932
[Optional Deprecated:]