Skip to content

Conversation

@masaori335
Copy link
Contributor

Prior to this change, all of the received payloads of DATA frames are written in the buffer of Http2Stream (request_buffer). And after HttpTunnel is set up, the data is copied to another buffer on Http2Stream::update_read_request.

It looks like we can directly write the payloads to the buffer which is created by HttpSM/HttpTunnel for the POST request.

Actually, the copy is switching chain of IOBufferBlock, so this doesn't have big performance improvement but this makes code much simpler.

@masaori335 masaori335 added this to the 10.0.0 milestone Sep 30, 2019
@masaori335 masaori335 self-assigned this Sep 30, 2019
@masaori335 masaori335 changed the title Reduce unnecesarry IOBufferBlock::alloc/dealloc on POST request over HTTP/2 Avoid unnecesarry copy on POST request over HTTP/2 Sep 30, 2019
@shinrich
Copy link
Member

Agreed that it simplifies the logic significantly if we can always rely the on the vio's buffer.

In the case of the POST data frame, can we be guaranteed that the HttpTunnel VIO is always set up when the DATA frame arrives?

@masaori335
Copy link
Contributor Author

masaori335 commented Oct 1, 2019

In the case of the POST data frame, can we be guaranteed that the HttpTunnel VIO is always set up when the DATA frame arrives?

Some DATA frames (65535 bytes, the initial window size) could arrive before the HttpTunnel is set up. But it's not a problem because the payloads of these DATA frames will be written in Http2Stream::request_buffer and copied to the new buffer when HttpSM::do_setup_post_tunnel() is called to set up the tunnel.

// Next order of business if copy the remaining data from the
// header buffer into new buffer
client_request_body_bytes = post_buffer->write(ua_buffer_reader, chunked ? ua_buffer_reader->read_avail() : post_bytes);
ua_buffer_reader->consume(client_request_body_bytes);

So this change just follows what the HTTP/1.1 component does. Http2Stream doesn't need to care about which buffer to write nor copy.

@apache apache deleted a comment from shinrich Oct 1, 2019
@masaori335
Copy link
Contributor Author

Looks like the AuTest failures are serious. It reproduced on my local box, I'm figuring out.

@masaori335 masaori335 added the WIP label Oct 3, 2019
@masaori335 masaori335 force-pushed the h2-post-perf branch 2 times, most recently from e9261f5 to b808ad3 Compare October 24, 2019 23:24
}
nbytes += stream->request_buffer.write(myreader, read_len);
nbytes += writer->write(myreader, read_len);
myreader->consume(nbytes);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Off-topic, but, consuming nbytes in the while loop looks wrong.

@masaori335 masaori335 removed the WIP label Dec 13, 2019
@masaori335 masaori335 merged commit c55001b into apache:master Dec 13, 2019
@zwoop
Copy link
Contributor

zwoop commented Dec 22, 2019

Cherry-picked to v9.0.x branch.

@zwoop zwoop modified the milestones: 10.0.0, 9.0.0 Dec 22, 2019
@zwoop
Copy link
Contributor

zwoop commented Jan 23, 2020

@shinrich and I suspect this might be the root cause of the ASAN issues on Docs (and their prod). I'm doing tests right now without this on Docs.

@zwoop zwoop modified the milestones: 9.0.0, 8.1.0 Apr 6, 2020

// Try to be smart and only signal if there was additional data
int send_event = VC_EVENT_READ_READY;
if (read_vio.ntodo() == 0 || (this->recv_end_stream && this->read_vio.nbytes != INT64_MAX)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not (read_vio.ntodo() == 0 || this->recv_end_stream) here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, this is for some AuTest failures. This checks the upper layer (HttpSM/HttpTunnel) called do_io_read(this, INT64_MAX, XXXX). Yes, it's a bit unreliable. ( I should have left some comments )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants