rpc performance degradation

[ugorji]

@markgoodhead made a comment in #113 (see #113 (comment) )

Bump for this - we've just identified a recent change to master here which had a dramatic effect (negatively) on our application RPC performance and don't yet know which commits to master have done this.

[ugorji]

@markgoodhead

I know I removed buffering from the RPC codebase, as it forced the potential consumption of more bytes than necessary. To manage this, we exposed a BufferedReader and BufferedWriter accessors. This wasn't a good design.

I will add a NewReadWriteCloser function that takes a buf size and allows you create a bufio wrapper and pass that in. Hopefully that works for you.

[markgoodhead]

Yes it was related to the buffer changes - we'd already resolved by essentially implementing what you suggested:

type bufferedConn struct {
	io.ReadCloser
	*bufio.Writer
}

...

bc := bufferedConn{
		ReadCloser: conn,
		Writer:     bufio.NewWriterSize(conn, 131072),
}
var mh codec.MsgpackHandle
rpcCodec := codec.MsgpackSpecRpc.ClientCodec(bc, &mh)

[ugorji]

This is good, and much better (IMO) than increasing the surface area of the API with a new function when it is just as easy to create yourself.

I will quickly remove the NewReadWriteCloser() function and exposed type, and just update the documentation with some sample code (basically what you have here).

[ugorji]

Closed via f894406

[ugorji]

@markgoodhead with the new change, you shouldn't have to create a bufferedConn anymore.

The downside to buffering only existed during a read, as we might read more than we should into the buffer without providing a way to see it.

However, during write, it is completely ok to buffer, as long as we flush at the end of each write (which we always do). Consequently, I added write buffering back implicitly.