Keeping gogoprotobuf codec fast and thread safe

[rohitsaboo]

/cc @awalterschulze

Hi grpc-go team,

It would be great to have your advice on the best way to proceed keeping gogoprotobuf codec both efficient and thread safe.

The generated grpc stubs call grpc/call.go:Invoke, grpc/call.go:sendRequest, and grpc/rpc_util.go:encode, in that order. A custom codec may be simultaneously called from different go routines. I assume grpc expects codecs to be thread-safe. Therefore, using a static memory buffer inside a codec can result in thread-safety issues.

Not having to allocate and deallocate buffers greatly increases efficiency in Golang. In C++, grpc provides arena support which partially ameliorates this issue. Without going down the arena route, one way could be that gogoprotobuf keeps a "free list" that is recycled, but that is effectively doing memory management oneself and goes against Golang's principles.

How would you suggest doing this while managing to attain both objectives?

Thanks,
Rohit

[menghanl]

Closing stale issue.
And https://github.com/golang/protobuf/ would be a better place to discuss issues on protobuf.

[awalterschulze]

Uhm I don't think its stale, we are still waiting on your reply?

[dfawley]

Sorry for the delay in response.

For now, this is not possible given the current API. However, we will be investigating changing this as part of #1455 -- one idea is to see if the codec implements "Free([]byte)", and if so, call that when we are done with the buffer it returned.