Skip to content

[UBP/baseserver] Allow gRPC server to receive much larger messages #12003

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 9, 2022

Conversation

andrew-farries
Copy link
Contributor

Description

Increase baseserver's gRPC maximum receivable message size to 100MB. Change the usage client's maximum send and receive values to the same size.

The usage component is currently failing with this error:

{"@type":"type.googleapis.com/google.devtools.clouderrorreporting.v1beta1.ReportedErrorEvent","error":"failed to update invoices: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (33395302 vs. 16777216)","level":"error","message":"Reconciliation run failed.","serviceContext":{"service":"usage","version":"commit-8e227e13ef2caae8f56c26387a78546b9fa93b30"},"severity":"ERROR","time":"2022-08-09T11:33:09Z"}

This change should allow the UpdateInvoices RPC (and all other gRPC services registered with baseserver) to accept larger message sizes.

Related Issue(s)

Fixes #11997

How to test

Release Notes

NONE

Documentation

Werft options:

  • /werft with-preview

Andrew Farries added 3 commits August 9, 2022 11:16
Add a test to ensure that the gRPC server can receive very large message
payloads.

Use the billing RPC service `UpdateInvoices` RPC as the RPC to call as
that is the one we see failing in production.
Allow the gRPC server to receive messages up to 100MB in size.
@andrew-farries andrew-farries marked this pull request as ready for review August 9, 2022 13:37
@andrew-farries andrew-farries requested review from a team August 9, 2022 13:37
@github-actions github-actions bot added team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team labels Aug 9, 2022
@@ -298,6 +298,7 @@ func (s *Server) initializeGRPC() error {
opts = append(opts, grpc.Creds(credentials.NewTLS(tlsConfig)))
}

opts = append(opts, grpc.MaxRecvMsgSize(100*1024*1024))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we not change the gRPC base server default message size and let the caller overwrite the default message size? Because this is an optional for all gRPC server, WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can probably provide an option to set this.

The question is whether a user of the baseserver would ever care that the maximum message size was too large. IMO, probably not in which case an option is not needed. Perhaps we should add the option when it's needed.

@roboquat roboquat merged commit 33b1603 into main Aug 9, 2022
@roboquat roboquat deleted the af/fix-grpc-size branch August 9, 2022 14:57
@roboquat roboquat added deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production deployed Change is completely running in production labels Aug 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: webapp Meta team change is running in production deployed: workspace Workspace team change is running in production deployed Change is completely running in production release-note-none size/M team: webapp Issue belongs to the WebApp team team: workspace Issue belongs to the Workspace team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix gRPC message size error in UpdateInvoices call
4 participants