tonic v0.10.0: h2 protocol error

[Arjentix]

Bug Report

Version

$  cargo tree | grep tonic
├── tonic v0.10.0
└── tonic-build v0.10.0
├── tonic v0.10.0 (*)
└── tonic-build v0.10.0 (*)

In case you need prost version as well:

$  cargo tree | grep prost
├── prost v0.12.0
│   └── prost-derive v0.12.0 (proc-macro)
│   ├── prost v0.12.0 (*)
    ├── prost-build v0.12.0
    │   ├── prost v0.12.0
    │   │   └── prost-derive v0.12.0 (proc-macro) (*)
    │   ├── prost-types v0.12.0
    │   │   └── prost v0.12.0 (*)
├── prost v0.12.0 (*)

Platform

Darwin Amaterasu.local 22.6.0 Darwin Kernel Version 22.6.0: Wed Jul 5 22:22:52 PDT 2023; root:xnu-8796.141.3~6/RELEASE_ARM64_T8103 arm64

Description

Just tried new version of tonic & prost in my project which contains of gRPC server and client (with TLS enabled). Everything compiled well, but when running I got this error:

Caused by:
   0: status: Unknown, message: "h2 protocol error: http2 error: connection error detected: frame with invalid size", details: [], metadata: MetadataMap { headers: {} }
   1: transport error
   2: http2 error: connection error detected: frame with invalid size
   3: connection error detected: frame with invalid size

So I did a rollback to versions where everything works fine:

$  cargo tree | grep tonic         
├── tonic v0.9.2
└── tonic-build v0.9.2
├── tonic v0.9.2 (*)
└── tonic-build v0.9.2 (*)

$  cargo tree | grep prost
├── prost v0.11.9
│   └── prost-derive v0.11.9 (proc-macro)
│   ├── prost v0.11.9 (*)
    ├── prost-build v0.11.9
    │   ├── prost v0.11.9
    │   │   └── prost-derive v0.11.9 (proc-macro) (*)
    │   ├── prost-types v0.11.9
    │   │   └── prost v0.11.9 (*)
├── prost v0.11.9 (*)

[Arjentix]

Not reproducible when TLS is disabled

[honsunrise]

Same problem here

Update:

I have found the answer to the problem.

The issue occurs when using TLS while configuring the endpoint with an http:// schema.
It appears that there is a difference in how version 0.9 and 0.10 handle this situation.

[LucioFranco]

Do you get this error with 0.9 without changing any code? I would need to be able to reproduce this so if you could provide a minimal example that I can triger this with I can investigate.

[Arjentix]

Do you get this error with 0.9 without changing any code?

No, I get this error when going from 0.9 to 0.10, so it's 0.10. And yes, without changing any code, just toml files.

I would need to be able to reproduce this so if you could provide a minimal example that I can triger this with I can investigate.

I'll try to make one

[Arjentix]

@LucioFranco , here it is: https://github.com/Arjentix/tonic-bug-repr

It's basically a copy from examples/tls_client_auth with some changes

[LucioFranco]

Ok I don't think this is a tonic bug but more so a change to correct behavior. If you set http it will not use TLS so the acceptor handler on a server that accepts TLS fails. Then the return on the tcp stream from the h2 handshake is a tls error which the h2 framer can't parse since there is a frame size issue. So the fix here is to just switch to using https if you are speaking to a server that is speaking tls.

I am going to close the issue, feel free to reopen if this isn't solved from what I said above.

[andrehp]

@LucioFranco we had the same issue today when upgrading from 0.9 to 0.10. In our case we had set the URLs as grpc://... and we had to change to https://...

Given that we already initialize the grpc clients with TLS config, we didn't expect this behavior to change.

If this is the expected behavior I think it's worth adding documentation about it.

[LucioFranco]

@andrehp ah yeah grpc I don't think was explicitly supported before anyways. Yeah, it is an intended behavior so we should for sure document this in the Channel docs/Endpoint docs.