Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Internal panic in reqwest when transaction body is large #404

Closed
plaidfinch opened this issue Jan 26, 2022 · 0 comments · Fixed by #420
Closed

Internal panic in reqwest when transaction body is large #404

plaidfinch opened this issue Jan 26, 2022 · 0 comments · Fixed by #420
Labels
A-client Area: Design and implementation for client functionality C-bug Category: a bug E-easy Effort: Easy

Comments

@plaidfinch
Copy link
Collaborator

plaidfinch commented Jan 26, 2022

There is an open bug in reqwest (#668): it is an internal expectation that any valid Url is also a valid Uri. Per the current logic for parsing Uris and Urls, this is not always the case. The corner case that affects us is that when the length of the Url exceeds a certain threshold (unclear of the exact limit at time of writing), an internal .expect() fails in request, resulting in:

thread 'main' panicked at 'a parsed Url should always be a valid Uri: InvalidUri(TooLong)', /Users/$USER/.cargo/registry/src/github.com-1ecc6299db9ec823/reqwest-0.11.9/src/into_url.rs:70:14

We encounter this issue whenever we try to submit a transaction to the Tendermint RPC API which is sufficiently long to trigger this condition when included as hex-encoding in the URI, because we are using the URI-over-HTTP RPC API from Tendermint:

let rsp: serde_json::Value = reqwest::get(format!(
r#"http://{}:{}/broadcast_tx_sync?tx=0x{}"#,
self.node,
self.rpc_port,
hex::encode(&transaction.encode_to_vec())
))
.await?
.json()
.await?;

When the hex-encoding of the transaction is too long, the direct inclusion of that encoding in the URI of the GET request causes reqwest to panic.

This issue points to two things that need to be solved:

  1. Transactions are limited in size by the arbitrary bound of the size of a URI, and
  2. Transactions are often very very large.

This issue addresses (1); a follow-up issue will address (2).

Proposed solution to (1)

We should use the JSONRPC-over-HTTP endpoint for Tendermint instead of the URI-over-HTTP endpoint. This encodes the transaction body in the body of a POST, which can be arbitrarily large. See the documentation for the Tendermint RPC: https://docs.tendermint.com/master/rpc/.

@plaidfinch plaidfinch added A-client Area: Design and implementation for client functionality C-bug Category: a bug E-easy Effort: Easy labels Jan 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-client Area: Design and implementation for client functionality C-bug Category: a bug E-easy Effort: Easy
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant