Change hermes to use broadcast_tx_sync for transaction submission (#1091

)

* temp change for ft-transfer to send one msg/Tx

* event monitor: Bulk events from all transactions included in a block

* Update changelog

* temp change for ft-transfer to send one msg/Tx

* Optimize spawning of workers - draft

* Add back check to start workers only if channel is open

* Cleanup

* Check connection state

* temp change for ft-transfer to send one msg/Tx

* Improve config loading message (#933)

* Improve config load message

* Raised log level to error. Log link to config example in the guide.

* Changelog

Co-authored-by: Adi Seredinschi <adi@informal.systems>

* Migration to tx sync

* Add Tx simulate

* Add adjustment to the tx simulate result, some cleanup

* Nitpick in CosmosSdkChain::key_and_bytes

* Small cleanup

* Remove duplicate send_msgs method

* Cleanup config file

* Fix typo after refactoring

* Fix `query packet tx` description

* Rework `wait_for_block_commits` to use the `retry` crate

* Compute tx fee based on gas from tx simulate and gas price

* Re-add missing error type

* Combine `fee_denom` and `gas_price` into `gas_price` config option

* Add tests for `mul_ceil`

* Fix config serialization

* Remove `fee_amount` config property

* Update changelog

* Avoid op data regeneration if retries exhausted.

* Increase the number of retries while checking Tx is included in a block.

* Move `query packet tx` to `query tx events`

* better error msgs

* Add Display instance for GasPrice

* Fix default gas price denomination

* Improve some debug messages

* Rename `gas_price.amount` to `gase_price.price`

* Add configurable fee adjustment to make it more likely gas estimate ends up being enough

* Add Tx hash to ChainErr

* Fix config files

* Masked tonic::Code::NotFound result for query_client_connections.

* Modified cfg option from gas to max_gas

* Consistent trust_threshold in default config.toml

* Revert guide updates

* Nit: Imports for query.rs

* Print info message when Hermes starts

* Fix gas adjustement to be percentage on top of computed gas

* Variables renamed to be more explicit in calculate_fee and send_tx.

* Cached acct number in CosmosSDK

* Cache account information

* Remove noisy tracing in counterparty module

* Fix unused import

* Improve logs a bit by using the Display impl for Height

* Only show first 100 consensus states in logs

* Attempt to fix gas_limit

cherry-picked
0886401
from romac/channel-filter

* Adjust up to max_gas

Co-authored-by: Romain Ruetschi <romain@informal.systems>
Co-authored-by: Adi Seredinschi <adi@informal.systems>

CHANGELOG.md

@@ -18,25 +18,34 @@
 
 - [ibc-relayer]
   - Update the on-chain IBC client with supporting headers when light client verification
-    does bisection when verifying a header for a client update or a misbehaviour detection ([#673])
+    performs bisection when verifying a header for a client update or a misbehaviour detection ([#673])
+  - Add mitigation for chain impersonation attacks ([#1038])
+  - Determine gas fee dynamically per transaction ([#930])
+  - Submit transactions with `broadcast_tx_sync` and keep track of account sequences ([#986])
 
 ### BUG FIXES
 
 - [gaiad-manager]
-  - Removed the testnet command as not all networks support it. ([#1050])
-  - Update to compatibility with hermes's new `--hd-path` option.
+  - Removed the testnet command as not all networks support it ([#1050])
+  - Update for compatibility with Hermes's new `--hd-path` option
 
-- [ibc-relayer-cli]
-  - Fix for chain impersonation bug. ([#1038])
-  - Fix for partially open handshake bug of `channel create` CLI. ([#1064])
+- [ibc-relayer]
+  - Fix bug where channels were left partially open after `channel create` ([#1064])
+  - Prevent account sequence mismatch errors in many cases ([#919], [#978])
+  - Prevent timeouts when submitting transactins ([#977])
 
 ### BREAKING CHANGES
 
 - [ibc-relayer-cli]
-  - Removed `--coin-type` option from `keys restore` command. Use `--hd-path` instead. ([#1049])
-  
+  - Removed `--coin-type` option from `keys restore` command. Use `--hd-path` instead ([#1049])
+
 [#673]: https://github.com/informalsystems/ibc-rs/issues/673
 [#877]: https://github.com/informalsystems/ibc-rs/issues/877
+[#919]: https://github.com/informalsystems/ibc-rs/issues/919
+[#930]: https://github.com/informalsystems/ibc-rs/issues/930
+[#977]: https://github.com/informalsystems/ibc-rs/issues/977
+[#978]: https://github.com/informalsystems/ibc-rs/issues/978
+[#986]: https://github.com/informalsystems/ibc-rs/issues/986
 [#1038]: https://github.com/informalsystems/ibc-rs/issues/1038
 [#1049]: https://github.com/informalsystems/ibc-rs/issues/1049
 [#1050]: https://github.com/informalsystems/ibc-rs/issues/1050

ci/simple_config.toml

@@ -11,9 +11,8 @@ rpc_timeout = '10s'
 account_prefix = 'cosmos'
 key_name = 'testkey'
 store_prefix = 'ibc'
-fee_denom = 'stake'
-fee_amount = 10
-gas = 400000
+max_gas = 400000
+gas_price = { price = 0.001, denom = 'stake' }
 clock_drift = '5s'
 trusting_period = '14days'
 
@@ -26,8 +25,7 @@ rpc_timeout = '10s'
 account_prefix = 'cosmos'
 key_name = 'testkey'
 store_prefix = 'ibc'
-fee_denom = 'stake'
-fee_amount = 10
-gas = 400000
+max_gas = 400000
+gas_price = { price = 0.001, denom = 'stake' }
 clock_drift = '5s'
 trusting_period = '14days'

config.toml

@@ -19,15 +19,11 @@ rpc_timeout = '10s'
 account_prefix = 'cosmos'
 key_name = 'testkey'
 store_prefix = 'ibc'
-gas = 3000000
-fee_denom = 'stake'
-fee_amount = 10
+max_gas = 3000000
+gas_price = { price = 0.001, denom = 'stake' }
 clock_drift = '5s'
 trusting_period = '14days'
-
-[chains.trust_threshold]
-numerator = '1'
-denominator = '3'
+trust_threshold = { numerator = '1', denominator = '3' }
 
 [[chains]]
 id = 'ibc-1'
@@ -38,13 +34,8 @@ rpc_timeout = '10s'
 account_prefix = 'cosmos'
 key_name = 'testkey'
 store_prefix = 'ibc'
-gas = 3000000
-fee_denom = 'stake'
-fee_amount = 1000
+max_gas = 3000000
+gas_price = { price = 0.001, denom = 'stake' }
 clock_drift = '5s'
 trusting_period = '14days'
-
-[chains.trust_threshold]
-numerator = '1'
-denominator = '3'
-
+trust_threshold = { numerator = '1', denominator = '3' }

e2e/e2e/cmd.py

@@ -13,7 +13,7 @@ class Config:
     config_file: Path
     relayer_cmd: str
     log_level: str
-    max_retries: int = 5
+    max_retries: int = 10
 
 
 T = TypeVar('T')

e2e/e2e/packet.py

@@ -194,7 +194,7 @@ def query_unreceived_acks(
 def packet_send(c: Config, src: ChainId, dst: ChainId,
                 src_port: PortId, src_channel: ChannelId,
                 amount: int, height_offset: int, number_msgs: Optional[int] = None,
-                key: Optional[str] = None) -> Packet:
+                key: Optional[str] = 'user2') -> Packet:
 
     cmd = TxPacketSend(dst_chain_id=dst, src_chain_id=src,
                        src_port=src_port, src_channel=src_channel,

modules/src/query.rs

@@ -1,3 +1,5 @@
+use tendermint::abci::transaction::Hash;
+
 use crate::ics02_client::client_consensus::QueryClientEventRequest;
 use crate::ics04_channel::channel::QueryPacketEventDataRequest;
 
@@ -6,4 +8,8 @@ use crate::ics04_channel::channel::QueryPacketEventDataRequest;
 pub enum QueryTxRequest {
     Packet(QueryPacketEventDataRequest),
     Client(QueryClientEventRequest),
+    Transaction(QueryTxHash),
 }
+
+#[derive(Clone, Debug)]
+pub struct QueryTxHash(pub Hash);

relayer-cli/src/commands/query.rs

@@ -11,6 +11,7 @@ mod clients;
 mod connection;
 mod connections;
 mod packet;
+mod tx;
 
 /// `query` subcommand
 #[derive(Command, Debug, Options, Runnable)]
@@ -41,6 +42,10 @@ pub enum QueryCmd {
     /// The `query packet` subcommand
     #[options(help = "Query information about packets")]
     Packet(QueryPacketCmds),
+
+    /// The `query tx` subcommand
+    #[options(help = "Query information about transactions")]
+    Tx(tx::QueryTxCmd),
 }
 
 #[derive(Command, Debug, Options, Runnable)]

relayer-cli/src/commands/query/packet.rs

@@ -5,22 +5,20 @@ use serde::Serialize;
 use subtle_encoding::{Encoding, Hex};
 use tokio::runtime::Runtime as TokioRuntime;
 
+use ibc::ics02_client::client_state::ClientState;
 use ibc::ics04_channel::packet::{PacketMsgType, Sequence};
 use ibc::ics24_host::identifier::{ChainId, ChannelId, PortId};
 use ibc::Height;
-
 use ibc_proto::ibc::core::channel::v1::{
     PacketState, QueryPacketAcknowledgementsRequest, QueryPacketCommitmentsRequest,
     QueryUnreceivedAcksRequest, QueryUnreceivedPacketsRequest,
 };
-
+use ibc_relayer::chain::counterparty::channel_connection_client;
 use ibc_relayer::chain::{runtime::ChainRuntime, CosmosSdkChain};
 
 use crate::conclude::Output;
 use crate::error::{Error, Kind};
 use crate::prelude::*;
-use ibc::ics02_client::client_state::ClientState;
-use ibc_relayer::chain::counterparty::channel_connection_client;
 
 #[derive(Serialize, Debug)]
 struct PacketSeqs {

relayer-cli/src/commands/query/tx.rs

@@ -0,0 +1,13 @@
+//! `query tx` subcommand
+
+use abscissa_core::{Command, Options, Runnable};
+
+mod events;
+
+/// `query tx` subcommand
+#[derive(Command, Debug, Options, Runnable)]
+pub enum QueryTxCmd {
+    /// The `query tx events` subcommand
+    #[options(help = "Query the events emitted by transaction")]
+    Events(events::QueryTxEventsCmd),
+}

relayer-cli/src/commands/query/tx/events.rs

@@ -0,0 +1,56 @@
+use std::str::FromStr;
+use std::sync::Arc;
+
+use abscissa_core::{Command, Options, Runnable};
+use tokio::runtime::Runtime as TokioRuntime;
+use tracing::debug;
+
+use tendermint::abci::transaction::Hash;
+
+use ibc::ics24_host::identifier::ChainId;
+use ibc::query::{QueryTxHash, QueryTxRequest};
+
+use ibc_relayer::chain::runtime::ChainRuntime;
+use ibc_relayer::chain::CosmosSdkChain;
+
+use crate::conclude::Output;
+use crate::prelude::app_config;
+
+/// Query the events emitted by transaction
+#[derive(Clone, Command, Debug, Options)]
+pub struct QueryTxEventsCmd {
+    #[options(free, required, help = "identifier of the chain to query")]
+    chain_id: ChainId,
+
+    #[options(free, required, help = "transaction hash to query")]
+    hash: String,
+}
+
+// cargo run --bin hermes -- query tx events ibc-0 B8E78AD83810239E21863AC7B5FC4F99396ABB39EB534F721EEF43A4979C2821
+impl Runnable for QueryTxEventsCmd {
+    fn run(&self) {
+        let config = app_config();
+
+        debug!("Options: {:?}", self);
+
+        let chain_config = match config
+            .find_chain(&self.chain_id)
+            .ok_or_else(|| format!("chain '{}' not found in configuration file", self.chain_id))
+        {
+            Err(err) => return Output::error(err).exit(),
+            Ok(result) => result,
+        };
+
+        let rt = Arc::new(TokioRuntime::new().unwrap());
+        let (chain, _) = ChainRuntime::<CosmosSdkChain>::spawn(chain_config.clone(), rt).unwrap();
+
+        let res = chain.query_txs(QueryTxRequest::Transaction(QueryTxHash(
+            Hash::from_str(self.hash.as_str()).unwrap(),
+        )));
+
+        match res {
+            Ok(res) => Output::success(res).exit(),
+            Err(e) => Output::error(format!("{}", e)).exit(),
+        }
+    }
+}

relayer-cli/src/commands/start.rs

@@ -15,7 +15,10 @@ impl Runnable for StartCmd {
     fn run(&self) {
         let config = app_config();
 
-        match spawn_supervisor(config.clone()).and_then(|s| s.run()) {
+        match spawn_supervisor(config.clone()).and_then(|s| {
+            info!("Hermes has started");
+            s.run()
+        }) {
             Ok(()) => Output::success_msg("done").exit(),
             Err(e) => Output::error(format!("Hermes failed to start, last error: {}", e)).exit(),
         }

relayer-cli/tests/fixtures/two_chains.toml

@@ -12,7 +12,6 @@ account_prefix = 'cosmos'
 key_name = 'testkey'
 store_prefix = 'ibc'
 client_ids = ['cla1', 'cla2']
-gas = 200000
 clock_drift = '5s'
 trusting_period = '14days'
 
@@ -30,7 +29,6 @@ account_prefix = 'cosmos'
 key_name = 'testkey'
 store_prefix = 'ibc'
 client_ids = ['clb1']
-gas = 200000
 clock_drift = '5s'
 trusting_period = '14days'
 

relayer/Cargo.toml

@@ -54,6 +54,7 @@ dirs-next = "2.0.0"
 dyn-clone = "1.0.3"
 retry = { version = "1.2.1", default-features = false }
 async-stream = "0.3.2"
+fraction = {version = "0.8.0", default-features = false }
 
 [dependencies.tendermint]
 version = "=0.19.0"