"Chain xxx has no minimum gas price value configured for denomination yyy" on healthcheck

[freak12techno]

Summary of Bug

Getting the error on healthcheck:

target/debug/hermes health-check
2024-03-13T12:39:51.893654Z  INFO ThreadId(01) using default configuration from '/path/to/.hermes/config.toml'
2024-03-13T12:39:51.901030Z  INFO ThreadId(01) running Hermes v1.8.2+06dfbafb
2024-03-13T12:39:51.902289Z  INFO ThreadId(01) health_check{chain=osmosis-1}: performing health check...
2024-03-13T12:39:53.761306Z  WARN ThreadId(10) health_check{chain=osmosis-1}: chain 'osmosis-1' has no minimum gas price value configured for denomination 'uosmo'. This is usually a sign of misconfiguration, please check your chain and relayer configurations
2024-03-13T12:39:55.365919Z  WARN ThreadId(10) health_check{chain=osmosis-1}: Will use fallback value for max_block_time: `30s`. Error: response error: Internal error: genesis response is large, please use the genesis_chunked API instead (code: -32603)
2024-03-13T12:39:55.368411Z  INFO ThreadId(01) health_check{chain=osmosis-1}: chain is healthy
SUCCESS performed health check for all chains in the config

On genesis_chunked there's another opened issue on this repo, my concern here is minimal gas prices value, I have it set in config.toml:

[[chains]]
id = "osmosis-1"
type = "CosmosSdk"
rpc_addr = "http://216.250.252.46:26657"
grpc_addr = "http://216.250.252.46:9090"
rpc_timeout = "60s"
trusted_node = false
account_prefix = "osmo"
key_name = "relayer"
key_store_type = "Test"
store_prefix = "ibc"
default_gas = 1000000
max_gas = 120000000
gas_multiplier = 1.4
max_msg_num = 30
max_tx_size = 1800000
max_grpc_decoding_size = 33554432
clock_drift = "5s"
max_block_time = "30s"
ccv_consumer_chain = false
memo_prefix = "Relayed with ❤️  by 🐹 Quokka Stake"
sequential_batch_tx = false
trust_threshold = { numerator = "1", denominator = "3" }
gas_price = { price = 0.01, denom = 'uosmo' }
event_source = { mode = "push", url = "wss://rpc.osmosis.zone/websocket", batch_delay = "500ms" }
address_type = { derivation = "cosmos" }

[chains.packet_filter]
policy = "allow"
list = [
    ["transfer", "channel-73"],
    ["transfer", "channel-0"],
    ["transfer", "channel-781"],
    ["transfer", "channel-412"],
    ["transfer", "channel-2"],
]

and on my node as well.

Version

1.8.2, also tried it on master

Steps to Reproduce

1. Use this config above.
2. Try running hermes health-check or restart the app.

Acceptance Criteria

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate milestone (priority) applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[freak12techno]

On genesis_chunked, it seems like the same issue as this one: #3589

[romac]

What's the value of minimum-gas-prices in config/app.toml for the Osmosis node?

[freak12techno]

@romac just checked, this node has minimal-gas-prices as "0uosmo", so it's actually zero but is in fact set.
I also have it with other chains, for example I have Gitopia chain, with the following config:

[[chains]]
id = "gitopia"
type = "CosmosSdk"
rpc_addr = "https://rpc.gitopia.quokkastake.io/"
grpc_addr = "http://rpc.gitopia.quokkastake.io:9090"
rpc_timeout = "20s"
trusted_node = false
account_prefix = "gitopia"
key_name = "relayer"
key_store_type = "Test"
store_prefix = "ibc"
default_gas = 300000
max_gas = 3500000
gas_multiplier = 1.5
max_msg_num = 30
max_tx_size = 2000000
max_grpc_decoding_size = 33554432
clock_drift = "5s"
max_block_time = "30s"
ccv_consumer_chain = false
memo_prefix = "Relayed with ❤️  by 🐹 Quokka Stake"
sequential_batch_tx = false
event_source = { mode = "push", url = "wss://rpc.gitopia.quokkastake.io/websocket", batch_delay = "500ms" }
trust_threshold = { numerator = "1", denominator = "3" }
gas_price = { price = 0.0016, denom = "ulore" }
address_type = { derivation = "cosmos" }

and the following minimum-gas-prices as the folowing:

# The minimum gas prices a validator is willing to accept for processing a
minimum-gas-prices = "0.001ulore"

yet I have the same issue on this chain as well:

2024-03-13T15:16:36.768397Z  WARN ThreadId(55) health_check{chain=gitopia}: chain 'gitopia' has no minimum gas price value configured for denomination 'ulore'. This is usually a sign of misconfiguration, please check your chain and relayer configurations

[freak12techno]

@romac I think I figured it out: it queries the data from the on-chain endpoint, which yet doesn't return minimum-gas-price:

debug info:

        let min_gas_price: Vec<GasPrice> =
            self.query_config_params()?.map_or(vec![], |cfg_response| {
                print!("response: {:?}", cfg_response);
                parse_gas_prices(cfg_response.minimum_gas_price)
            });

hermes health-check and logs:

 ~/p/hermes   *$  target/debug/hermes health-check                                                                                                                       25.8s  Wed Mar 13 19:20:53 2024
2024-03-13T16:21:04.969916Z  INFO ThreadId(01) using default configuration from '/path/.hermes/config.toml'
2024-03-13T16:21:04.972522Z  INFO ThreadId(01) running Hermes v1.8.2+06dfbafb
2024-03-13T16:21:04.972950Z  INFO ThreadId(01) health_check{chain=osmosis-1}: performing health check...
2024-03-13T16:21:06.808225Z  WARN ThreadId(10) health_check{chain=osmosis-1}: chain 'osmosis-1' has no minimum gas price value configured for denomination 'uosmo'. This is usually a sign of misconfiguration, please check your chain and relayer configurations
2024-03-13T16:21:08.403358Z  WARN ThreadId(10) health_check{chain=osmosis-1}: Will use fallback value for max_block_time: `30s`. Error: response error: Internal error: genesis response is large, please use the genesis_chunked API instead (code: -32603)
2024-03-13T16:21:08.404199Z  INFO ThreadId(01) health_check{chain=osmosis-1}: chain is healthy
response: ConfigResponse { minimum_gas_price: "" }SUCCESS performed health check for all chains in the config

UPD: and REST endpoint of this node also returns empty minimum-gas-prices for whatever reason:

https://api-osmosis.sentinelgrowthdao.com/cosmos/base/node/v1beta1/config
{"minimum_gas_price":""}

[freak12techno]

UPD2: there are some chains (namely from those I have, it's 4 out of 6 chains: bitsong, jackal, gitopia, sentinel) which have outdated cosmos-sdk which doesn't even have this endpoint.
Example: on Gitopia: https://api.gitopia.quokkastake.io/cosmos/base/node/v1beta1/config

{
  "code": 12,
  "message": "Not Implemented",
  "details": [
  ]
}

Apparently if the chain doesn't support this endpoint, an empty vector of gas prices is used, causing this.

[romac]

Thanks so much for investigating this! How about we remove this warning altogether and only alert if the Hermes gas price is lower than the app minimum gas price, only if the latter is actually set?

[freak12techno]

@romac I think it makes sense to use the node's info if it actually was fetched. What do you think of the following?

• if the chain does not have this endpoint implemented (as in, cosmos-sdk version is too old) - maybe display a warning like "the chain's cosmos-sdk version is too old to fetch the node's minimum gas prices" (even better if Hermes won't actually do this request if it knows beforehand it's unsupported) or do/display nothing
• if the query returned error which is not "not implemented" - display a warning like "could not fetch node's minimum gas prices"
• if the query returned an empty string (like in osmosis in my example) - show a warning like the one that's currently displayed
• if the query returned non-empty string - use the same logic as we have now

[romac]

Yeah that's a more comprehensive solution indeed, sounds good! We'll get on it :)

[freak12techno]

@romac actually you can assign it to me, I want to actually try implementing it, if it's not an issue.