-
Notifications
You must be signed in to change notification settings - Fork 329
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hermes Causes Osmosis Sentry to Crash When Accessing gRPC #1220
Comments
Hello, Thanks for opening this issue, seems quite critical! Could you please provide further info, concretely:
|
I've also experienced this. Requires restart of the node. |
I run Hermes using a service that starts it with Do you have an e-mail address I can send the log and config.toml file to? |
@jackzampolin are you also using cosmovisor to run the node ? |
gRPC should not crash full nodes so some fix needs to go into abci/SDK for that. |
Hi @lightiv, can you provide an update on this, please? Is this still a problem with the latest release? |
Hi Need to update the be latest binary which I will do today and get back with you. Thanks. |
Oh, wait. By the time I finished the above post the console had timed out and when I reconnected the node was running. I will let it go for a bit. The above is Osmosis. I will try Akash also. |
@lightiv Are you still seeing the same issue with Hermes 0.7.2? Even if we haven't gotten to using pagination yet, this version should be much lighter load on the nodes now, both at startup and during normal operations. |
I have had this running for a couple of hours on the Akash and Osmosis nodes having issues and they have not become unresponsive. The Akash node is running fine and appears to be keeping up with the current block height. The Osmosis node is crawling and continually slowing. It will not be able to get to the current block height. |
Oh, sorry need to refresh. Let me update to v0.7.2 and get back to you. I am currently on v7.0.1. |
I have upgraded to v0.7.3 and I am not having any issues with the Osmosis node crashing. The problem now is that hermes times out accessing gRPC:
I have verified that gRPC is accessible externally:
|
Can you please share with us the relevant section of your Hermes config.toml? It should look similar to this: Also, what is the output of |
The issues I was having has been resolved by installing everything on one box. I am now relaying. |
Crate
Hermes Relayer
Summary of Bug
Hermes Causes Osmosis Sentry to Crash When Accessing gRPC. gRPC is enable by default and the Osmosis Sentry node runs fine BUT when I enable Hermes to access the node it cause it to crash. I see that a transaction will output a long, long string of data and the node stops reading blocks.
Version
git version 2.25.1
Steps to Reproduce
For me, In stall the Osmosis sentry and then configure Hermes to access it. The problem occurs withing 5 minutes
Acceptance Criteria
Hermes 6 is able to access gRPC for Osmosis witout killing the node
For Admin Use
The text was updated successfully, but these errors were encountered: