-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Idle thread leak in session manager #4089
Comments
related how? |
Lots of idle threads could lead to thread starvation / context switching issues since |
interesting, need to investigate where this can happen |
this appears to be a bug in tokio-console with nested spawns (which is the case here) the list of idle tasks is definitely reported wrong, because we restrict how many outbound connections can be active at once, I wonder if we should wrap this in a timeout task |
Ah thanks, I'll close this. Since we were already discussing in this thread I wanted to check my understanding - for the rpc servers they are using their own threads with |
Describe the bug
I wanted to try out
tokio-console
from the existing PR. After having that running for a bit, I noticed idle threads continuously growing which might be related to other RPC timeouts others have been running into.After adding some logging, it looks like in session manager in
start_pending_outbound_session
every once in a while the the future never returns and a thread is left idle.reth/crates/net/network/src/session/mod.rs
Lines 797 to 810 in aaf2d2c
Steps to reproduce
tokio-console
and watch idle threads growNode logs
No response
Platform(s)
Mac (Apple Silicon)
What version/commit are you on?
5885ba6
What database version are you on?
1
If you've built Reth from source, provide the full command you used
RUSTFLAGS="--cfg tokio_unstable" cargo install --locked --path bin/reth --bin reth --features tokio-console
Code of Conduct
The text was updated successfully, but these errors were encountered: