Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metrics for worker communications #6892

Closed
hendrikmakait opened this issue Aug 16, 2022 · 1 comment · Fixed by #6936
Closed

Add metrics for worker communications #6892

hendrikmakait opened this issue Aug 16, 2022 · 1 comment · Fixed by #6936
Assignees
Labels
diagnostics enhancement Improve existing functionality or make things work better

Comments

@hendrikmakait
Copy link
Member

To better understand worker communications, their performance and required resources, we should track more metrics. Specifically, it would be interesting to track the number of open incoming/outgoing connections and comm_nbytes in Worker.get_metrics. This could help debug issues such as #6208 or coiled/benchmarks#229 (comment) and help take decisions on possible solutions for those (#6208 (comment))

@hendrikmakait hendrikmakait added enhancement Improve existing functionality or make things work better diagnostics labels Aug 16, 2022
@fjetter
Copy link
Member

fjetter commented Aug 17, 2022

If we go down this road I believe it would be interesting to add comm_nbytes (or a derivative of some sort) to the dashboard. I suspect that unmanaged memory is often actual communication overhead and it would be nice if we could separate this visually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
diagnostics enhancement Improve existing functionality or make things work better
Projects
None yet
2 participants