Thread to monitor the kernel usage (CPU, Memory...)

[echarles]

While implementing a jupyterlab extension to monitor the kernel usage (https://github.com/Quansight/jupyterlab-kernel-usage), a few PRs have been merged to use psutil to return Memory, CPU usage of the machine and the kernel processes.

• Add usage_request and usage_reply based on psutil #805
• Add hostname to the usage reply #865
• Add the PID to the resource usage reply #908

This has worked great for most of the metrics, but we hit now an issue for the kernel CPU reported on Quansight/jupyterlab-kernel-usage#11. psutil returns the CPU based on the last call, so it looks like everytime we ask the CPU via the kernel websocket, it return zero as psutil considers it is the first time the request is made, see info on:

• https://psutil.readthedocs.io/en/latest/#psutil.cpu_percent
• https://stackoverflow.com/questions/24367424/cpu-percentinterval-none-always-returns-0-regardless-of-interval-value-python/27229967#27229967

Please note that similar issue is reported on the jupyter-resource-usage extension (which monitors the server usage, not the kernel usage) jupyter-server/jupyter-resource-usage#107

A solution would be to maintain a separated thread that would be responsible to return that CPU in a correct way based on psutil.

What would be the impact to create such as separated thread? What are the already existing threads? Does it sound like a reasonable approach? Any other idea?

[blink1073]

What are the already existing threads?

It looks like the existing threads are heartbeat, iopub, and control.

Jupyter Client has threads for iopub, shell, stdin, control, and heartbeat.

I think it is worth trying to add a another thread, but maybe this is a good opportunity to add a performance regression check. SciPy uses airspeed velocity.

[echarles]

It looks like the existing threads are heartbeat, iopub, and control.

Thx @blink1073 Great to know ipykernel is already kind with thread hosting. I guess an additional one monitor, or maybe telemetry (thinking about the discussion in jupyter-server jupyter-server/jupyter_server#780 - maybe offtopic here, not sure?) could be added.

Jupyter Client has threads for iopub, shell, stdin, control, and heartbeat.

if we add a thread in ipykernel to monitor the resources, does that imply we ned also a thread in jupyter-client. My first intake was no, but I may miss context.

I think it is worth trying to add a another thread,

+1

but maybe this is a good opportunity to add a performance regression check. SciPy uses airspeed velocity.

It would be good. I guess that is not a small task to get that performance benchmarking setup.

[blink1073]

does that imply we need also a thread in jupyter-client

I don't believe so, since client is launching ipykernel in a subprocess

I guess that is not a small task to get that performance benchmarking setup.

I don't think it needs to be too comprehensive to start.

[dimpase]

Adding psutil as a dependency makes it impossible to install ipython on Cygwin. (psutil refused to merge @embray's patch for Cygwin in https://github.com/embray/psutil).

[dimpase]

@mkoeppe