Suggestion: Kernel Doctor - notebook for diagnosing kernel problems

[rchiodo]

@kieferrm had this suggestion (it's been used before by other teams).

When a kernel startup fails, open up something (maybe a notebook) that the user can use to diagnose the problem themselves.

Examples might include:

• Run the exact python command we just ran that failed (so the user can see the explicit output)
• Run commands that generate the pip list and allow it to be sent to us
• Run commands that pip uninstall/install stuff (if we know this might fix the problem, like with the zmq.constants failure)

Basically a self help wizard.

[rebornix]

maybe a notebook

I love this idea a lot. It can open a special jupyter notebook with a diagnostic kernel and users can read the guidances (markdown cells) and run code cells with troubleshooting scripts.

[DonJayamanne]

Things we should solve

• Make it easy to file a bug if they failed to diagnose themselves
  • With all information of everything they tried,
  • Else they end up going through all of these steps again, that will be a frustrating workflow
• ipykernel & other dependency installation fail
  • Conda activation commands are incorrect, hence users are blocked.
• Detect missing, outdated, incompatible pacakges/dependencies
• Detect kernel startup failures & analyse errors & unblock user or give them more context
• Detect kernel crashes (inspect error message & unblock user or give them more context)
• Provide ability for user to unblock themselves (by installing/uninstalling necessary packages or the like, e.g. rename files that override builtins)
• Detect and list incompatible packages/conda versions/python versions and the like & provide links for more information
  • These are things that can be detected, and we should build this database of incompatible Python/conda/package versions, etc
• Detect file names that could override other packages.
• Provide ability to run user code in a controlled manner so as to narrow down the issue
  • Helps us determine if its a package or the like that's the culprit or version of Python etc..

Diagnosing:

• Can user run the same code in the terminal via ipython console

Logging
* Env information
* Has workspace folder
* Whether we have successfully completed each step (selecting right kernel, detecting dependencies, getting env variables, starting kernel, setting up initial state such as cwd, etc), then running a simple cell
* Sometimes thing could fall over when importing packages that end up messing with the python env.
* List python packages
* List conda packages
* Test interrupting, test restarting (capture logs of each)
* Whether we were able to get activation commands or not
* Output from the kernel (sometimes kernels comms crash and the only way to determine is this by inspecting the stdout/stderr of kernel processes)
* Output from the Jupyter server (sometimes the kernels don't respond at all and we can see the fact that we're re-trying in there), if kernels don't respond errors are in the stdout/stderr or jupyter server, but we don't get this information from the jupyterlab API)

[DonJayamanne]

Closing this as I don't think this is required anymore, if we do need such a thing then we can re-visit the requirements