[ENH] - include conda env activation for kernel with custom environment

[creative-resort]

What docker image(s) is this feature applicable to?

base-notebook

What changes are you proposing?

I'd like to find out, how to best integrate the activation of a custom environment upon kernel launch.

Problem, that needs this improvement:

When the kernel in the custom environment is started from JupyterLab, the conda environment is unchanged the "base" environment, where Jupyter is installed and run from, because the .bashrc with its conda environment activation never runs in that context. This means, that the PATH environment variable is also the one of the "base" environment, even if the python executable is from the custom env … resulting in modules, that are not purely Python, but properly installed in the custom env, to fail loading.

If I do a conda custom env activation e.g. via shell script, that I call via the kernelspec, the PATH is correct and loading those modules, that previously failed (e.g. plotly kaleido), works.

Questions:

1. How should changes to the kernelspec, that ipykernel install creates, be implemented optimally, in the Dockerfile?
2. Is providing and calling a kernel-launch.sh script a good idea, or should just the standard command be augmented with conda activate customenv , instead?

"argv": [
    "~/.jupyter/lab/launch-kernel.sh",
    "custom-env",
    "{connection_file}"
 ],

This is the reference to the custom env kernel creation:

docker-stacks/docs/using/recipes.md

Lines 88 to 102 in b378681

	RUN mamba create --yes -p "${CONDA_DIR}/envs/${conda_env}" python=${py_ver} ipython ipykernel && \
	mamba clean --all -f -y
	# alternatively, you can comment out the lines above and uncomment those below
	# if you'd prefer to use a YAML file present in the docker build context
	# COPY --chown=${NB_UID}:${NB_GID} environment.yml "/home/${NB_USER}/tmp/"
	# RUN cd "/home/${NB_USER}/tmp/" && \
	# mamba env create -p "${CONDA_DIR}/envs/${conda_env}" -f environment.yml && \
	# mamba clean --all -f -y
	# create Python kernel and link it to jupyter
	RUN "${CONDA_DIR}/envs/${conda_env}/bin/python" -m ipykernel install --user --name="${conda_env}" && \
	fix-permissions "${CONDA_DIR}" && \
	fix-permissions "/home/${NB_USER}"

How does this affect the user?

The user would be able to make use of conda modules, that use the full spectrum of conda environments, right away, without having to wonder, why additional binaries can't be found and without manually manipulating the PATH environment variable or absolute paths.

Anything else?

No response

[bribroder]

Sounds like you could use nb_conda_kernels
https://github.com/Anaconda-Platform/nb_conda_kernels

[mathbunnyru]

@creative-resort sorry for the really late answer.
I might have a few solutions here (they are untested though).

As you told, we have a recipe for using custom environment.
https://github.com/jupyter/docker-stacks/blob/main/docs/using/recipe_code/custom_environment.dockerfile

It has RUN echo "conda activate ${env_name}" >> "${HOME}/.bashrc".
Unfortunately, .bashrc is not run by jupyter, that's why environment is not changed.

1. First way of solving this would be to add -l to the start-notebook.sh script. You can easily change the first line of this script in your inherited image.
2. We have a thing called Startup hooks. You can create an exectutable activate_custom_env.sh file in /usr/local/bin/before-notebook.d/ with the following content (conda activate ${env_name}). It should activate your custom environment right before launching notebook.

Please, tell me if this works for you.