Jupyter Server Documents

This extension is composed of a Python package named jupyter_server_documents for the server extension and a NPM package named @jupyter/server-documents for the frontend extension.

Overview

Jupyter Server Documents is a powerful extension for JupyterLab that enhances your notebook experience by moving the state of documents to the server. Key features include:

Real-Time Collaboration

• New, optimized real-time collabortion engine for Jupyter Server
• Significant performance and memory usage improvements
• Proper handling of out-of-band changes (e.g. git checkout)
• No data loss or duplication

Enhanced Performance

• Optimized memory usage through intelligent output management
• Significantly faster loading of notebooks with large outputs
• Reduced memory footprint for notebooks with extensive output data
• Automatic saving of changes to prevent data loss

Improved Kernel Management

• More reliable kernel execution and connection handling
• Enhanced stability with better error recovery mechanisms
• Streamlined message handling between kernels and the frontend
• Improved kernel restart and reconnection capabilities

Efficient Output Handling

• Smart output storage separates large outputs from notebook documents
• Output streaming controls prevent memory issues with unbounded outputs
• Faster notebook navigation even with extensive computational outputs
• Lazy loading of outputs for improved responsiveness

This extension provides the foundation for a more collaborative, stable, and performant JupyterLab experience, especially valuable for data scientists and researchers working with complex notebooks or in team environments.

Requirements

• JupyterLab >= 4.4.0

Installation

To install the extension, run:

pip install jupyter_server_documents

To remove the extension, run:

pip uninstall jupyter_server_documents

Contributing

Setting up a development environment

We recommend using Micromamba to create and manage a custom Python environment for developing this package. Micromamba provides most of the same CLI as Conda, but works much faster.

First, set up a development environment by running this command from the root of this repository:

micromamba env create -f dev-environment.yml

This creates a new Python environment named serverdocs and automatically installs this extension's build dependencies, required for local development. Then, activate the environment:

micromamba activate serverdocs

Activating an environment is required to access any Python packages installed in that environment. You should activate the environment before developing any changes to the jupyter_server_documents package locally.

Development install

After ensuring that the serverdocs environment is activated, you can install an editable copy of jupyter_server_documents into your environment by running the script below.

jlpm dev:install

Notes about the development installation:

• jlpm is JupyterLab's pinned version of yarn that is installed with JupyterLab. In other words, jlpm can be considered an alias of yarn.

• jlpm dev:install runs the dev:install NPM script defined in package.json.

• The dev:install step uses uv as a faster, more modern replacement for pip.

After completing this, you should have a working, editable copy of jupyter_server_documents in your environment. Run jupyter lab and open JupyterLab in a browser to verify that jupyter_server_documents is installed.

Development process

When making new changes to your local copy of jupyter_server_documents, different commands need to be run depending on the types of changes made. Without running these commands, the new changes are not reflected in JupyterLab.

Here is a summary of the commands to run after making changes:

• After updating package.json or yarn.lock: run jlpm install to install the frontend dependencies.

• After updating any frontend (TS/TSX/JS/CSS): run jlpm build to build the lab extension (i.e. the frontend).

• After updating any backend (Python) file: restart the server to reload the server extension (i.e. the backend).

  • Note that there is no build process when updating a Python file, since Python is a scripting language.

• After updating entry points or other package metadata in pyproject.toml: run jlpm dev:uninstall && jlpm dev:install to re-do the development installation. The package metadata is not updated automatically after local changes, even when installing the package in editable mode.

• Finally, refresh the JupyterLab page in the browser to load the new frontend assets and use the new backend.

Building on change (frontend only)

You can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension.

# Watch the source directory in one terminal, automatically rebuilding when needed
jlpm watch
# Run JupyterLab in another terminal
jupyter lab

With the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt).

By default, the jlpm build command generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command:

jupyter lab build --minimize=False

Note that the steps described here only update the application in response to changes to frontend (TypeScript) files. Changes to any backend (Python) files still require restarting the Jupyter Server.

Development uninstall

To undo the development installation, run this command:

jlpm dev:uninstall

In development mode, you will also need to remove the symlink created by jupyter labextension develop command. To find its location, you can run jupyter labextension list to figure out where the labextensions folder is located. Then you can remove the symlink named @jupyter/server-documents within that folder.

Testing the extension

Server tests

This extension is using Pytest for Python code testing.

Install test dependencies (needed only once):

pip install -e ".[test]"
# Each time you install the Python package, you need to restore the front-end extension link
jupyter labextension develop . --overwrite

To execute them, run:

pytest -vv -r ap --cov jupyter_server_documents

Frontend tests

This extension is using Jest for JavaScript code testing.

To execute them, execute:

jlpm
jlpm test

Integration tests

This extension uses Playwright for the integration tests (aka user level tests). More precisely, the JupyterLab helper Galata is used to handle testing the extension in JupyterLab.

More information are provided within the ui-tests README.

Packaging the extension

See RELEASE

Troubleshooting

If you are seeing the frontend extension, but it is not working, check that the server extension is enabled:

jupyter server extension list

If the server extension is installed and enabled, but you are not seeing the frontend extension, check the frontend extension is installed:

jupyter labextension list