A defined interface for working with a cache of jupyter notebooks.
If you have a number of notebooks whose execution outputs you want to ensure are kept up to date, without having to re-execute them every time (particularly for long running code, or text-based formats that do not store the outputs).
The notebooks must have deterministic execution outputs:
- You use the same environment to run them (e.g. the same installed packages)
- They run no non-deterministic code (e.g. random numbers)
- They do not depend on external resources (e.g. files or network connections) that change over time
For example, it is utilised by jupyter-book, to allow for fast document re-builds.
pip install jupyter-cacheFor development:
git clone https://github.com/executablebooks/jupyter-cache
cd jupyter-cache
git checkout develop
pip install -e .[cli,code_style,testing]See the documentation for usage.
Some desired requirements (not yet all implemented):
- Persistent
- Separates out "edits to content" from "edits to code cells". Cell rearranges and code cell changes should require a re-execution. Content changes should not.
- Allow parallel access to notebooks (for execution)
- Store execution statistics/reports
- Store external assets: Notebooks being executed often require external assets: importing scripts/data/etc. These are prepared by the users.
- Store execution artefacts: created during execution
- A transparent and robust cache invalidation: imagine the user updating an external dependency or a Python module, or checking out a different git branch.
jupyter-cache follows the Executable Book Contribution Guide. We'd love your help!
Code style is tested using flake8,
with the configuration set in .flake8,
and code formatted with black.
Installing with jupyter-cache[code_style] makes the pre-commit
package available, which will ensure this style is met before commits are submitted, by reformatting the code
and testing for lint errors.
It can be setup by:
>> cd jupyter-cache
>> pre-commit installOptionally you can run black and flake8 separately:
>> black .
>> flake8 .Editors like VS Code also have automatic code reformat utilities, which can adhere to this standard.