Skip to content

Conversation

@deitch
Copy link
Collaborator

@deitch deitch commented Sep 19, 2025

Adds two commits.

First, adds ability to set a cache dir and a bundles dir. Each runtime can report a cache dir and, if set, will be mounted into the container. Similarly, it can report a bundles dir, which also will be mounted.

These are particularly useful for onnxruntime, where it creates a bundle from onnx, and then runs the bundle. Compiling the onnx to the bundle is by far the majority of time to get running (sometimes >90%). Those artifacts are cached in a given directory, but if it is ephemeral inside the container, it is lost. That means each run after the first takes up to ten times as long as it should be until it can start inference.

Second, for simplicity sake, adds a nekko run --dry-run option, so it can print out what it will run in docker run and then exit.

Signed-off-by: Avi Deitcher <avi@deitcher.net>
Signed-off-by: Avi Deitcher <avi@deitcher.net>
@deitch deitch force-pushed the onnxruntime-capture-artifacts branch from 676d83d to 8e1df34 Compare September 19, 2025 08:11
@deitch deitch merged commit 2e36ee0 into main Sep 19, 2025
2 checks passed
@deitch deitch deleted the onnxruntime-capture-artifacts branch September 19, 2025 08:15
@deitch deitch mentioned this pull request Sep 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants