DM-45079: Exclude notebooks if required services are not provisioned #359
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This got bigger than expected, as usual. I'm glad to break it up into separate PRs for subsets of the commits.
Don't run notebooks in environments that don't have the necessary services
Putting the required services in the notebook metadata can be done in JupyterLab, though is not the nicest UX for this. We now have an in-repo config file (
mobu.yaml
), maybe it would be better to keep a manifest in there? I like having it closer to the notebook though. I'm open to other suggestions.Consolidate all notebook filtering logic to the business
... instead of being split/duplicated in the business and the CI worker.
I'd like to mark the check run as neutral in this case, but I think it can wait until another PR.
There are a few ways to do it:
SolitaryResult
. The more I think about this, given this notebook runner is the only case of this third type of result from a solitary execution (aside from 'success' or 'failure'), the less opposed I am to do it...NotebookRunner
. I don't love this, because:NotebookRunner
outside of the filtering that causes a wacky situation like thisNotebookRunner
return aSolitaryNeutralResult
. This is probably the right way to do it, but it would add a lot of changes to this already-big-ish PR.Another related thing would be putting execution logs in the github check details. I haven't done this so far because I'm worried that this would make it easy to accidentally leak some kind of sensitive info.
Testing
exclude_dirs
overrides flock exclude_dirs