-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add validation mutator for volume artifact_path
#2050
Conversation
arifact_path
artifact_path
CreateVolumeRequestContent: &catalog.CreateVolumeRequestContent{ | ||
CatalogName: "catalogN", | ||
Name: "volumeN", | ||
VolumeType: "MANAGED", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the VolumeType
necessary for this test?
// Fast mutators with only in-memory checks | ||
JobClusterKeyDefined(), | ||
JobTaskClusterSpec(), | ||
SingleNodeCluster(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do all of these return errors?
Warnings currently only show up at the end of deploy, so they're not actionable (yet).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, SingleNodeCluster()
returns a warning that is useful even with a deployment. The others, though indeed, are no longer useful.
I can follow up with a PR that serializes the warnings (and below like INFO) on a rolling basis in the bundle.Seq
mutator. That should make the warnings useful before deployment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The others, though indeed, are no longer useful.
How so?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to revisit how we output these diags anyway, so no need to address here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How so?
You'd end up with runtime errors on JobClusterKeyDefined
. The validation warning has a runtime error equivalent:
.venv➜ bundle-playground git:(master) ✗ databricks bundle validate
Warning: job_cluster_key c1 is not defined
at resources.jobs.foo.tasks[0].job_cluster_key
in databricks.yml:17:28
Error: terraform apply: exit status 1
Error: cannot create job: Job cluster 'c1' is not defined in field 'job_clusters'.
with databricks_job.foo,
on bundle.tf.json line 35, in resource.databricks_job.foo:
35: }
JobTaskClusterSpec
returns errors though so it's useful for returning an error early.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should probably promote this warning to an error though.
If integration tests don't run automatically, an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
Bundles: * Fix finding Python within virtualenv on Windows ([#2034](#2034)). * Include missing field descriptions in JSON schema ([#2045](#2045)). * Add validation for volume referenced from `artifact_path` ([#2050](#2050)). * Handle `${workspace.file_path}` references in source-linked deployments ([#2046](#2046)). * Set the write bit for files written during template initialization ([#2068](#2068)).
Bundles: * Fix finding Python within virtualenv on Windows ([#2034](#2034)). * Include missing field descriptions in JSON schema ([#2045](#2045)). * Add validation for volume referenced from `artifact_path` ([#2050](#2050)). * Handle `${workspace.file_path}` references in source-linked deployments ([#2046](#2046)). * Set the write bit for files written during template initialization ([#2068](#2068)).
Changes
This PR:
workspace.artifact_path
does not exist.bundle validate
andbundle deploy
compared to before on just deployments.bundle deploy
, which earlier were only run onbundle validate
.Tests
Unit tests and manually. Also, existing integration tests provide coverage (
TestUploadArtifactToVolumeNotYetDeployed
,TestUploadArtifactFileToVolumeThatDoesNotExist
)Examples:
and