Added `control_run_state` flag to the `databricks_job` resource for continuous jobs #2466

mgyucht · 2023-07-06T13:37:58Z

Changes

This PR introduces a new flag, control_run_state, to replace the always_running flag. This flag only applies to continuous jobs. Its behavior is described below:

For jobs with pause_status = PAUSED, it is a no-op on create and stops the active job run on update (if applicable).
For jobs with pause_status = UNPAUSED, it starts a job run on create and stops the active job run on update (if applicable).

The job does not need to be started, as that is handled by the Jobs service itself.

This fixes #2130.

Todo:

Add tests

Tests

make test run locally
relevant change in docs/ folder
covered with integration tests in internal/acceptance
relevant acceptance tests are passing
using Go SDK

codecov-commenter · 2023-07-06T13:41:50Z

Codecov Report

Merging #2466 (ca5e84c) into master (0a23d5e) will increase coverage by 0.11%.
The diff coverage is 79.41%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2466      +/-   ##
==========================================
+ Coverage   88.13%   88.24%   +0.11%     
==========================================
  Files         144      145       +1     
  Lines       11917    11973      +56     
==========================================
+ Hits        10503    10566      +63     
+ Misses        942      925      -17     
- Partials      472      482      +10

Impacted Files	Coverage Δ
jobs/resource_job.go	`91.36% <75.86%> (-4.44%)`	⬇️
catalog/data_metastore.go	`100.00% <100.00%> (ø)`
provider/provider.go	`94.08% <100.00%> (+0.03%)`	⬆️

... and 1 file with indirect coverage changes

borremosch-db

This PR appears to use /jobs/runs/cancel to restart the currently active continuous run. As part of exponential backoff for continuous jobs, @kevin-thiebaut is changing the restart mechanism to use /jobs/run-now instead. @kevin-thiebaut could you weigh in to determine whether the approach in this PR still works after we introduce exponential backoff?

kevin-thiebaut · 2023-07-10T08:55:46Z

This PR appears to use /jobs/runs/cancel to restart the currently active continuous run. As part of exponential backoff for continuous jobs, @kevin-thiebaut is changing the restart mechanism to use /jobs/run-now instead. @kevin-thiebaut could you weigh in to determine whether the approach in this PR still works after we introduce exponential backoff?

From what I understand, the terraform provider will call jobs/update, then jobs/runs/cancel with the current implementation, is that right?
If so this approach would not work when the change to enable continuous jobs exponential backoff is rolled out because jobs/runs/cancel will not reset the exponential backoff sequence. This means that even if the active run is cancelled, the next run could start up to 280 minutes later.
The fix is to replace jobs/runs/cancel by jobs/run-now when exponential backoff is enabled as this endpoint will also cancel the active run, but also reset the exponential backoff sequence.
One thing to note is that /jobs/run-now is currently not allowed for continuous jobs (before the flag is enabled).

kevin-thiebaut

Looks good 👌

kevin-thiebaut · 2023-07-17T07:48:36Z

jobs/resource_job.go

+				if js.MaxConcurrentRuns > 1 {
+					return fmt.Errorf("`control_run_state` must be specified only with `max_concurrent_runs = 1`")
+				}


nit: It might be a slightly confusing error message there, as for continuous jobs, max_concurrent_runs is set to one and is not changeable.
I think it would be better to put it after the message about continuous being required.

This reverts commit 2721738.

alexott · 2023-07-21T07:29:23Z

docs/resources/job.md

+  When migrating from `always_running` to `control_run_state`, set `continuous` as follows:
+  ```
+  continuous {
+    pause_status = "UNPAUSED"


UNPAUSED is default, so you can simply do continuos {}

If not set remotely, it will default to unpaused, but if set remotely, it will not overwrite this field. I'll document it as you've said though

docs/resources/job.md

alexott · 2023-07-21T07:49:51Z

jobs/resource_job.go

+			Default:       false,
+			Type:          schema.TypeBool,
+			Deprecated:    "always_running will be replaced by control_run_state in the next major release.",
+			ConflictsWith: []string{"control_run_state"},


Should we add that it's also conflicting with continuos ?

Good point. I think that would be a breaking change right? Wouldn't we want to save that for a major version change, at which point we'll just remove the field entirely? Today, at least the job is restarted, even if TF provider fails if the job is continuous, but this would block the update in the first place.

if it leads to problems I think that it could be ok (we have both github issue & internal ES ticket because of using continuous with always_running)

alexott

lgtm

…ontinuous jobs (#2466) This PR introduces a new flag, control_run_state, to replace the always_running flag. This flag only applies to continuous jobs. Its behavior is described below: For jobs with pause_status = PAUSED, it is a no-op on create and stops the active job run on update (if applicable). For jobs with pause_status = UNPAUSED, it starts a job run on create and stops the active job run on update (if applicable). The job does not need to be started, as that is handled by the Jobs service itself. This fixes #2130.

* first draft * account client check * account client check * Fixed `databricks_service_principals` data source issue with empty filter (#2185) * fix `databricks_service_principals` data source issue with empty filter * fix acc tests * Allow rotating `token` block in `databricks_mws_workspaces` resource by only changing `comment` field (#2114) Tested manually for the following cases Without this PR the provider recreates the entire workspace on a token update With changes in this PR only the token is refreshed When both token and storage_configuration_id are changed then the entire workspace is recreated Additional unit tests also added that allow checks that patch workspace calls are not made when only token is changed Also added an integration test to check tokens are successfully updated * Excludes roles in scim API list calls to reduce load on databricks scim service (#2181) * Exclude roles in scim API list calls * more test fixes * Update SDK to v0.6.0 (#2186) * Update SDK to v0.6.0 * go mod tidy * update sdk to 0.7.0 * add integration tests * fix acceptance tests * fix tests * add account-level API support for `metastore_data_access` * add account API support for `databricks_storage_credential` * address feedback * refactor to `WorkspaceOrAccountRequest` * fix acceptance tests * Release v1.21.0 (#2471) Release v1.21.0 of the Terraform Provider for Databricks. ## Changes * Added condition_task to the [`databricks_job`](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource (private preview) ([#2459](#2459)). * Added `AccountData`, `AccountClient` and define generic databricks data utilites for defining workspace and account-level data sources ([#2429](#2429)). * Added documentation link to existing Databricks Terraform modules ([#2439](#2439)). * Added experimental compute field to [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource ([#2401](#2401)). * Added import example to doc for [databricks_group_member](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/group_member) resource ([#2453](#2453)). * Added support for subscriptions in dashboards & alert SQL tasks in [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) ([#2447](#2447)). * Fixed model serving integration test ([#2460](#2460), [#2461](#2461)). * Fixed [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource file arrival trigger parameter name ([#2438](#2438)). * Fixed catalog_workspace_binding_test ([#2463](#2463), [#2451](#2451)). No breaking changes in this release. * Install mlflow cluster using in model serving test if the cluster is already running (#2470) * Bump golang.org/x/mod from 0.11.0 to 0.12.0 (#2462) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.11.0 to 0.12.0. - [Commits](golang/mod@v0.11.0...v0.12.0) --- updated-dependencies: - dependency-name: golang.org/x/mod dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Exporter: make resource names more unique to avoid duplicate resources errors (#2452) This includes following changes: * Add user ID to the `databricks_user` resource name to avoid clashes on names like, `user+1` and `user_1` * Add user/sp/group ID to the name of the `databricks_group_member` resource * Remove too aggressive name normalization pattern that also leads to the generation of the duplicate resource names for different resources * Add documentation notes about legacy cluster type & data access (#2437) * Add documentation notes about legacy cluster type & data access * Update docs/resources/cluster.md Co-authored-by: Miles Yucht <miles@databricks.com> * Update docs/resources/mount.md Co-authored-by: Miles Yucht <miles@databricks.com> --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Use random catalog name in SQL table integration tests (#2473) The fixed value prevented concurrent integration test runs. * Link model serving docs to top level README (#2474) * Add one more item to the troubleshooting guide (#2477) It's related to use OAuth for authentication but not providing `account_id` in the provider configuration. * Added `databricks_access_control_rule_set` resource for managing account-level access (#2371) * Added `acl_principal_id` attribute to `databricks_user`, `databricks_group` & `databricks_service_principal` for easier use with `databricks_access_control_rule_set` (#2485) It should simplify specification of principals in the `databricks_access_control_rule_set` so instead of this (string with placeholders): ``` grant_rules { principals = ["groups/${databricks_group.ds.display_name}"] role = "roles/servicePrincipal.user" } ``` it will be simpler to refer like this: ``` grant_rules { principals = [databricks_group.ds.acl_principal_id] role = "roles/servicePrincipal.user" } ``` * Added support for Unity Catalog `databricks_metastores` data source (#2017) * add documentation for databricks_metastores data source * add API endpoint for listing metastores * add metastores data resource * add test for metastores data source * add metastores datasource to resource mapping * fix reference to wrong resource docs * add a Metastores struct for the response of the API, use this in the datSource * update terraform specific object attributes * add new data test * remove slice_set property from MetastoreData * use databricks-go-sdk for data_metastore.go * removed listMetastores endpoint since it's unused * make sure tests also use the unitycatalog.MetastoreInfo from the sdk * remove redundant resource * test -dev * fix * fmt * cleanup * Added AccountClient to DatabricksClient and AccountData * upd * cleanup * accountLevel * upd * add example * list * cleanup * docs * remove dead code * wip * use maps * upd * cleanup * comments * - * remove redundant test --------- Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: vuong-nguyen <44292934+nkvuong@users.noreply.github.com> * Added support for Unity Catalog `databricks_metastore` data source (#2492) Enable fetching account level metastore information through id for a single metastore. * Supported new Delve binary name format (#2497) https://github.com/go-delve/delve/blob/master/CHANGELOG.md#1210-2023-06-23 changes the naming of the delve debug binary. This PR changes isInDebug to accommodate old and new versions of Delve. * Add code owners for Terraform (#2498) * Removed unused dlvLoadConfig configuration from settings.json (#2499) * Fix provider after updating SDK to 0.13 (#2494) * Fix provider after updating SDK to 0.13 * add unit test * split test * Added `control_run_state` flag to the `databricks_job` resource for continuous jobs (#2466) This PR introduces a new flag, control_run_state, to replace the always_running flag. This flag only applies to continuous jobs. Its behavior is described below: For jobs with pause_status = PAUSED, it is a no-op on create and stops the active job run on update (if applicable). For jobs with pause_status = UNPAUSED, it starts a job run on create and stops the active job run on update (if applicable). The job does not need to be started, as that is handled by the Jobs service itself. This fixes #2130. * Added exporter for `databricks_workspace_file` resource (#2493) * Preliminary changes to make workspace files implementation - make `NotebooksAPI.List` to return directories as well when called in the recursive mode (same as non-recursive behavior) - Because of that, remove the separate `ListDirectories` - Extend `workspace.ObjectStatus` with additional fields (will be required for incremental notebooks export) - Cache listing of all workspace objects, and then use it for all operations - list notebooks, list directories, list workspace files * Added exporting of workspace files --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Supported boolean values in `databricks_sql_alert` alerts (#2506) * Added more common issues for troubleshooting (#2486) * add troubleshooting * fix doc category --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Fixed handling of comments in `databricks_sql_table` resource (#2472) * column comments and single quote escape * Delimiter collision avoidance table comment * compatible with user single quote escape * unit tests for parseComment * corrected fmt --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Added clarification that `databricks_schema` and `databricks_sql_table` should be imported by their full name, not just by name (#2491) Co-authored-by: Miles Yucht <miles@databricks.com> * Updated `databricks_user` with `force = true` to check for error message prefix (#2510) This fixes #2500 * fix force delete * remove orphaned code * fix acceptance tests * upgrade go sdk * fix metastoreinfo struct * docs update * fix acceptance tests * fix tests * updated docs * fix tests * rename test * update tests * fix tests * fix test * add state upgrader * fix struct * fix tests * feedback * feedback * fix acc test * fix test * fix test * fix test * feedback * fix acc tests * feedback --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com> Co-authored-by: Miles Yucht <miles@databricks.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Ott <alexey.ott@databricks.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Co-authored-by: Gautham Sunjay <gauthamsunjay17@gmail.com> Co-authored-by: guillesd <74136033+guillesd@users.noreply.github.com> Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: Tanmay Rustagi <88379306+tanmay-db@users.noreply.github.com> Co-authored-by: Fabian Jakobs <fabian.jakobs@databricks.com> Co-authored-by: klus <lus.karol@gmail.com>

* first draft * account client check * account client check * Fixed `databricks_service_principals` data source issue with empty filter (#2185) * fix `databricks_service_principals` data source issue with empty filter * fix acc tests * Allow rotating `token` block in `databricks_mws_workspaces` resource by only changing `comment` field (#2114) Tested manually for the following cases Without this PR the provider recreates the entire workspace on a token update With changes in this PR only the token is refreshed When both token and storage_configuration_id are changed then the entire workspace is recreated Additional unit tests also added that allow checks that patch workspace calls are not made when only token is changed Also added an integration test to check tokens are successfully updated * Excludes roles in scim API list calls to reduce load on databricks scim service (#2181) * Exclude roles in scim API list calls * more test fixes * Update SDK to v0.6.0 (#2186) * Update SDK to v0.6.0 * go mod tidy * update sdk to 0.7.0 * add integration tests * fix acceptance tests * fix tests * add account-level API support for `metastore_data_access` * add account API support for `databricks_storage_credential` * address feedback * refactor to `WorkspaceOrAccountRequest` * fix acceptance tests * Release v1.21.0 (#2471) Release v1.21.0 of the Terraform Provider for Databricks. * Added condition_task to the [`databricks_job`](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource (private preview) ([#2459](#2459)). * Added `AccountData`, `AccountClient` and define generic databricks data utilites for defining workspace and account-level data sources ([#2429](#2429)). * Added documentation link to existing Databricks Terraform modules ([#2439](#2439)). * Added experimental compute field to [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource ([#2401](#2401)). * Added import example to doc for [databricks_group_member](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/group_member) resource ([#2453](#2453)). * Added support for subscriptions in dashboards & alert SQL tasks in [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) ([#2447](#2447)). * Fixed model serving integration test ([#2460](#2460), [#2461](#2461)). * Fixed [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource file arrival trigger parameter name ([#2438](#2438)). * Fixed catalog_workspace_binding_test ([#2463](#2463), [#2451](#2451)). No breaking changes in this release. * Install mlflow cluster using in model serving test if the cluster is already running (#2470) * Bump golang.org/x/mod from 0.11.0 to 0.12.0 (#2462) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.11.0 to 0.12.0. - [Commits](golang/mod@v0.11.0...v0.12.0) --- updated-dependencies: - dependency-name: golang.org/x/mod dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Exporter: make resource names more unique to avoid duplicate resources errors (#2452) This includes following changes: * Add user ID to the `databricks_user` resource name to avoid clashes on names like, `user+1` and `user_1` * Add user/sp/group ID to the name of the `databricks_group_member` resource * Remove too aggressive name normalization pattern that also leads to the generation of the duplicate resource names for different resources * Add documentation notes about legacy cluster type & data access (#2437) * Add documentation notes about legacy cluster type & data access * Update docs/resources/cluster.md Co-authored-by: Miles Yucht <miles@databricks.com> * Update docs/resources/mount.md Co-authored-by: Miles Yucht <miles@databricks.com> --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Use random catalog name in SQL table integration tests (#2473) The fixed value prevented concurrent integration test runs. * Link model serving docs to top level README (#2474) * Add one more item to the troubleshooting guide (#2477) It's related to use OAuth for authentication but not providing `account_id` in the provider configuration. * Added `databricks_access_control_rule_set` resource for managing account-level access (#2371) * Added `acl_principal_id` attribute to `databricks_user`, `databricks_group` & `databricks_service_principal` for easier use with `databricks_access_control_rule_set` (#2485) It should simplify specification of principals in the `databricks_access_control_rule_set` so instead of this (string with placeholders): ``` grant_rules { principals = ["groups/${databricks_group.ds.display_name}"] role = "roles/servicePrincipal.user" } ``` it will be simpler to refer like this: ``` grant_rules { principals = [databricks_group.ds.acl_principal_id] role = "roles/servicePrincipal.user" } ``` * Added support for Unity Catalog `databricks_metastores` data source (#2017) * add documentation for databricks_metastores data source * add API endpoint for listing metastores * add metastores data resource * add test for metastores data source * add metastores datasource to resource mapping * fix reference to wrong resource docs * add a Metastores struct for the response of the API, use this in the datSource * update terraform specific object attributes * add new data test * remove slice_set property from MetastoreData * use databricks-go-sdk for data_metastore.go * removed listMetastores endpoint since it's unused * make sure tests also use the unitycatalog.MetastoreInfo from the sdk * remove redundant resource * test -dev * fix * fmt * cleanup * Added AccountClient to DatabricksClient and AccountData * upd * cleanup * accountLevel * upd * add example * list * cleanup * docs * remove dead code * wip * use maps * upd * cleanup * comments * - * remove redundant test --------- Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: vuong-nguyen <44292934+nkvuong@users.noreply.github.com> * Added support for Unity Catalog `databricks_metastore` data source (#2492) Enable fetching account level metastore information through id for a single metastore. * Supported new Delve binary name format (#2497) https://github.com/go-delve/delve/blob/master/CHANGELOG.md#1210-2023-06-23 changes the naming of the delve debug binary. This PR changes isInDebug to accommodate old and new versions of Delve. * Add code owners for Terraform (#2498) * Removed unused dlvLoadConfig configuration from settings.json (#2499) * Fix provider after updating SDK to 0.13 (#2494) * Fix provider after updating SDK to 0.13 * add unit test * split test * Added `control_run_state` flag to the `databricks_job` resource for continuous jobs (#2466) This PR introduces a new flag, control_run_state, to replace the always_running flag. This flag only applies to continuous jobs. Its behavior is described below: For jobs with pause_status = PAUSED, it is a no-op on create and stops the active job run on update (if applicable). For jobs with pause_status = UNPAUSED, it starts a job run on create and stops the active job run on update (if applicable). The job does not need to be started, as that is handled by the Jobs service itself. This fixes #2130. * Added exporter for `databricks_workspace_file` resource (#2493) * Preliminary changes to make workspace files implementation - make `NotebooksAPI.List` to return directories as well when called in the recursive mode (same as non-recursive behavior) - Because of that, remove the separate `ListDirectories` - Extend `workspace.ObjectStatus` with additional fields (will be required for incremental notebooks export) - Cache listing of all workspace objects, and then use it for all operations - list notebooks, list directories, list workspace files * Added exporting of workspace files --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Supported boolean values in `databricks_sql_alert` alerts (#2506) * Added more common issues for troubleshooting (#2486) * add troubleshooting * fix doc category --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Fixed handling of comments in `databricks_sql_table` resource (#2472) * column comments and single quote escape * Delimiter collision avoidance table comment * compatible with user single quote escape * unit tests for parseComment * corrected fmt --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Added clarification that `databricks_schema` and `databricks_sql_table` should be imported by their full name, not just by name (#2491) Co-authored-by: Miles Yucht <miles@databricks.com> * Updated `databricks_user` with `force = true` to check for error message prefix (#2510) This fixes #2500 * fix force delete * remove orphaned code * fix acceptance tests * upgrade go sdk * fix metastoreinfo struct * docs update * fix acceptance tests * fix tests * updated docs * fix tests * rename test * update tests * fix tests * fix test * add state upgrader * fix struct * fix tests * feedback * feedback * fix acc test * fix test * fix test * fix test * feedback * fix acc tests * feedback --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com> Co-authored-by: Miles Yucht <miles@databricks.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Ott <alexey.ott@databricks.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Co-authored-by: Gautham Sunjay <gauthamsunjay17@gmail.com> Co-authored-by: guillesd <74136033+guillesd@users.noreply.github.com> Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: Tanmay Rustagi <88379306+tanmay-db@users.noreply.github.com> Co-authored-by: Fabian Jakobs <fabian.jakobs@databricks.com> Co-authored-by: klus <lus.karol@gmail.com>

* first draft * account client check * account client check * Fixed `databricks_service_principals` data source issue with empty filter (#2185) * fix `databricks_service_principals` data source issue with empty filter * fix acc tests * Allow rotating `token` block in `databricks_mws_workspaces` resource by only changing `comment` field (#2114) Tested manually for the following cases Without this PR the provider recreates the entire workspace on a token update With changes in this PR only the token is refreshed When both token and storage_configuration_id are changed then the entire workspace is recreated Additional unit tests also added that allow checks that patch workspace calls are not made when only token is changed Also added an integration test to check tokens are successfully updated * Excludes roles in scim API list calls to reduce load on databricks scim service (#2181) * Exclude roles in scim API list calls * more test fixes * Update SDK to v0.6.0 (#2186) * Update SDK to v0.6.0 * go mod tidy * update sdk to 0.7.0 * add integration tests * fix acceptance tests * fix tests * add account-level API support for `metastore_data_access` * add account API support for `databricks_storage_credential` * address feedback * refactor to `WorkspaceOrAccountRequest` * fix acceptance tests * Release v1.21.0 (#2471) Release v1.21.0 of the Terraform Provider for Databricks. ## Changes * Added condition_task to the [`databricks_job`](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource (private preview) ([#2459](#2459)). * Added `AccountData`, `AccountClient` and define generic databricks data utilites for defining workspace and account-level data sources ([#2429](#2429)). * Added documentation link to existing Databricks Terraform modules ([#2439](#2439)). * Added experimental compute field to [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource ([#2401](#2401)). * Added import example to doc for [databricks_group_member](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/group_member) resource ([#2453](#2453)). * Added support for subscriptions in dashboards & alert SQL tasks in [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) ([#2447](#2447)). * Fixed model serving integration test ([#2460](#2460), [#2461](#2461)). * Fixed [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource file arrival trigger parameter name ([#2438](#2438)). * Fixed catalog_workspace_binding_test ([#2463](#2463), [#2451](#2451)). No breaking changes in this release. * Install mlflow cluster using in model serving test if the cluster is already running (#2470) * Bump golang.org/x/mod from 0.11.0 to 0.12.0 (#2462) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.11.0 to 0.12.0. - [Commits](golang/mod@v0.11.0...v0.12.0) --- updated-dependencies: - dependency-name: golang.org/x/mod dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Exporter: make resource names more unique to avoid duplicate resources errors (#2452) This includes following changes: * Add user ID to the `databricks_user` resource name to avoid clashes on names like, `user+1` and `user_1` * Add user/sp/group ID to the name of the `databricks_group_member` resource * Remove too aggressive name normalization pattern that also leads to the generation of the duplicate resource names for different resources * Add documentation notes about legacy cluster type & data access (#2437) * Add documentation notes about legacy cluster type & data access * Update docs/resources/cluster.md Co-authored-by: Miles Yucht <miles@databricks.com> * Update docs/resources/mount.md Co-authored-by: Miles Yucht <miles@databricks.com> --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Use random catalog name in SQL table integration tests (#2473) The fixed value prevented concurrent integration test runs. * Link model serving docs to top level README (#2474) * Add one more item to the troubleshooting guide (#2477) It's related to use OAuth for authentication but not providing `account_id` in the provider configuration. * Added `databricks_access_control_rule_set` resource for managing account-level access (#2371) * Added `acl_principal_id` attribute to `databricks_user`, `databricks_group` & `databricks_service_principal` for easier use with `databricks_access_control_rule_set` (#2485) It should simplify specification of principals in the `databricks_access_control_rule_set` so instead of this (string with placeholders): ``` grant_rules { principals = ["groups/${databricks_group.ds.display_name}"] role = "roles/servicePrincipal.user" } ``` it will be simpler to refer like this: ``` grant_rules { principals = [databricks_group.ds.acl_principal_id] role = "roles/servicePrincipal.user" } ``` * Added support for Unity Catalog `databricks_metastores` data source (#2017) * add documentation for databricks_metastores data source * add API endpoint for listing metastores * add metastores data resource * add test for metastores data source * add metastores datasource to resource mapping * fix reference to wrong resource docs * add a Metastores struct for the response of the API, use this in the datSource * update terraform specific object attributes * add new data test * remove slice_set property from MetastoreData * use databricks-go-sdk for data_metastore.go * removed listMetastores endpoint since it's unused * make sure tests also use the unitycatalog.MetastoreInfo from the sdk * remove redundant resource * test -dev * fix * fmt * cleanup * Added AccountClient to DatabricksClient and AccountData * upd * cleanup * accountLevel * upd * add example * list * cleanup * docs * remove dead code * wip * use maps * upd * cleanup * comments * - * remove redundant test --------- Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: vuong-nguyen <44292934+nkvuong@users.noreply.github.com> * Added support for Unity Catalog `databricks_metastore` data source (#2492) Enable fetching account level metastore information through id for a single metastore. * Supported new Delve binary name format (#2497) https://github.com/go-delve/delve/blob/master/CHANGELOG.md#1210-2023-06-23 changes the naming of the delve debug binary. This PR changes isInDebug to accommodate old and new versions of Delve. * Add code owners for Terraform (#2498) * Removed unused dlvLoadConfig configuration from settings.json (#2499) * Fix provider after updating SDK to 0.13 (#2494) * Fix provider after updating SDK to 0.13 * add unit test * split test * Added `control_run_state` flag to the `databricks_job` resource for continuous jobs (#2466) This PR introduces a new flag, control_run_state, to replace the always_running flag. This flag only applies to continuous jobs. Its behavior is described below: For jobs with pause_status = PAUSED, it is a no-op on create and stops the active job run on update (if applicable). For jobs with pause_status = UNPAUSED, it starts a job run on create and stops the active job run on update (if applicable). The job does not need to be started, as that is handled by the Jobs service itself. This fixes #2130. * Added exporter for `databricks_workspace_file` resource (#2493) * Preliminary changes to make workspace files implementation - make `NotebooksAPI.List` to return directories as well when called in the recursive mode (same as non-recursive behavior) - Because of that, remove the separate `ListDirectories` - Extend `workspace.ObjectStatus` with additional fields (will be required for incremental notebooks export) - Cache listing of all workspace objects, and then use it for all operations - list notebooks, list directories, list workspace files * Added exporting of workspace files --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Supported boolean values in `databricks_sql_alert` alerts (#2506) * Added more common issues for troubleshooting (#2486) * add troubleshooting * fix doc category --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Fixed handling of comments in `databricks_sql_table` resource (#2472) * column comments and single quote escape * Delimiter collision avoidance table comment * compatible with user single quote escape * unit tests for parseComment * corrected fmt --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Added clarification that `databricks_schema` and `databricks_sql_table` should be imported by their full name, not just by name (#2491) Co-authored-by: Miles Yucht <miles@databricks.com> * Updated `databricks_user` with `force = true` to check for error message prefix (#2510) This fixes #2500 * fix force delete * remove orphaned code * fix acceptance tests * upgrade go sdk * fix metastoreinfo struct * docs update * fix acceptance tests * fix tests * updated docs * fix tests * rename test * update tests * fix tests * fix test * add state upgrader * fix struct * fix tests * feedback * feedback * fix acc test * fix test * fix test * fix test * feedback * fix acc tests * feedback --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com> Co-authored-by: Miles Yucht <miles@databricks.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Ott <alexey.ott@databricks.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Co-authored-by: Gautham Sunjay <gauthamsunjay17@gmail.com> Co-authored-by: guillesd <74136033+guillesd@users.noreply.github.com> Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: Tanmay Rustagi <88379306+tanmay-db@users.noreply.github.com> Co-authored-by: Fabian Jakobs <fabian.jakobs@databricks.com> Co-authored-by: klus <lus.karol@gmail.com>

* refactor to support account-level ip acl * add doc * add acceptance tests * Refactor `databricks_schema` to Go SDK (#2572) * refactor `databricks_schema` to Go SDK * clean up * Refactor `databricks_external_location` to Go SDK (#2546) * refactor external location to Go SDK * keep force_new * switch struct to sdk * Refresh `databricks_grants` with latest permissible grants (#2567) * refresh latest grants * fix test * fix typo * add test * Fixed databricks_access_control_rule_set integration test in Azure (#2591) * Update Go SDK to v0.17.0 (#2599) * Update Go SDK to 0.17.0 * go mod tidy * RunJobTask job_id type fix (#2588) * fix run job id bug * update tests * update exporter tests * Add `databricks_connection` resource to support Lakehouse Federation (#2528) * first draft * add foreign catalog * update doc * Fixed `databricks_job` resource to clear instance-specific attributes when `instance_pool_id` is specified (#2507) NodeTypeID cannot be set in jobsAPI.Update() if InstancePoolID is specified. If both are specified, assume InstancePoolID takes precedence and NodeTypeID is only computed. Closes #2502. Closes #2141. * Added `full_refresh` attribute to the `pipeline_task` in `databricks_job` (#2444) This allows to force full refresh of the pipeline from the job. This fixes #2362 * Configured merge queue for the provider (#2533) * misc doc updates (#2516) * Bump github.com/databricks/databricks-sdk-go from 0.13.0 to 0.14.1 (#2523) Bumps [github.com/databricks/databricks-sdk-go](https://github.com/databricks/databricks-sdk-go) from 0.13.0 to 0.14.1. - [Release notes](https://github.com/databricks/databricks-sdk-go/releases) - [Changelog](https://github.com/databricks/databricks-sdk-go/blob/main/CHANGELOG.md) - [Commits](databricks/databricks-sdk-go@v0.13.0...v0.14.1) --- updated-dependencies: - dependency-name: github.com/databricks/databricks-sdk-go dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Miles Yucht <miles@databricks.com> * Fix IP ACL read (#2515) * Add support for `USE_MARKETPLACE_ASSETS` privilege to metastore (#2505) * Update docs to include USE_MARKETPLACE_ASSETS privilege * Add USE_MARKETPLACE_ASSETS to metastore privileges * Add git job_source to job resource (#2538) * Add git job_source to job resource * lint * fix test * Use go sdk type * Allow search SQL Warehouses by name in `databricks_sql_warehouse` data source (#2458) * Allow search SQL Warehouses by name in `databricks_sql_warehouse` data source Right now it's possible to search only by the warehouse ID, but it's not always convenient although it's possible by using `databricks_sql_warehouses` data source + explicit filtering. This PR adds a capability to search by either SQL warehouse name or ID. This fixes #2443 * Update docs/data-sources/sql_warehouse.md Co-authored-by: Miles Yucht <miles@databricks.com> * Address review comments also change documentation a bit to better match the data source - it was copied from the resource as-is. * More fixes from review * code review comments --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Late jobs support (aka health conditions) in `databricks_job` resource (#2496) * Late jobs support (aka health conditions) in `databricks_job` resource Added support for `health` block that is used to detect late jobs. Also, this PR includes following changes: * Added `on_duration_warning_threshold_exceeded` attribute to email & webhook notifications (needed for late jobs support) * Added `notification_settings` on a task level & use jobs & task notification structs from Go SDK * Reorganized documentation for task block as it's getting more & more attributes * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Update docs/resources/job.md Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * address review comments * add list of tasks * more review chanes --------- Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> Co-authored-by: Miles Yucht <miles@databricks.com> * feedback * update struct * add suppress diff * fix suppress diff * fix acceptance tests * test feedback * make id a pair * better sensitive options handling * reorder id pair --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: marekbrysa <53767523+marekbrysa@users.noreply.github.com> Co-authored-by: Alex Ott <alexey.ott@databricks.com> Co-authored-by: Miles Yucht <miles@databricks.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: bvdboom <bvdboom@users.noreply.github.com> Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> * Documentation changes (#2576) * add troubleshooting guide for grants/permissions config drifts * update authentication in index * feedback * Exporter: Incremental export of notebooks, SQL objects and some other resources (#2563) * First pass on incremental export of notebooks/files & SQL objects * use optional modified_at field * Add `-incremental` & `-update-since` command-line options * Incremental generation of `import.sh` and `vars.tf` * Incremental export of the TF resources themselves * Added incremental support for the rest of the objects also updated documentation & compatibility matrix * Add tests for incremental generation * Add clarification about periodic full export * Store last run timestamp in the file on disk and use with `-incremental` * Address initial review comments * Incremental export of MLflow Webhooks, expanded tests to cover merge of variables * fix test * Fix reflection method marshallJSON for CMK in mws workspace (#2605) * Fix reflection method marshallJSON for CMK in mws workspace * Add UT * Add missing documentation for CMK support on GCP (#2604) * add CMK on GCP to docs * feedback * Add `owner` parameter to `databricks_share` resource (#2594) * add `owner` parameter to `databricks_share` * suppress diff * Exporter: command-line option to control output format for notebooks (#2569) New command-line option `-notebooksFormat` allows to export notebooks in DBC and IPython formats. This fixes #2568 * Fix creation of views with comments using `databricks_sql_table` (#2589) * mark column type as omitempty * add acc test * escape names for sql * add test & suppress_diff * fix tests * fix acc test * update doc * fix acc tests * fix doc * Add account-level API support for Unity Catalog objects (#2182) * first draft * account client check * account client check * Fixed `databricks_service_principals` data source issue with empty filter (#2185) * fix `databricks_service_principals` data source issue with empty filter * fix acc tests * Allow rotating `token` block in `databricks_mws_workspaces` resource by only changing `comment` field (#2114) Tested manually for the following cases Without this PR the provider recreates the entire workspace on a token update With changes in this PR only the token is refreshed When both token and storage_configuration_id are changed then the entire workspace is recreated Additional unit tests also added that allow checks that patch workspace calls are not made when only token is changed Also added an integration test to check tokens are successfully updated * Excludes roles in scim API list calls to reduce load on databricks scim service (#2181) * Exclude roles in scim API list calls * more test fixes * Update SDK to v0.6.0 (#2186) * Update SDK to v0.6.0 * go mod tidy * update sdk to 0.7.0 * add integration tests * fix acceptance tests * fix tests * add account-level API support for `metastore_data_access` * add account API support for `databricks_storage_credential` * address feedback * refactor to `WorkspaceOrAccountRequest` * fix acceptance tests * Release v1.21.0 (#2471) Release v1.21.0 of the Terraform Provider for Databricks. * Added condition_task to the [`databricks_job`](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource (private preview) ([#2459](#2459)). * Added `AccountData`, `AccountClient` and define generic databricks data utilites for defining workspace and account-level data sources ([#2429](#2429)). * Added documentation link to existing Databricks Terraform modules ([#2439](#2439)). * Added experimental compute field to [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource ([#2401](#2401)). * Added import example to doc for [databricks_group_member](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/group_member) resource ([#2453](#2453)). * Added support for subscriptions in dashboards & alert SQL tasks in [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) ([#2447](#2447)). * Fixed model serving integration test ([#2460](#2460), [#2461](#2461)). * Fixed [databricks_job](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/job) resource file arrival trigger parameter name ([#2438](#2438)). * Fixed catalog_workspace_binding_test ([#2463](#2463), [#2451](#2451)). No breaking changes in this release. * Install mlflow cluster using in model serving test if the cluster is already running (#2470) * Bump golang.org/x/mod from 0.11.0 to 0.12.0 (#2462) Bumps [golang.org/x/mod](https://github.com/golang/mod) from 0.11.0 to 0.12.0. - [Commits](golang/mod@v0.11.0...v0.12.0) --- updated-dependencies: - dependency-name: golang.org/x/mod dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Exporter: make resource names more unique to avoid duplicate resources errors (#2452) This includes following changes: * Add user ID to the `databricks_user` resource name to avoid clashes on names like, `user+1` and `user_1` * Add user/sp/group ID to the name of the `databricks_group_member` resource * Remove too aggressive name normalization pattern that also leads to the generation of the duplicate resource names for different resources * Add documentation notes about legacy cluster type & data access (#2437) * Add documentation notes about legacy cluster type & data access * Update docs/resources/cluster.md Co-authored-by: Miles Yucht <miles@databricks.com> * Update docs/resources/mount.md Co-authored-by: Miles Yucht <miles@databricks.com> --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Use random catalog name in SQL table integration tests (#2473) The fixed value prevented concurrent integration test runs. * Link model serving docs to top level README (#2474) * Add one more item to the troubleshooting guide (#2477) It's related to use OAuth for authentication but not providing `account_id` in the provider configuration. * Added `databricks_access_control_rule_set` resource for managing account-level access (#2371) * Added `acl_principal_id` attribute to `databricks_user`, `databricks_group` & `databricks_service_principal` for easier use with `databricks_access_control_rule_set` (#2485) It should simplify specification of principals in the `databricks_access_control_rule_set` so instead of this (string with placeholders): ``` grant_rules { principals = ["groups/${databricks_group.ds.display_name}"] role = "roles/servicePrincipal.user" } ``` it will be simpler to refer like this: ``` grant_rules { principals = [databricks_group.ds.acl_principal_id] role = "roles/servicePrincipal.user" } ``` * Added support for Unity Catalog `databricks_metastores` data source (#2017) * add documentation for databricks_metastores data source * add API endpoint for listing metastores * add metastores data resource * add test for metastores data source * add metastores datasource to resource mapping * fix reference to wrong resource docs * add a Metastores struct for the response of the API, use this in the datSource * update terraform specific object attributes * add new data test * remove slice_set property from MetastoreData * use databricks-go-sdk for data_metastore.go * removed listMetastores endpoint since it's unused * make sure tests also use the unitycatalog.MetastoreInfo from the sdk * remove redundant resource * test -dev * fix * fmt * cleanup * Added AccountClient to DatabricksClient and AccountData * upd * cleanup * accountLevel * upd * add example * list * cleanup * docs * remove dead code * wip * use maps * upd * cleanup * comments * - * remove redundant test --------- Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: vuong-nguyen <44292934+nkvuong@users.noreply.github.com> * Added support for Unity Catalog `databricks_metastore` data source (#2492) Enable fetching account level metastore information through id for a single metastore. * Supported new Delve binary name format (#2497) https://github.com/go-delve/delve/blob/master/CHANGELOG.md#1210-2023-06-23 changes the naming of the delve debug binary. This PR changes isInDebug to accommodate old and new versions of Delve. * Add code owners for Terraform (#2498) * Removed unused dlvLoadConfig configuration from settings.json (#2499) * Fix provider after updating SDK to 0.13 (#2494) * Fix provider after updating SDK to 0.13 * add unit test * split test * Added `control_run_state` flag to the `databricks_job` resource for continuous jobs (#2466) This PR introduces a new flag, control_run_state, to replace the always_running flag. This flag only applies to continuous jobs. Its behavior is described below: For jobs with pause_status = PAUSED, it is a no-op on create and stops the active job run on update (if applicable). For jobs with pause_status = UNPAUSED, it starts a job run on create and stops the active job run on update (if applicable). The job does not need to be started, as that is handled by the Jobs service itself. This fixes #2130. * Added exporter for `databricks_workspace_file` resource (#2493) * Preliminary changes to make workspace files implementation - make `NotebooksAPI.List` to return directories as well when called in the recursive mode (same as non-recursive behavior) - Because of that, remove the separate `ListDirectories` - Extend `workspace.ObjectStatus` with additional fields (will be required for incremental notebooks export) - Cache listing of all workspace objects, and then use it for all operations - list notebooks, list directories, list workspace files * Added exporting of workspace files --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Supported boolean values in `databricks_sql_alert` alerts (#2506) * Added more common issues for troubleshooting (#2486) * add troubleshooting * fix doc category --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Fixed handling of comments in `databricks_sql_table` resource (#2472) * column comments and single quote escape * Delimiter collision avoidance table comment * compatible with user single quote escape * unit tests for parseComment * corrected fmt --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Added clarification that `databricks_schema` and `databricks_sql_table` should be imported by their full name, not just by name (#2491) Co-authored-by: Miles Yucht <miles@databricks.com> * Updated `databricks_user` with `force = true` to check for error message prefix (#2510) This fixes #2500 * fix force delete * remove orphaned code * fix acceptance tests * upgrade go sdk * fix metastoreinfo struct * docs update * fix acceptance tests * fix tests * updated docs * fix tests * rename test * update tests * fix tests * fix test * add state upgrader * fix struct * fix tests * feedback * feedback * fix acc test * fix test * fix test * fix test * feedback * fix acc tests * feedback --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com> Co-authored-by: Miles Yucht <miles@databricks.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Alex Ott <alexey.ott@databricks.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Co-authored-by: Gautham Sunjay <gauthamsunjay17@gmail.com> Co-authored-by: guillesd <74136033+guillesd@users.noreply.github.com> Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: Tanmay Rustagi <88379306+tanmay-db@users.noreply.github.com> Co-authored-by: Fabian Jakobs <fabian.jakobs@databricks.com> Co-authored-by: klus <lus.karol@gmail.com> * Fix UC acceptance test (#2613) * fix acc test * remove deprecated field from sdk * Release v1.24.0 (#2614) * Release v1.24.0 * Release v1.24.0 * Release v1.24.0 * Release v1.24.0 * Fixed verification of workspace reachability by using scim/me which is always available (#2618) * add flag to skip verification * cleanup * cleanup * - * test * test * test * test * - * - * Release v1.24.1 (#2625) * Release v1.24.1 * go upd * new line * Add doc strings for ResourceFixtures (#2633) * Add doc strings for ResourceFixtures * fmt * Update qa/testing.go Co-authored-by: Miles Yucht <miles@databricks.com> * Update qa/testing.go Co-authored-by: Miles Yucht <miles@databricks.com> --------- Co-authored-by: Miles Yucht <miles@databricks.com> * Bump github.com/hashicorp/hcl/v2 from 2.17.0 to 2.18.0 (#2636) Bumps [github.com/hashicorp/hcl/v2](https://github.com/hashicorp/hcl) from 2.17.0 to 2.18.0. - [Release notes](https://github.com/hashicorp/hcl/releases) - [Changelog](https://github.com/hashicorp/hcl/blob/main/CHANGELOG.md) - [Commits](hashicorp/hcl@v2.17.0...v2.18.0) --- updated-dependencies: - dependency-name: github.com/hashicorp/hcl/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * terrafmt; updated share and share recipient docs (#2641) * update documentation (#2644) * fix response * remove preview path * rename test * fix create call * add wait for acceptance tests * fix test * feedback * use golang struct * add wait time for acc test --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Miles Yucht <miles@databricks.com> Co-authored-by: Krishna Swaroop K <krishna.swaroop@databricks.com> Co-authored-by: marekbrysa <53767523+marekbrysa@users.noreply.github.com> Co-authored-by: Alex Ott <alexey.ott@databricks.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: bvdboom <bvdboom@users.noreply.github.com> Co-authored-by: shreyas-goenka <88374338+shreyas-goenka@users.noreply.github.com> Co-authored-by: Gabor Ratky <gabor.ratky@databricks.com> Co-authored-by: Rohan Kabra <rohan.kabra@databricks.com> Co-authored-by: Serge Smertin <259697+nfx@users.noreply.github.com> Co-authored-by: Pieter Noordhuis <pieter.noordhuis@databricks.com> Co-authored-by: Gautham Sunjay <gauthamsunjay17@gmail.com> Co-authored-by: guillesd <74136033+guillesd@users.noreply.github.com> Co-authored-by: Tanmay Rustagi <tanmay.rustagi@databricks.com> Co-authored-by: Tanmay Rustagi <88379306+tanmay-db@users.noreply.github.com> Co-authored-by: Fabian Jakobs <fabian.jakobs@databricks.com> Co-authored-by: klus <lus.karol@gmail.com> Co-authored-by: Oleh Mykolaishyn <owlleg6@gmail.com>

mgyucht added 4 commits July 5, 2023 12:21

first work

2ff58b0

Merge branch 'master' into control-run-state

97dfb29

Docs and tweaks

b74b205

fix test

fad6f2f

mgyucht requested review from alexott and borremosch-db July 6, 2023 13:38

borremosch-db reviewed Jul 10, 2023

View reviewed changes

borremosch-db requested a review from kevin-thiebaut July 10, 2023 07:40

mgyucht added 5 commits July 12, 2023 14:24

Merge branch 'master' into control-run-state

7833027

Address feedback

40517f9

tests

9afa614

Merge branch 'master' into control-run-state

4cd0c3e

more tests

2233a4f

kevin-thiebaut approved these changes Jul 17, 2023

View reviewed changes

mgyucht added 8 commits July 20, 2023 10:52

Add integration test

0305c38

Merge branch 'master' into control-run-state

f3bce0d

address comment

ef651b0

Support new delve binary name format

2721738

fix unit test

3595ef7

Revert "Support new delve binary name format"

ffb48c4

This reverts commit 2721738.

Merge branch 'master' into control-run-state

13f90ca

Merge branch 'master' into control-run-state

f2cc98f

mgyucht requested review from a team as code owners July 20, 2023 14:09

alexott reviewed Jul 21, 2023

View reviewed changes

mgyucht added 3 commits July 21, 2023 13:09

Handle paused continuous jobs

581dfb8

Only use RunNow for UNPAUSED jobs

ca5e84c

Merge branch 'master' into control-run-state

6ace4f2

alexott approved these changes Jul 21, 2023

View reviewed changes

mgyucht merged commit ce26b1c into master Jul 21, 2023

mgyucht deleted the control-run-state branch July 21, 2023 13:38

mgyucht mentioned this pull request Aug 1, 2023

Release v1.22.0 #2544

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added `control_run_state` flag to the `databricks_job` resource for continuous jobs #2466

Added `control_run_state` flag to the `databricks_job` resource for continuous jobs #2466

mgyucht commented Jul 6, 2023 •

edited

Loading

codecov-commenter commented Jul 6, 2023 •

edited

Loading

borremosch-db left a comment

kevin-thiebaut commented Jul 10, 2023 •

edited

Loading

kevin-thiebaut left a comment

kevin-thiebaut Jul 17, 2023

alexott Jul 21, 2023

mgyucht Jul 21, 2023

alexott Jul 21, 2023

mgyucht Jul 21, 2023

alexott Jul 21, 2023

alexott left a comment

Added control_run_state flag to the databricks_job resource for continuous jobs #2466

Added control_run_state flag to the databricks_job resource for continuous jobs #2466

Conversation

mgyucht commented Jul 6, 2023 • edited Loading

Changes

Tests

codecov-commenter commented Jul 6, 2023 • edited Loading

Codecov Report

borremosch-db left a comment

Choose a reason for hiding this comment

kevin-thiebaut commented Jul 10, 2023 • edited Loading

kevin-thiebaut left a comment

Choose a reason for hiding this comment

kevin-thiebaut Jul 17, 2023

Choose a reason for hiding this comment

alexott Jul 21, 2023

Choose a reason for hiding this comment

mgyucht Jul 21, 2023

Choose a reason for hiding this comment

alexott Jul 21, 2023

Choose a reason for hiding this comment

mgyucht Jul 21, 2023

Choose a reason for hiding this comment

alexott Jul 21, 2023

Choose a reason for hiding this comment

alexott left a comment

Choose a reason for hiding this comment

Added `control_run_state` flag to the `databricks_job` resource for continuous jobs #2466

Added `control_run_state` flag to the `databricks_job` resource for continuous jobs #2466

mgyucht commented Jul 6, 2023 •

edited

Loading

codecov-commenter commented Jul 6, 2023 •

edited

Loading

kevin-thiebaut commented Jul 10, 2023 •

edited

Loading