add support for sparse checkouts in the `databricks_repo` #1869

alexott · 2022-12-21T17:57:11Z

No description provided.

xiaodongw-db

LGTM, I'm not familiar with Terraform so I'll leave others to approve it.

xiaodongw-db · 2022-12-21T19:00:04Z

docs/resources/repo.md

+
+* `patterns` - array of paths (directories) that will be used for sparse checkout.  List of patterns could be updated in-place.
+
+Addition or removal of the `sparse_checkout` configuration block will lead to recreation of the repo.


I assume this is the specific behavior of Terraform? Our API doesn't have this behavior.

Our API gives error when you're trying to add sparse_checkout to the existing repository - I double checked this. I'm happy to change it if you confirm that it shouldn't get an error.

also, if you do first with sparse_checkout and then remove that block, it won't restore the full checkout, although won't give you an error

The current API behavior is:

You cannot enable/disable sparse checkout after a repository is created, so if you create a repository without sparse checkout, setting sparse_checkout in Update request will fail.

If you specify sparse_checkout in Update request, it'll update the patterns to the new value; otherwise the sparse checkout setting won't be changed.

Thanks for pointing it out, I'll update the API doc to make it clearer.

then yes, Terraform behaviour is matching these rules you can update list of the patterns for repo with sparse_checkout, but you can add or remove this block to the existing repository - it will be recreated with the new settings.

# Version changelog ### 1.8.0 * Added support for GCP (Public Preview) with state upgraders for [databricks_mws_workspaces](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/mws_workspaces) ([#1871](#1871), [#1886](#1886), [#1748](#1748), [#1879](#1879)). * Added documentation for [databricks_permission_assignment](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/permission_assignment) ([#1880](#1880)). * Added `azure_attributes` to clusters in [databricks_pipeline](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/pipeline) resource ([#1854](#1854)). * Added [databricks_instance_pool](https://registry.terraform.io/providers/databricks/databricks/latest/docs/data-sources/instance_pool) data source ([#1907](#1907)). * Added [databricks_provider](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/provider) resource for Delta Sharing ([#1572](#1572)). * Added acceptance tests for [databricks_cluster](https://registry.terraform.io/providers/databricks/databricks/latest/docs/data-sources/cluster) data and [databricks_sql_endpoint](https://registry.terraform.io/providers/databricks/databricks/latest/docs/data-sources/sql_endpoint) data ([#1882](#1882)). * Added search by name for [databricks_cluster](https://registry.terraform.io/providers/databricks/databricks/latest/docs/data-sources/cluster) data source ([#1901](#1901)). * Added support for sparse checkouts in the [databricks_repo](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/repo) ([#1869](#1869)). * Added `config_reference` attribute to [databricks_secret](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/secret) for easier reference from Spark configuration ([#1898](#1898)). * Added `datarbicks_directory` data source ([#1902](#1902)). * Extended [databricks_catalog](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/catalog) resource to support Delta Sharing Catalog ([#1887](#1887)). * Improved [databricks_metastore_assignment](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/metastore_assignment) resource ([#1900](#1900)). * Improved [databricks_grants](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/grants) error messages ([#1888](#1888)). * Improved [databricks_group](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/group) documentation ([#1816](#1816)). * Improved `data_security_mode` documentation for [databricks_cluster](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/cluster) ([#1830](#1830)). * Replace the incorrect `id` attribute with the correct `sp_id` in service principal data source docs example ([#1873](#1873)). * Use NVMe disk size for selection in the [databricks_node_type](https://registry.terraform.io/providers/databricks/databricks/latest/docs/data-sources/node_type) data source ([#1856](#1856)). * Allowed to override resource ID for specific data source ([#1885](#1885)). Experimental Resource Exporter * Added command-line flag to export all users and service principals ([#1893](#1893)). * Added emit for secret scopes from Spark Conf and Env Vars ([#1897](#1897)). * Fixed issue with exporting of resources with simple name ([#1891](#1891)). * Generate references that are matching only to a prefix ([#1890](#1890)). * Updated list of supported services and associated resources ([#1894](#1894)). * Use [databricks_group_role](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/group_role) instead of deprecated [databricks_group_instance_profile](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/group_instance_profile) ([#1905](#1905)). * Use `dbfs_path` as reference for init scripts in clusters ([#1892](#1892)). * Logging improvements ([#1895](#1895)). Updated dependency versions: * Bump google.golang.org/api from 0.104.0 to 0.105.0 ([#1863](#1863)).

…ks#1869)

add support for sparse checkouts in the databricks_repo

48f5972

alexott requested review from nfx, xiaodongw-db and a team December 21, 2022 17:57

xiaodongw-db reviewed Dec 21, 2022

View reviewed changes

nfx approved these changes Dec 22, 2022

View reviewed changes

nfx merged commit 1e16efd into master Dec 22, 2022

nfx deleted the repos-sparse-checkout branch December 22, 2022 09:42

nfx mentioned this pull request Jan 10, 2023

Release v1.8.0 #1915

Merged

michael-berk pushed a commit to michael-berk/terraform-provider-databricks that referenced this pull request Feb 15, 2023

Added support for sparse checkouts in the databricks_repo (databric…

8db1f09

…ks#1869)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for sparse checkouts in the `databricks_repo` #1869

add support for sparse checkouts in the `databricks_repo` #1869

alexott commented Dec 21, 2022

xiaodongw-db left a comment

xiaodongw-db Dec 21, 2022

alexott Dec 21, 2022

alexott Dec 21, 2022 •

edited

Loading

xiaodongw-db Dec 21, 2022

alexott Dec 21, 2022


		* `patterns` - array of paths (directories) that will be used for sparse checkout. List of patterns could be updated in-place.

		Addition or removal of the `sparse_checkout` configuration block will lead to recreation of the repo.

add support for sparse checkouts in the databricks_repo #1869

add support for sparse checkouts in the databricks_repo #1869

Conversation

alexott commented Dec 21, 2022

xiaodongw-db left a comment

Choose a reason for hiding this comment

xiaodongw-db Dec 21, 2022

Choose a reason for hiding this comment

alexott Dec 21, 2022

Choose a reason for hiding this comment

alexott Dec 21, 2022 • edited Loading

Choose a reason for hiding this comment

xiaodongw-db Dec 21, 2022

Choose a reason for hiding this comment

alexott Dec 21, 2022

Choose a reason for hiding this comment

add support for sparse checkouts in the `databricks_repo` #1869

add support for sparse checkouts in the `databricks_repo` #1869

alexott Dec 21, 2022 •

edited

Loading