Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE] Issue with Users folder and databricks_notebook resource on databricks_user object #1777

Closed
rohitnijhawan opened this issue Nov 17, 2022 · 1 comment · Fixed by #1890
Assignees
Labels
exporter TF configuration generator

Comments

@rohitnijhawan
Copy link
Contributor

Configuration

resource "databricks_notebook" "users_<REDACTED>_ag_com_test_1566736564850376" {
  source = "${path.module}/notebooks/Users/<REDACTED>@ag.com/test/yodlee_demographics_2.py"
  path   = "/Users/<REDACTED>@ag.com/test/yodlee_demographics_2"
}

Expected Behavior

Notebook should be created in databricks workspace under the user's directory and subdirectory tree

Actual Behavior

Databricks provider's user (engineer) reports that they see errors:

Error: cannot create notebook: Folder Users is protected
with databricks_notebook.users_ag_com_test_1566736564850376,
on notebooks.tf line 8233, in resource "databricks_notebook" "users
ag_com_test_1566736564850376":
8233: resource "databricks_notebook" "users
_ag_com_test_1566736564850376" {

Problems occur when a notebook is not in the user's root path
this works ie: /Users/johnsmith@databricks.com/MyExporterNotebook.py
this does not ie: /Users/johnsmith@databricks.com/exporter/MyExporterNotebook.py

Steps to Reproduce

  1. Run exporter on a workspace with user directories, subdirectories and notebooks
  2. Run import (terraform init, terraform plan, terraform apply) off exporter's output into a new or empty workspace

Important notes

Adding a

depends_on = [databricks_user.johnsmith_ag_com] 

to the notebook resource fixes the problem

@rohitnijhawan rohitnijhawan added the exporter TF configuration generator label Nov 17, 2022
@rohitnijhawan
Copy link
Contributor Author

We have tested this fix and it works by editing the output from the exporter
for a given notebook if you get this output

resource "databricks_notebook" "users_userx1_ag_com_test_1566736564850376" {
  source = "${path.module}/notebooks/Users/userx1@ag.com/test/yodlee_demographics_2.py"
  path   = "/Users/userx1@ag.com/test/yodlee_demographics_2"
}

change that to

resource "databricks_notebook" "users_userx1_ag_com_test_1566736564850376" {
  source = "${path.module}/notebooks/Users/userx1@ag.com/test/yodlee_demographics_2.py"
  ----> path   = "/Users/${databricks_user.userx1_ag_com.user_name}/test/yodlee_demographics_2"
}

alexott added a commit that referenced this issue Jan 2, 2023
There are cases when reference isn't a full match of a given value but a prefix of it. For
example, user's notebook has a path of `/Users/user@domain/notebook`.  To make it working
correctly we need to emit user or service principal, and then make a correct reference
with dependency on emitted user or notebook, like,
`${databricks_user.user_domain.home}/notebook`.

This PR adds a new field to dependency specification: `MatchType` that may have following values:

- `prefix` - value of attribute of resource is a prefix of a given attribute
- `exact` (or empty string) - must have exact matching (previous behaviour)

this fixes #1777
alexott added a commit that referenced this issue Jan 3, 2023
There are cases when reference isn't a full match of a given value but a prefix of it. For
example, user's notebook has a path of `/Users/user@domain/notebook`.  To make it working
correctly we need to emit user or service principal, and then make a correct reference
with dependency on emitted user or notebook, like,
`${databricks_user.user_domain.home}/notebook`.

This PR adds a new field to dependency specification: `MatchType` that may have following values:

- `prefix` - value of attribute of resource is a prefix of a given attribute
- `exact` (or empty string) - must have exact matching (previous behaviour)

this fixes #1777
@nfx nfx closed this as completed in #1890 Jan 4, 2023
nfx pushed a commit that referenced this issue Jan 4, 2023
* Exporter: generate references that are matching only to a prefix

There are cases when reference isn't a full match of a given value but a prefix of it. For
example, user's notebook has a path of `/Users/user@domain/notebook`.  To make it working
correctly we need to emit user or service principal, and then make a correct reference
with dependency on emitted user or notebook, like,
`${databricks_user.user_domain.home}/notebook`.

This PR adds a new field to dependency specification: `MatchType` that may have following values:

- `prefix` - value of attribute of resource is a prefix of a given attribute
- `exact` (or empty string) - must have exact matching (previous behaviour)

this fixes #1777

* improving test coverage & emit user/sp for repos

* Address PR feedback
michael-berk pushed a commit to michael-berk/terraform-provider-databricks that referenced this issue Feb 15, 2023
…abricks#1890)

* Exporter: generate references that are matching only to a prefix

There are cases when reference isn't a full match of a given value but a prefix of it. For
example, user's notebook has a path of `/Users/user@domain/notebook`.  To make it working
correctly we need to emit user or service principal, and then make a correct reference
with dependency on emitted user or notebook, like,
`${databricks_user.user_domain.home}/notebook`.

This PR adds a new field to dependency specification: `MatchType` that may have following values:

- `prefix` - value of attribute of resource is a prefix of a given attribute
- `exact` (or empty string) - must have exact matching (previous behaviour)

this fixes databricks#1777

* improving test coverage & emit user/sp for repos

* Address PR feedback
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exporter TF configuration generator
Projects
None yet
2 participants