[ISSUE] Resource databricks_permissions doesn't handle error on cluster terminated 30+ days ago

[rafa-am]

Similar to issue #1177 (involving databricks_cluster resource). Impact on terraform projects containing unpinned clusters that have been terminated 30+ days ago.

When applying access control policy to the cluster using resource "databricks_permissions" "cluster_usage", it will return the following error message:

Error: Cannot access cluster <cluster_id> that was terminated or unpinned more than 30 days ago.

Configuration

resource "databricks_permissions" "cluster_usage" {
  cluster_id = databricks_cluster.cluster.cluster_id

  dynamic "access_control" {
    for_each = local.owners
    content {
      user_name        = trimspace(access_control.value)
      permission_level = "CAN_MANAGE"
    }
  }

  access_control {
    group_name       = "Ambassador Data Science (Clusters Managers)"
    permission_level = "CAN_MANAGE"
  }

  access_control {
    group_name       = "Data Engineers"
    permission_level = "CAN_MANAGE"
  }
}

Expected Behavior

As same as databricks_cluster resource when handling to clusters terminated 30+ days ago, e.g., the provider should probably treat that specific error message as a "valid action" when the command terraform plan is ran, and address just a warning about cluster state.

Actual Behavior

An access cluster error raises and breaks the pipeline.

Steps to Reproduce

1. Have a terraform project with an unpinned cluster terminated 30+ days ago and thus deleted.
2. terraform apply

Terraform and provider versions

• Terraform: v1.0.3
• Databricks provider: 0.5.4

Important Factoids

Related to [ISSUE] Project with cluster terminated 30+ days ago made unusable. #1177

[nfx]

@rafa-am This is more concrete description of the issue, thank you. Please attach debug logs with responses from permissions api (/api/2.0/permissions/clusters/...). This is essential to fixing this issue.

Please also create ticket with Databricks Support and ask to involve IAM team.

Will be able to pick this up in the second half of April. For now, please use terraform cli to remove affected resources from the state or just taint them.

[nfx]

Apparently, HTTP 400 error returned by permissions API, not just clusters API.

The fix for this should involve copying "wrapMissingError" from clusters Get api to getting list of permissions api.

[dhohengassner]

Thanks for bringing this up 👍

I also get hit by this for different environments.
Is there any fix being worked on? Any good workarounds?

Thanks!

[nfx]

You should create a support ticket. And good symptomatic fix is to rm permissions resource from state or taint it. See links in related threads.