[ISSUE] Group datasource should sort members, instance profiles and groups property for consistent state between runs

[niraj8241]

Terraform Version

Terraform v0.12.21
+ provider.databricks v0.2.9

Affected Resource(s)

Please list the resources as a list, for example:

• databricks_group_member

Terraform Configuration Files

variable "user_names" {
  type = list(string)
  default = ["test@example.com", "test1@example.com"]
}

resource "databricks_user" "user" {
  count     = length(var.user_names)
  user_name = sort(var.user_names)[count.index]
  display_name = var.user_names[count.index]
}

resource "databricks_group_member" "group_member" {
  count     = length(var.user_names)
  group_id   = var.group_id
  member_id  = databricks_user.user[count.index].id
}

Expected Behavior

Terraform should not re-create the groups on every run and should produce any changes.

Actual Behavior

After running for the first time. Everything works fine but re-running the plan, states that there will be changes and the group membership will be destroyed.
Output:

**First Run**
niraj@Workstation:~/repo/terraform-databricks$ terraform apply --auto-approve
databricks_user.user[1]: Creating...
databricks_group.group: Creating...
databricks_user.user[0]: Creating...
databricks_user.user[1]: Creation complete after 1s [id=5768733437510966]
databricks_user.user[0]: Creation complete after 1s [id=703188293121903]
databricks_group.group: Creation complete after 1s [id=4440206158560509]
data.databricks_group.group_data: Refreshing state...
databricks_group_member.vip_member[1]: Creating...
databricks_group_member.vip_member[0]: Creating...
databricks_group_member.vip_member[1]: Creation complete after 1s [id=4440206158560509|5768733437510966]
databricks_group_member.vip_member[0]: Creation complete after 1s [id=4440206158560509|703188293121903]

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

Outputs:

test = [
  "703188293121903",
  "5768733437510966",
]

**Second Run**
niraj@Workstation:~/repo/terraform-databricks$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

databricks_group.group: Refreshing state... [id=4440206158560509]
databricks_user.user[1]: Refreshing state... [id=5768733437510966]
databricks_user.user[0]: Refreshing state... [id=703188293121903]
databricks_group_member.vip_member[1]: Refreshing state... [id=4440206158560509|5768733437510966]
databricks_group_member.vip_member[0]: Refreshing state... [id=4440206158560509|703188293121903]

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place
-/+ destroy and then create replacement
 <= read (data resources)

Terraform will perform the following actions:

  # data.databricks_group.group_data will be read during apply
  # (config refers to values not yet known)
 <= data "databricks_group" "group_data"  {
      + allow_cluster_create       = (known after apply)
      + allow_instance_pool_create = (known after apply)
      + display_name               = "niraj-terraform-test"
      + groups                     = (known after apply)
      + id                         = (known after apply)
      + instance_profiles          = (known after apply)
      + members                    = (known after apply)
    }

  # databricks_group_member.vip_member[0] must be replaced
-/+ resource "databricks_group_member" "vip_member" {
      ~ group_id  = "4440206158560509" -> (known after apply) # forces replacement
      ~ id        = "4440206158560509|703188293121903" -> (known after apply)
        member_id = "703188293121903"
    }

  # databricks_group_member.vip_member[1] must be replaced
-/+ resource "databricks_group_member" "vip_member" {
      ~ group_id  = "4440206158560509" -> (known after apply) # forces replacement
      ~ id        = "4440206158560509|5768733437510966" -> (known after apply)
        member_id = "5768733437510966"
    }

  # databricks_user.user[0] will be updated in-place
  ~ resource "databricks_user" "user" {
        active                     = true
        allow_cluster_create       = false
        allow_instance_pool_create = false
      ~ display_name               = "test@example.com " -> "test@example.com"
        id                         = "703188293121903"
        user_name                  = "test@example.com"
    }

Plan: 2 to add, 1 to change, 2 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. Create a variable as list(string) for usernames
2. Create databricks_group_member resource by iterating over the variable
3. Re-run the plan.

[nfx]

Remove assignment for display name, as it generally should hold “First Last” type of name pair. Then rerun and it should work.

please also rename the issue accordingly

[nfx]

to those making a PR: this should be a stateDiffFunc

[niraj8241]

@nfx It seems like this doesn't resolve the issue. I tried removing the "Display Name" but it always re-creates the group membership.

[nfx]

@niraj8241 , could you post exact configuration you're using to reproduce this issue, as logs and provided configuration refer to different things.

From logs, it's evident, that you're using data "databricks_group". The bug is related to members not being sorted.

[nfx]

@niraj8241 released :)

[Anbukugan]

@nfx I am using provider 0.3.4 version. Still facing the same issue with data source reapply.

[nfx]

@Anbukugan use v0.4.9, v0.3.x is too old and no longer receiving any fixes.