Terraform unable to destroy and replace resources due to dependencies

[elizajanus-ally]

Terraform Version

v0.13.7

Affected Resource(s)

Please list the resources as a list, for example:

• pagerduty_schedule
• pagerduty_escalation_policy
• pagerduty_team_membership

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

locals {

  example_user1 = {
    id = "EXAMPLE"
    role = "observer"
  }

  example_user2 = {
    id = "EXAMPLE"
    role = "observer"
  }

  example_members = [
    local.example_user1,
    local.example_user2
  ]
}

resource "pagerduty_team" "c3" {
  name        = "example"
  description = "example pagerduty team"
}

resource "pagerduty_team_membership" "example_membership" {
  for_each = { for example_members in local.example_members : example_members.id => example_members }
  user_id = each.key
  team_id = pagerduty_team.example.id
  role    = each.value.role
}

resource "pagerduty_schedule" "example_p1_schedule" {
  name      = "example p1 Primary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-06-27T09:00:00.00Z"
    rotation_virtual_start       = "2022-06-27T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id
  }
}

resource "pagerduty_schedule" "example_p1_schedule_secondary" {
  name      = "example p1 Secondary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-07-04T09:00:00.00Z"
    rotation_virtual_start       = "2022-07-04T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id
  }
}

resource "pagerduty_schedule" "example_p2_schedule" {
  name      = "example p2 Primary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-06-27T09:00:00.00Z"
    rotation_virtual_start       = "2022-06-27T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id
  }
}

resource "pagerduty_schedule" "example_p2_schedule_secondary" {
  name      = "example p2 Secondary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-07-04T09:00:00.00Z"
    rotation_virtual_start       = "2022-07-04T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id
  }
}


resource "pagerduty_schedule" "example_p3_schedule" {
  name      = "example p3 Primary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-06-27T07:00:00.00Z"
    rotation_virtual_start       = "2022-06-27T07:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id
    
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 1
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 2
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 3
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 4
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 5
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
  }
}

resource "pagerduty_schedule" "example_p3_schedule_secondary" {
  name      = "example p3 Secondary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-07-04T07:00:00.00Z"
    rotation_virtual_start       = "2022-07-04T07:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id

    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 1
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 2
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 3
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 4
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 5
        start_time_of_day = "07:00:00"
        duration_seconds  = 46800
    }
  }
}

resource "pagerduty_schedule" "example_p4_schedule" {
  name      = "example p4 Primary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-06-27T09:00:00.00Z"
    rotation_virtual_start       = "2022-06-27T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id
  
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 1
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 2
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 3
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 4
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 5
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
  }
}

resource "pagerduty_schedule" "example_p4_schedule_secondary" {
  name      = "example p4 Secondary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-07-04T09:00:00.00Z"
    rotation_virtual_start       = "2022-07-04T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id
  
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 1
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 2
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 3
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 4
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 5
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
  }
}



resource "pagerduty_schedule" "example_p5_schedule" {
  name      = "example p5 Primary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-06-27T09:00:00.00Z"
    rotation_virtual_start       = "2022-06-27T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id

    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 1
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 2
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 3
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 4
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 5
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
  }
}

resource "pagerduty_schedule" "example_p5_schedule_secondary" {
  name      = "example p5 Secondary Schedule"
  time_zone = "America/New_York"
  teams     = [pagerduty_team.example.id]
  
  layer {
    name                         = "example schedule"
    start                        = "2022-07-04T09:00:00.00Z"
    rotation_virtual_start       = "2022-07-04T09:00:00.00Z"
    rotation_turn_length_seconds = 86400*7
    users                        = local.example_members[*].id

    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 1
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 2
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 3
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 4
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
    restriction {
        type              = "weekly_restriction"
        start_day_of_week = 5
        start_time_of_day = "09:00:00"
        duration_seconds  = 27000
    }
  }
}

locals {
  director = {
      name = "ExampleUser3",
      id = "EXAMPLE"
  }
  sr_director = {
      name = "ExampleUser4"
      id = "EXAMPLE"
  }  
}

resource "pagerduty_escalation_policy" "example_P1_policy" {
  name      = "example p1 policy"
  teams     = [pagerduty_team.example.id]

  rule {
    escalation_delay_in_minutes = 7

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p1_schedule.id
    }
  }
  rule {
    escalation_delay_in_minutes = 14

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p1_schedule.id
    }

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p1_schedule_secondary.id
    }
  }
  
  rule {
    escalation_delay_in_minutes = 18

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
  rule {
    escalation_delay_in_minutes = 21

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p1_schedule.id
    }

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p1_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 25

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
  rule {
    escalation_delay_in_minutes = 30

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p1_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p1_schedule_secondary.id
    }
    target {
      type = "user_reference"
      id   = local.director.id
    }
    target {
      type = "user_reference"
      id   = local.sr_director.id
    }
  }
}

resource "pagerduty_escalation_policy" "example_P2_policy" {
  name      = "example P2 policy"

  teams     = [pagerduty_team.example.id]
  rule {
    escalation_delay_in_minutes = 10

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p2_schedule.id
    }
  }
  rule {
    escalation_delay_in_minutes = 20

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p2_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p2_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 30

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
  rule {
    escalation_delay_in_minutes = 40

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p2_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p2_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 45

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
  rule {
    escalation_delay_in_minutes = 60

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p2_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p2_schedule_secondary.id
    }
    target {
      type = "user_reference"
      id   = local.director.id
    }
    target {
      type = "user_reference"
      id   = local.sr_director.id
    }
  }
}

resource "pagerduty_escalation_policy" "example_P3_policy" {
  name      = "example P3 policy"

  teams     = [pagerduty_team.example.id]
  rule {
    escalation_delay_in_minutes = 30

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p3_schedule.id
    }
  }
  rule {
    escalation_delay_in_minutes = 60

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p3_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p3_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 90

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
  rule {
    escalation_delay_in_minutes = 120

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p3_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p3_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 180

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
}
resource "pagerduty_escalation_policy" "example_P4_policy" {
  name      = "example P4 policy"

  teams     = [pagerduty_team.example.id]
  rule {
    escalation_delay_in_minutes = 60

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p4_schedule.id
    }
  }
  rule {
    escalation_delay_in_minutes = 120

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p4_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p4_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 240

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p4_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p4_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 300

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
}
resource "pagerduty_escalation_policy" "example_P5_policy" {
  name      = "example P5 policy"

  teams     = [pagerduty_team.example.id]
  rule {
    escalation_delay_in_minutes = 180

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p5_schedule.id
    }
  }
  rule {
    escalation_delay_in_minutes = 240

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p5_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p5_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 360

    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p5_schedule.id
    }
    target {
      type = "schedule_reference"
      id   = pagerduty_schedule.example_p5_schedule_secondary.id
    }
  }
  rule {
    escalation_delay_in_minutes = 600

    target {
      type = "user_reference"
      id   = local.director.id
    }
  }
}

Expected Behavior

When I significantly change a resource, such as a schedule, Terraform forces the deletion and recreation of that resource. I expect that when I change a resource and Terraform attempts to destroy it, that operation will succeed because the provider would be aware of dependencies within resources, and delete the resources in an order that allows the operation to work.

Actual Behavior

When I significantly alter a schedule and Terraform tries to delete it, Terraform fails to delete the resource because of dependencies with escalation policies. When I try to remove a user from a team, that modification also fails due to dependencies with escalation policies.

Errors when attempting to replace schedules in Terraform and removing a user from a team:

Error: DELETE API call to https://api.pagerduty.com/schedules/P4IO8IG failed 400 Bad Request. Code: 2001, Errors: [Schedule can't be deleted if it's being used by escalation policies], Message: Invalid Input Provided
Error: DELETE API call to https://api.pagerduty.com/schedules/PSD8NMJ failed 400 Bad Request. Code: 2001, Errors: [Schedule can't be deleted if it's being used by escalation policies], Message: Invalid Input Provided
Error: DELETE API call to https://api.pagerduty.com/schedules/PWPWOC9 failed 400 Bad Request. Code: 2001, Errors: [Schedule can't be deleted if it's being used by escalation policies], Message: Invalid Input Provided
Error: DELETE API call to https://api.pagerduty.com/teams/PJX7BMI/users/P01CAFE failed 400 Bad Request. Code: 2001, Errors: [User cannot be removed as they belong to an escalation policy on this team], Message: Invalid Input Provided
Error: DELETE API call to https://api.pagerduty.com/schedules/P8UZ9Y6 failed 400 Bad Request. Code: 2001, Errors: [Schedule can't be deleted if it's being used by escalation policies], Message: Invalid Input Provided```

### Steps to Reproduce
Please list the steps required to reproduce the issue, for example:
1. Modify pagerduty schedules significantly or modify the resource names. Remove a team member from team.
2. `terraform apply`


### References
Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:
- [#337](https://github.com/PagerDuty/terraform-provider-pagerduty/issues/337)

[imjaroiswebdev]

Hi @elizajanus-ally the change aiming to solve this issue with pagerduty_team_membership was merged in #558 and is available from the version v2.6.0 of the Terraform Provider. On top of that, the update that hopefully will solve the issue with destroying pagerduty_schedule have been pushed in #561. When this latter PR be merged this fix would be available in one of our next Terraform Provider release. Thank you for the example code, it was very helpful.

[imjaroiswebdev]

@elizajanus-ally An update for fixing the issue related with deleting Schedules being used by Escalation Policies was merged on #561 and It has been released in version v2.6.1 of PagerDuty TF Provider, so updating to the new version hopefully You all would be able to gracefully delete Schedules without getting stuck with into this error. Please if You find any further issues related with this error after updating don't hesitate on reopening this Issue ✌🏽

[drastawi]

@imjaroiswebdev there are still two use cases. one will be solved in #564 and one is when the schedule is the only remaining one in an escalation policy.

[deniojunior-hotmart]

Hi! I'm using provider version 2.11.0 and I'm still getting this error when trying to drop a user. I tries to delete the team_membership before deleting the user from the schedule, so it fails because the user belongs to a schedule.

I have tried to explicit add the dependency in the schedule resource:

resource "pagerduty_schedule" "primary" {
[...]
depends_on = [pagerduty_user.users, pagerduty_team_membership.team_membership]

But I'm still getting the error below

Error: DELETE API call to https://api.pagerduty.com/teams/P2A6JL1/users/PO21JXJ failed 400 Bad Request. Code: 2001, Errors: [User cannot be removed as they belong to an escalation policy on this team], Message: Invalid Input Provided

[bschaeffer]

Ping since experiencing the same thing as @deniojunior-hotmart.

[drastawi]

@imjaroiswebdev not sure if I should open a separate issue, but the same affects teams when you assign an escalation policy to a new team.

[runofthemill]

Running into the same issue as @deniojunior-hotmart in a terraform apply where two users are being deleted along with updates to remove them from schedules & escalation policies they belong to.

Despite adding explicit dependencies to the schedule & escalation policy resources, the user destroy action is attempted first and fails.

edit: @deniojunior-hotmart @bschaeffer I found a bit of a workaround by leveraging data "pagerduty_users" "all_users" {}, filtering out the users that were to be deleted, and using the resulting local as the user list to use in schedule/escalation policy resources