azuredevops_serviceendpoint_azurecr failing to destroy

[bwesolowskicf]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and Azure DevOps Provider) Version

Terraform v1.9.0
microsoft/azuredevops v1.2.0

Affected Resource(s)

• azuredevops_serviceendpoint_azurecr

Terraform Configuration Files

resource "azurerm_user_assigned_identity" "devops_acr" {
  location            = data.azurerm_resource_group.identity.location
  name                = "identity-shared-devops-acr-${var.app_short_name}"
  resource_group_name = data.azurerm_resource_group.identity.name
}

resource "azuredevops_serviceendpoint_azurecr" "acr" {
  project_id                             = local.azuredevops_project.id
  service_endpoint_name                  = "DOCKER-ACR-${var.app_short_name}"
  resource_group                         = "RG"
  service_endpoint_authentication_scheme = "WorkloadIdentityFederation"
  azurecr_spn_tenantid                   = "TENANT_ID"
  azurecr_name                           = "ACRNAME"
  azurecr_subscription_id                = "SUB_ID"
  azurecr_subscription_name              = "SUB_NAME"
  credentials {
    serviceprincipalid = azurerm_user_assigned_identity.devops_acr.client_id
  }
}

resource "azurerm_federated_identity_credential" "acr_identity_credential" {
  depends_on          = [azuredevops_serviceendpoint_azurecr.acr]
  name                = "acr_identity_credential_${var.app_short_name}"
  resource_group_name = azurerm_user_assigned_identity.devops_acr.resource_group_name
  parent_id           = azurerm_user_assigned_identity.devops_acr.id
  audience            = ["api://AzureADTokenExchange"]
  issuer              = azuredevops_serviceendpoint_azurecr.acr.workload_identity_federation_issuer
  subject             = azuredevops_serviceendpoint_azurecr.acr.workload_identity_federation_subject
}

Debug Output

Panic Output

Expected Behavior

We have yaml files for team-specific configuration and we create resources such as Azure DevOps project per such yaml file.
In our QA pipeline we have tests for creating new resources and (since we had some issues with terraform destroy in the past) we achieve that by deploying all resources, running some tests and then we delete the yaml file and run terraform apply. That way all "Temp" resources are deleted and on next run we can recreate them again.

What we expect here is for the service endpoint to be deleted.

Actual Behavior

We get an error.

Error:  Delete service endpoint error Cannot delete manually created Workload Identity Federation Docker Registry service connection while related App Registration <ID OF THE CREATED IDENTITY> exists in Azure under tenant <OUR AZURE TENANT ID>.

Steps to Reproduce

1. Put the above code as a module and invoke the module in main.tf
2. terraform apply
3. Remove the module invocation from main.tf
4. terraform apply - this will try to remove all resources and fail on the service connection

Important Factoids

References

• #0000

[xuzhang3]

@bwesolowskicf This is controlled by service. Service will check the related resources in the AAD when trying to delete the service connection.

[bwesolowskicf]

@xuzhang3 Hi, I'm not quite sure what does 'controlled by service' mean. Is there anything I can do to resolve this issue?

[xuzhang3]

@bwesolowskicf ADO service will check the related app status in AAD when you trying to delete azuredevops_serviceendpoint_azurecr which means that if the application (azurerm_federated_identity_credential) still exists, the deletion will be rejected. You can add time_sleep to wait a moment after azurerm_federated_identity_credential deleted

[karts499]

unfortunately the issue is still there. I have tried to delete the same resource as above. The only workaround I have found working for me is to delete the secret via the portal or do a terraform destroy -target before the main task.
@xuzhang3 time_sleep wont work for destroy. the main problem is that terraform tries to the destroy the azuredevops_serviceendpoint_azurecr first and not the azurerm_federated_identity_credential. As far as I know there is no way to influence the destruction sequence. Not yet...

[xuzhang3]

@karts499 Have you tried depends_on?

[Fresa]

Same problem with azuredevops_serviceendpoint_azurerm. Works when running a second time, which might be due to eventual consistency I guess. Wouldn't it make sense that the provider retries a couple of times if this is expected?

[karts499]

@karts499 Have you tried depends_on?
yes I did, wont change the outcome. You simply can´t change the destroy order. depends_on just works as intented for the creation. For destruction terraform just tries to delete both not simultaneously but in the wrong order...

[xuzhang3]

Normally when a new Azure Container Registry or AzureRM service endpoint created, a new APP will be created the the AAD. In my test env I can can create new Azure Container registry or AzureRM service endpoint and destroy it with no errors, although the APP still exists in ADD.
In some cases, when we destroy the service connection, the service also tries to destroy the application but I cannot reproduce error.

App in MS Entra: