Unable to create azurerm_api_management_api with imported content file with azurerm > 3.69.0

[mennlo]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.5.7

AzureRM Provider Version

3.69.0

Affected Resource(s)/Data Source(s)

azurerm_api_management_api

Terraform Configuration Files

data "azurerm_api_management" "apim" {
  name                = var.apim_name
  resource_group_name = var.resource_group_name
}

resource "azurerm_api_management_api" "api" {
  for_each = var.managed_apis

  name                  = var.instance_id != null ? "${each.key}-${var.instance_id}" : each.key
  display_name          = "${each.value.display_name} [${upper(var.env)}]"
  resource_group_name   = data.azurerm_api_management.apim.resource_group_name
  api_management_name   = data.azurerm_api_management.apim.name
  revision              = "current"
  soap_pass_through     = each.value.import_type == "wsdl" ? true : false
  path                  = local.is_shared_service ? "${local.app_context}/${each.value.service_path}" : each.value.service_path
  protocols             = ["https"]
  subscription_required = each.value.subscription_required
  version               = each.value.version != null ? each.value.version : null
  version_set_id        = each.value.version_set != null ? azurerm_api_management_api_version_set.version_set[each.value.version_set].id : null

  dynamic "import" {
    for_each = each.value.api_file == null ? [] : [1]
    content {
      content_format = each.value.import_type
      content_value  = file("${var.paths.apis}/${each.value.api_file}")

      dynamic "wsdl_selector" {
        for_each = each.value.wsdl_selector

        content {
          service_name  = wsdl_selector.value.service_name
          endpoint_name = wsdl_selector.value.endpoint_name
        }
      }
    }
  }
  depends_on = [azurerm_api_management_api_version_set.version_set]
}

EXAMPLE managed_apis var:

managed_apis = {
  "testapi" = {
    display_name = "test"
    service_path = "test"
    import_type  = "openapi"
    // This product id will have to match one of the the product id's in the 
    // Azure Portal in the APIs -> Products section of the APIM.  If you don't
    // see one that matches the project your API is for then reach out your
    // team lead (if not using a shared services apim) or the Cloud Services
    // team in MS Teams (if using share services apim)
    product_id            = "test"
    api_file              = "test.openapi.yaml"
    subscription_required = false
    wsdl_selector         = []
  },
}

EXAMPLE test.openapi.yaml

openapi: 3.0.0
info:
  title: title
  description: endpoint
  version: 1.0.0
servers:
  - url: "https://redacted.com/test/v1/test/test"
    description: environment
paths:
  /default:
    post:
      operationId: default
      summary: Default
      description: Default operation
      responses:
        "200":
          description: Accepted
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/response"
components:
  schemas:
    response:
      type: object
      properties:
        status:
          type: string
          example: success
  securitySchemes:
    basicAuth:
      type: http
      scheme: basic

Debug Output/Panic Output

terraform [init -upgrade=false]
  Running command terraform with args [init -upgrade=false]
  
  Initializing the backend...
  
  Initializing provider plugins...
  - Finding hashicorp/azurerm versions matching "< 4.0.0"...
  - Installing hashicorp/azurerm v3.73.0...
  - Installed hashicorp/azurerm v3.73.0 (signed by HashiCorp)
  
  Terraform has created a lock file .terraform.lock.hcl to record the provider
  selections it made above. Include this file in your version control repository
  so that Terraform can guarantee to make the same selections by default when
  you run "terraform init" in the future.
  
  Terraform has been successfully initialized!
  
  You may now begin working with Terraform. Try running "terraform plan" to see
  any changes that are required for your infrastructure. All Terraform commands
  should now work.
  
  If you ever set or change modules or backend configuration for Terraform,
  rerun this command to reinitialize your working directory. If you forget, other
  commands will detect it and remind you to do so if necessary.
  terraform [apply -input=false -auto-approve -var managed_apis={"test-api-v1" = {"display_name" = "Test API", "service_path" = "test/test", "subscription_required" = false, "wsdl_selector" = [], "product_id" = "test", "api_file" = "test.openapi.yaml", "import_type" = "openapi", "version" = null, "version_set" = null}} -var project_context=testing -var version_sets=[] -var instance_id=256 -var env=dev -var api_operations=[] -var apim_product={} -var api_policies=[] -var operation_policies=[] -var named_values={} -var resource_group_name=rg-redacted-01 -var apim_name=apim-redacted-01 -var paths={"apis" = "/home/runner/work/terraform-azure-managed-api-module/terraform-azure-managed-api-module/test/apis", "policies" = null} -no-color -lock=false]
  Running command terraform with args [apply -input=false -auto-approve -var managed_apis={"test-api-v1" = {"display_name" = "Test API", "service_path" = "test/test", "subscription_required" = false, "wsdl_selector" = [], "product_id" = "test", "api_file" = "test.openapi.yaml", "import_type" = "openapi", "version" = null, "version_set" = null}} -var project_context=testing -var version_sets=[] -var instance_id=256 -var env=dev -var api_operations=[] -var apim_product={} -var api_policies=[] -var operation_policies=[] -var named_values={} -var resource_group_name=rg-redacted-01 -var apim_name=apim-redacted-01 -var paths={"apis" = "/home/runner/work/terraform-azure-managed-api-module/terraform-azure-managed-api-module/test/apis", "policies" = null} -no-color -lock=false]
  data.azurerm_api_management.apim: Reading...
  data.azurerm_api_management.apim: Read complete after 1s [id=/subscriptions/***/resourceGroups/rg-redacted-01/providers/Microsoft.ApiManagement/service/apim-redacted-01]
  
  Terraform used the selected providers to generate the following execution
  plan. Resource actions are indicated with the following symbols:
    + create
  
  Terraform will perform the following actions:
  
    # azurerm_api_management_api.api["test-api-v1"] will be created
    + resource "azurerm_api_management_api" "api" {
        + api_management_name   = "apim-redacted-01"
        + api_type              = (known after apply)
        + display_name          = "Test API [DEV]"
        + id                    = (known after apply)
        + is_current            = (known after apply)
        + is_online             = (known after apply)
        + name                  = "test-api"
        + path                  = "test/test"
        + protocols             = [
            + "https",
          ]
        + resource_group_name   = "rg-redacted-01"
        + revision              = "current"
        + service_url           = (known after apply)
        + soap_pass_through     = false
        + subscription_required = false
        + version               = (known after apply)
        + version_set_id        = (known after apply)
  
        + import {
            + content_format = "openapi"
            + content_value  = <<-EOT
                  openapi: 3.0.0
                  info:
                    title: title
                    description: endpoint
                    version: 1.0.0
                  servers:
                    - url: "https://redacted.com/test/v1/test/test"
                      description: sandbox environment
                  paths:
                    /default:
                      post:
                        operationId: default
                        summary: Default
                        description: Default operation
                        responses:
                          "200":
                            description: Accepted
                            content:
                              application/json:
                                schema:
                                  $ref: "#/components/schemas/response"
                  components:
                    schemas:
                      response:
                        type: object
                        properties:
                          status:
                            type: string
                            example: success
                    securitySchemes:
                      basicAuth:
                        type: http
                        scheme: basic
              EOT
          }
      }
  
    # azurerm_api_management_product_api.product_api["test-api-v1"] will be created
    + resource "azurerm_api_management_product_api" "product_api" {
        + api_management_name = "apim-redacted-01"
        + api_name            = "test-api"
        + id                  = (known after apply)
        + product_id          = "test"
        + resource_group_name = "rg-redacted-01"
      }
  
  Plan: 2 to add, 0 to change, 0 to destroy.
  azurerm_api_management_api.api["test-api-v1"]: Creating...
  azurerm_api_management_api.api["test-api-v1"]: Still creating... [10s elapsed]
  
  Warning: Argument is deprecated
  
    with azurerm_api_management_api.api["test-api-v1"],
    on main.tf line 14, in resource "azurerm_api_management_api" "api":
    14:   soap_pass_through     = each.value.import_type == "wsdl" ? true : false
  
  `soap_pass_through` will be removed in favour of the property `api_type` in
  version 4.0 of the AzureRM Provider
  
  (and one more similar warning elsewhere)
  
  Error: creating/updating Api (Subscription: "***"
  Resource Group Name: "rg-redacted-01"
  Service Name: "apim-redacted-01"
  Api: "test-api"): polling after CreateOrUpdate: executing request: unexpected status 404 with error: ResourceNotFound: Api not found.
  
    with azurerm_api_management_api.api["test-api-v1"],
    on main.tf line 6, in resource "azurerm_api_management_api" "api":

Expected Behaviour

The azurerm_api_management_api should be created successfully.

I'm trying to create an azurerm_api_management_api using an openapi type imported file.

Actual Behaviour

With any version > 3.69.0, The azurerm_api_management_api creation fails with the following error:

  Error: creating/updating Api (Subscription: "***"
  Resource Group Name: "rg-redacted-01"
  Service Name: "apim-dedacted-01"
  Api: "test-api"): polling after CreateOrUpdate: executing request: unexpected status 404 with error: ResourceNotFound: Api not found.

If you used <= 3.69.0 the creation is successful.

Steps to Reproduce

terraform apply

Important Factoids

No response

References

No response

[crizzs]

Im facing this issue too. Whenever we tried to apply changes to the apim, the outcomes are unpredictable as well.

[rcskosir]

@mennlo Thanks for taking the time to open this issue. Please subscribe to PR #23348 that @sinbai has created.

[zhallgato]

It looks like the PR didn't solve the problem. It still occurs to me on v3.75.0.

Can we reopen this issue or should I make a new one?

[mennlo]

It looks like the PR didn't solve the problem. It still occurs to me on v3.75.0.

Can we reopen this issue or should I make a new one?

It worked for me after using v3.75.0.

[DominikKrissVisma]

I also have this issue, also when using AzureRM provider in version 3.75.0 (mbfrahry FYI)

[rcskosir]

@zhallgato, @mennlo, @DominikKrissVisma Thank you for the update, I will reopen this issue.

[rcskosir]

Hey @sinbai, mind taking another look at this one?

[sinbai]

Hi @DominikKrissVisma, @zhallgato , sorry that PR could not solve your problem. Since the PR already addressed mennlo's issue , I assume that your case might be different. Is it possible to provide detailed terraform configuration (easily reproducible and contains variable values - except sensitive information) and reproduce steps to help reproduce and resolve the problem? In addition, it might help a lot if Terraform trace logs could also be provided. Thanks in advance.

[DominikKrissVisma]

hello,
this is terraform file (ranamed from tf to txt):
genericapi.txt
notice we are using azurerm_api_management_api_version_set, and revision in order to force API recreation. The reason is that when API yaml file changes, like there is new endpoint added, those changes are not always deployed via terraform. When this happens we are expiriencing the 404 issue:

Api: "int-c-client-systems-v2"): polling after CreateOrUpdate: executing request: unexpected status 404 with error: ResourceNotFound: Api not found.

  with azurerm_api_management_api.generic_client_systems["ccc"],
  on api_management_generic.tf line 31, in resource "azurerm_api_management_api" "generic_client_systems":
  31: resource "azurerm_api_management_api" "generic_client_systems"

the issue is that API is actualy created during that run, but not tracked in the TF state file, therefore after this happes we need to import the API into state file.

[hieumoscow]

This issue still persists in 3.80, I had to downgrade to 3.68 for the resource to create. Any timeline to fix this issue?

[christianfosli]

I tried to work-around this by downgrading from 3.80 to 3.68 but it seems the state is no longer compatible, so it's not the easiest work-around.

I get a bunch of

│ The current state of azurerm_api_management_product_policy.<policyName>
│ was created by a newer provider version than is currently selected. Upgrade
│ the azurerm provider to work with this state.

│ The current state of azurerm_api_management_api_policy.<policyName>
│ was created by a newer provider version than is currently selected. Upgrade
│ the azurerm provider to work with this state.

Sorry for the noise :-)

[tombuildsstuff]

@sinbai looks like the original PR didn't resolve this issue - since you sent the original PR to fix this, can you take another look into this one? Thanks!

[sinbai]

Can anyone here provide more contexts in terms of step-by-step-repro-this-issue?

[christianfosli]

Can anyone here provide more contexts in terms of step-by-step-repro-this-issue?

I can probably make one, but I can't make the time the next couple of days.

I can start by sharing some more information about the resource were this issue occurred to me on azurerm 3.80, though.

The resource looks like this

resource "azurerm_api_management_api" "api" {
  name                  = "clientname-${var.env}-api-${var.api_name}" # e.g. "clientname-d-api-clientapi-internal"
  api_management_name   = var.api_management_name         # e.g. "clientname-d-apim-common"
  resource_group_name   = var.resource_group_name         # e.g. "clientname-d-rg-common"
  revision              = var.revision                    # e.g. "50923", "50930", ...
  display_name          = var.display_name                # e.g. "clientname clientapi internal dev"
  path                  = "${var.api_name}"               # e.g. "clientapi"
  protocols             = ["https"]
  subscription_required = var.subscription_required       # e.g. true
  service_url           = var.service_url                 # e.g. https://clientname-d-aks.norwayeast.cloudapp.azure.com/clientapi
  import {
    content_format = "openapi"
    content_value  = file(var.openapi_definition)         # a fairly ordinary OpenAPI v3 spec in json format
                                                          # with info.version set to "internal"
  }
  lifecycle {
    # The version number is pulled from the BUILD_ID, so it changes constantly.
    # This will only update if the actual API has changed.
    ignore_changes = [revision]
  }
}

We have just one azurerm_api_management_api resource with this name+api_management_name,
so the "revision" property is just being used to track which CI pipeline run it came from.

[zhallgato]

I have some new information.

We had 6 APIMs in different environments (4 of them in East US and 2 of them in East US 2 Azure regions).

We only experienced the above-mentioned issue on those APIMs which were in East US 2 region.

So, we recreated these two APIMs in East US, and since the recreation we no longer experience the issue. Everything works fine.

Maybe the issue is related only to specific Azure regions. (maybe there are some delays in these regions?)

[cbruce80]

This is happening to me too. We are in US East 2 as well and running azurerm 3.80. We are also importing a fairly large openapi definition.

[kjetiloen]

For us, it seems to happen when trying to add apis with "large" (openapi) definition files (+300KB, +90 operations). Smaller definition files worked fine, and reverting to 3.69.0 works for everything. Location Norway East.

[sinbai]

I really appreciate everyone's help. It has been reproduced and is under investigation. Thank you so much!

[celsocoutinho-tangany]

I am hitting this issue as well, on 3.81.0. My openai definition is quite small, having only 3 operations

[sinbai]

base-layer: the PollUntilDone of CreateOrUpdateThenPoll stops polling when the API returns resource not found error on first poll hashicorp/go-azure-sdk

Hi @tombuildsstuff I assume that this is a base layer issue. For the case of long running operation poller and the API return Location refers to itself, the 404 errors should be additionally handled in the base layer, what do you think? Please refer to issue #740 for details.

[sinbai]

Hi @mrickly thank you very much for the information, I could reproduce your situation. I have filed an Terraform SDK issue to track it. Could you please track it for more updates?

[mrickly]

Thank you @sinbai , I will track the new issue.

[sharatbhaskar1988]

@manicminer and @sharatbhaskar1988 - I seem to have a solution that at least worked for my particular version of this issue.

When I was deploying, I was using two copies of a mostly identical template to create two different APIs. The only differences were the names of the Function Apps.

Both templates were using the same setting for "path":

resource "azurerm_api_management_api" "aaaf_apim_api_funcapp_001" {
  name                              = "api-${var.aaaf_funcapp_001_name}"
  resource_group_name               = var.aaaf_resource_group_name
  api_management_name               = var.aaaf_apim_name
  revision                          = "1"
  display_name                      = var.aaaf_funcapp_001_name
  path                              = "public"
  protocols                         = [ "https" ]
  import {
    content_format                  = "openapi"
    content_value                   = file("modules/100_funcapps/funcapp_001/openapi_spec.yaml")
  }
    depends_on                      = [ azurerm_api_management_backend.aaaf_apim_backend_funcapp_001 ]
}

By ensuring that "path" was set differently in both templates the problem went away. I hope this is useful to you.

@deankelly780 i have an open api doc which have different paths like /a/b/c and /d/e/f. Not sure what should i set in the path property.

[J0F3]

@sinbai I just want to let you know that this error can happens during the initial deployment (terraform apply) of an API also. When the API is a bit more complex meaning the api has a lot of resources etc. the creation of the API on the API Management can take some time. In such a case also the wrong polling URL is used and the creation of the API fails also with an 404 error. Even tough the API is actually created on the API Management. Any subsequent deployment will then succeed because the API is actually already created on the API Management.

In the TF_TRACE Log the root cause is good recognizable:

Frist the correct PUT request to create the API is made:

[DEBUG] provider.terraform-provider-azurerm_v3.99.0_x5.exe: PUT https://management.azure.com/subscriptions/1cc5c6b6-15cb-4082-ad3f-38874b1c6488/resourceGroups/caz-test-gaia-switzerlandnorth-rg/providers/Microsoft.ApiManagement/service/jfe-test-apim/apis/jfe-test-api;rev=1?api-version=2022-08-01

For wich the Azure response correctly contents the Location header:

[DEBUG] provider.terraform-provider-azurerm_v3.99.0_x5.exe: AzureRM Response for https://management.azure.com/subscriptions/1cc5c6b6-15cb-4082-ad3f-38874b1c6488/resourceGroups/caz-test-gaia-switzerlandnorth-rg/providers/Microsoft.ApiManagement/service/jfe-test-apim/apis/jfe-test-api;rev=1?api-version=2022-08-01: 
HTTP/2.0 202 Accepted
Content-Length: 0
Cache-Control: no-cache
Date: Tue, 16 Apr 2024 16:05:59 GMT
Expires: -1
Location: https://management.azure.com/subscriptions/1cc5c6b6-15cb-4082-ad3f-38874b1c6488/resourceGroups/caz-test-gaia-switzerlandnorth-rg/providers/Microsoft.ApiManagement/service/jfe-test-apim/apis/jfe-test-api;rev=1?api-version=2022-08-01&asyncId=661ea1e75ddd170c48700615&asyncCode=201
Pragma: no-cache
Server: Microsoft-HTTPAPI/2.0
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Ms-Correlation-Request-Id: 0aff170c-5357-ea12-c025-fe9c194313de
X-Ms-Ratelimit-Remaining-Subscription-Writes: 1199
X-Ms-Request-Id: 0aff170c-5357-ea12-c025-fe9c194313de
X-Ms-Routing-Request-Id: GERMANYWESTCENTRAL:20240416T160559Z:6e839db5-1f26-41f2-8120-635fd7f0594f

But then instead that the URL from the Location header is get polled (according to Track asynchronous Azure operations: url-to-monitor-status) azurerm makes a GET request with the URL of the actual API resource which does not exist yet:

[DEBUG] provider.terraform-provider-azurerm_v3.99.0_x5.exe: AzureRM Request: 
GET /subscriptions/1cc5c6b6-15cb-4082-ad3f-38874b1c6488/resourceGroups/caz-test-gaia-switzerlandnorth-rg/providers/Microsoft.ApiManagement/service/jfe-test-apim/apis/jfe-test-api;rev=1?api-version=2022-08-01 HTTP/1.1
Host: management.azure.com
User-Agent: HashiCorp/go-azure-sdk (Go-http-Client/1.1 api/2022-08-01) HashiCorp Terraform/1.8.0 (+https://www.terraform.io) Terraform Plugin SDK/2.10.1 terraform-provider-azurerm/3.99.0 pid-222c6c49-1b0a-5959-a213-6608f9eb8820
Accept: application/json; charset=utf-8; IEEE754Compatible=false
Content-Type: application/json; charset=utf-8
Odata-Maxversion: 4.0
Odata-Version: 4.0
X-Ms-Correlation-Request-Id: 0aff170c-5357-ea12-c025-fe9c194313de
Accept-Encoding: gzip

Consequently Azure response then with 404 which lead then to the failure of the deployment:

[DEBUG] provider.terraform-provider-azurerm_v3.99.0_x5.exe: AzureRM Response for https://management.azure.com/subscriptions/1cc5c6b6-15cb-4082-ad3f-38874b1c6488/resourceGroups/caz-test-gaia-switzerlandnorth-rg/providers/Microsoft.ApiManagement/service/jfe-test-apim/apis/jfe-test-api;rev=1?api-version=2022-08-01: 
HTTP/2.0 404 Not Found
Content-Length: 79
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Date: Tue, 16 Apr 2024 16:06:09 GMT
Expires: -1
Pragma: no-cache
Server: Microsoft-HTTPAPI/2.0
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Content-Type-Options: nosniff
X-Ms-Correlation-Request-Id: 0aff170c-5357-ea12-c025-fe9c194313de
X-Ms-Ratelimit-Remaining-Subscription-Reads: 11999
X-Ms-Request-Id: 0aff170c-5357-ea12-c025-fe9c194313de
X-Ms-Routing-Request-Id: GERMANYWESTCENTRAL:20240416T160610Z:81bc76dc-904c-40ea-b8b5-d333aa296505

{"error":{"code":"ResourceNotFound","message":"Api not found.","details":null}}: 
[TRACE] provider.terraform-provider-azurerm_v3.99.0_x5.exe: Called downstream: @module=sdk.helper_schema tf_req_id=5b5f079c-857a-58f5-64c4-58242ddc6e19 tf_resource_type=azurerm_api_management_api @caller=github.com/hashicorp/terraform-plugin-sdk/v2@v2.29.0/helper/schema/resource.go:910 tf_provider_addr=provider tf_rpc=ApplyResourceChange 
[TRACE] provider.terraform-provider-azurerm_v3.99.0_x5.exe: Received downstream response: tf_resource_type=azurerm_api_management_api tf_rpc=ApplyResourceChange diagnostic_warning_count=0 tf_proto_version=5.4 tf_provider_addr=provider tf_req_duration_ms=12132 tf_req_id=5b5f079c-857a-58f5-64c4-58242ddc6e19 @caller=github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/tf5serverlogging/downstream_request.go:40 @module=sdk.proto diagnostic_error_count=1 
[ERROR] provider.terraform-provider-azurerm_v3.99.0_x5.exe: Response contains error diagnostic: @module=sdk.proto diagnostic_detail="" diagnostic_severity=ERROR
  diagnostic_summary=
  | creating/updating Api (Subscription: "1cc5c6b6-15cb-4082-ad3f-38874b1c6488"
  | Resource Group Name: "caz-test-gaia-switzerlandnorth-rg"
  | Service Name: "jfe-test-apim"
  | Api: "jfe-test-api;rev=1"): polling after CreateOrUpdate: executing request: unexpected status 404 (404 Not Found) with error: ResourceNotFound: Api not found.
   tf_proto_version=5.4 tf_provider_addr=provider @caller=github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/internal/diag/diagnostics.go:58 tf_req_id=5b5f079c-857a-58f5-64c4-58242ddc6e19 tf_resource_type=azurerm_api_management_api tf_rpc=ApplyResourceChange timestamp="2024-04-16T18:06:10.163+0200"
2024-04-16T18:06:10.163+0200 [TRACE] provider.terraform-provider-azurerm_v3.99.0_x5.exe: Served request: tf_rpc=ApplyResourceChange tf_proto_version=5.4 tf_resource_type=azurerm_api_management_api tf_req_id=5b5f079c-857a-58f5-64c4-58242ddc6e19 @caller=github.com/hashicorp/terraform-plugin-go@v0.19.0/tfprotov5/tf5server/server.go:872 @module=sdk.proto tf_provider_addr=providert

So I think this will get probably also fixed with hashicorp/go-azure-sdk#962 .

[sinbai]

@J0F3 thank you for providing the above information. Per the TF Log above, I assume that once #962 is resolved, the "Api not found" error would not occur in the above situation. So could you please follow # 962 for more updates?

[J0F3]

@sinbai Yes I assume the same, so I will follow #962

Thank you!

[DominikKrissVisma]

Seems like I have a workaround for importing new OAS from file into the API in APIM. The workaround also utilizes revisions to make import happen, and it benefits from APIM revisions design. It creates new revision and after import it will "release" it = making it current revision, which is basically "zero downtime" deployment. There is no API recreation with this approach.

First we need to create API, with hardcoded revision 1:

resource "azurerm_api_management_api" "cars_api" {
  name                = "cars-api"
  resource_group_name = data.azurerm_resource_group.this.name
  api_management_name = azurerm_api_management.this.name
  revision            = "1" 
  display_name        = "Cars API"
  protocols           = ["https"]
  version             = var.version
  path                = "cars"
  version_set_id      = azurerm_api_management_api_version_set.this.id
  subscription_required = false
}

This will act as base api containing revision 1. There are no operations yet, those are imported by null_resource using AzureCLI:

resource "null_resource" "import_oas" {
  triggers = {
    cars_api_revision = var.revision
  }

  # import OAS as revision
  provisioner "local-exec" {
    command = "az apim api import -g my-resource-group -n my-api-management --path \"/cars\" --specification-format OpenApi --specification-path ./files/openapi/cars_api_v2_oas.yaml --api-revision ${var.revision} --api-version-set-id cars-api --api-id cars-api --subscription-required false"
  }
  
  # make newly imported revision current on the API
  provisioner "local-exec" {
    command = "az apim api release create -g my-resource-group -n my-api-management --api-id cars-api --api-revision ${var.revision}"
  }

  depends_on = [ azurerm_api_management_api.cars_api ]
}

First local-exec is importing the OAS into revision defined by var.revision. Second local-exec is making imported revision current.

Some remarks:

• make sure your oas.yaml has empty description, more here
• if you are creating custom policies onto imported OAS operations than you need to force recreation of the policy, it will not be created automaticaly and will "stick" with operations of the previous revision:

  lifecycle {
    replace_triggered_by = [
      null_resource.import_oas.id
    ]
  }

• I also played with calling the AzureCLI from different pipeline step after terraform apply, however than I had issues with the custom policies on some operations.
• All created revisions and imported OAS remains within the API on Azure, so also rollback of the API is easy (cosidering no custom policies).

Hopefully it might be helpfull for some of you guys.

[didaskein]

We have the same issue on North Europe deployment and for big swagger file (more than 600ko).
At the end the swagger is well deploy on Apim but the terraform state is not aligned to the reality and it's a mess with the retry... :-(

[kaaquist]

We had a similar issue like this. The error from terraform apply was:

Error: creating/updating Api (Subscription: "********-****-****-************"
│ Resource Group Name: "rg-xxx-dev"
│ Service Name: "apim-xxx-dev"
│ Api: "xxx;rev=1"): polling after CreateOrUpdate: executing request: unexpected status 404 with error: ResourceNotFound: Api not found.
│ 
│   with module.apim_api.azurerm_api_management_api.api,
│   on ../../../../modules/apim_api/resource/main.tf line 12, in resource "azurerm_api_management_api" "api":
│   12: resource "azurerm_api_management_api" "api" {
│ 

After a lot of debugging I went to the portal to set up the APIM manually. What I found was that we had exceeded the max amount of operations, error in portal.azure.com - (error):

Maximum amount of Operations (1000) reached

We use the Consumption sku - and there the limit is 1.000 link

So I deleted some of our endpoints to limit the amount of operations we had in dev, and then I was able to run terraform apply.
So in our case it seems that the 404 error is covering up the fact the we have reached a quota limit for the operations on our api management plan/sku -> "Consumption".

Hope that this will help others.

[cemusta]

This problem is still happening on azurrm 3.107.0 and terraform 1.8.5

However, after some try and error; I noticed it works fine with some changes in the spec yaml file:

openapi: 3.0.0
# servers: []

When I reduced my openapi version to 3.0.0 from 3.0.1 and removed servers block, import worked fine. Not sure why but the problem is happening when latest version openapi & servers block defined.
`

[skycloudnest]

Hello, I also have this problem what is the reason here is my module:

resource "azurerm_api_management_api" "api" {
  name                  = lower(replace(var.api_settings.name, " ", "-"))
  description           = var.api_settings.description
  resource_group_name   = var.resource_group_name
  api_management_name   = var.api_management_name
  service_url           = var.api_settings.service_url
  revision              = var.api_settings.revision
  display_name          = var.api_settings.name
  path                  = var.api_settings.basepath
  protocols             = ["https"]
  subscription_required = var.api_settings.subscription_required

  soap_pass_through = var.soap_pass_through

  import {
    content_format = "openapi" 
    content_value  = file(var.api_settings.openapi_file_path)
  }
}

[thorbenheins]

We also encounter a 404 for huge (> 65.000 lines) yml OAPI Specs. Looks to me as if the PollUntilDone somehow is "impatient". The API deploys in the end and the az cli can successfully fetch the resource about a minute later. Unforunately tf has already decided to stop polling by then.

[joshblackmoor]

We also have this issue running 3.116.0.

After a lot of trial and error I figured out that if you try to apply the same openAPI file that is already being used in an existing API within the same APIM it will result in a 404 (we have a use case to temporarily have the same openAPI file deployed within the same instance just on a different path).

To get around this I deployed the new API with a different openAPI file and then after applying I updated the terraform to use the correct file and change the revision.

I also tested the same scenario but through Azure portal manually but couldn't replicate the issue. Azure activity logs also do not show any errors when getting the 404.

Hopefully this may help someone!

[sinbai]

We have the same issue on North Europe deployment and for big swagger file (more than 600ko). At the end the swagger is well deploy on Apim but the terraform state is not aligned to the reality and it's a mess with the retry... :-(

We also encounter a 404 for huge (> 65.000 lines) yml OAPI Specs. Looks to me as if the PollUntilDone somehow is "impatient". The API deploys in the end and the az cli can successfully fetch the resource about a minute later. Unforunately tf has already decided to stop polling by then.

Hi @didaskein , @thorbenheins thank you for providing info. From the description, I assume this is another 404 issue. It does not seem to be the same issue as the 404 error caused by #962. Is it possible to provide terraform config (include swagger file/yml OAPI file) that could help repro and troubleshooting?

[sinbai]

We also have this issue running 3.116.0.

After a lot of trial and error I figured out that if you try to apply the same openAPI file that is already being used in an existing API within the same APIM it will result in a 404 (we have a use case to temporarily have the same openAPI file deployed within the same instance just on a different path).

To get around this I deployed the new API with a different openAPI file and then after applying I updated the terraform to use the correct file and change the revision.

I also tested the same scenario but through Azure portal manually but couldn't replicate the issue. Azure activity logs also do not show any errors when getting the 404.

Hopefully this may help someone!

Hi @joshblackmoor appreciate your feedback. I am currently following up on the 404 error caused by #962. For your situation, I am not sure if it is related to #962, is it possible to provide terraform config (include openAPI file) and steps to reproduce the symptoms mentioned below to help me troubleshoot?

After a lot of trial and error I figured out that if you try to apply the same openAPI file that is already being used in an existing API within the same APIM it will result in a 404 (we have a use case to temporarily have the same openAPI file deployed within the same instance just on a different path).

[sinbai]

Hello, I also have this problem what is the reason here is my module:

resource "azurerm_api_management_api" "api" {
  name                  = lower(replace(var.api_settings.name, " ", "-"))
  description           = var.api_settings.description
  resource_group_name   = var.resource_group_name
  api_management_name   = var.api_management_name
  service_url           = var.api_settings.service_url
  revision              = var.api_settings.revision
  display_name          = var.api_settings.name
  path                  = var.api_settings.basepath
  protocols             = ["https"]
  subscription_required = var.api_settings.subscription_required

  soap_pass_through = var.soap_pass_through

  import {
    content_format = "openapi" 
    content_value  = file(var.api_settings.openapi_file_path)
  }
}

Hi @skycloudnest could you please first verify the validity of the import file (file(var.api_settings.openapi_file_path))? If the import file is incorrect, I assume that your case caused by #962. I am already following up on the fix for # 962. If there is no problem with the import file, could you provide a minimal reproducible TF configuration (including the import file) and repro steps to help reproduce this issue?

[qfyra]

This is still an issue in azurerm 4.3.0. If it's any help, I don't think it's necessarily related to content files. Commenting out the import yields the same error message. And in our case, a subsequent manual Plan and Apply actually created the API but resulted in "polling after CreateOrUpdate: polling failed". Activity logs in Azure reported successfully created API. Next manual Plan and Apply resulted in "already exists - to be managed via Terraform this resource needs to be imported into the State".

[J0F3]

@qfyra
Yes, you are correct. It has actually nothing to do with the content of the files. It is a bug in the azurerm provider which does not correctly poll the status of the long running operation of the Azure API and instead just shows a misleading error.
Even when the file to import has actually an error the azurerm provider does not show the correct error and just cuts of the actual error reported by the Azure API, wich just leaves you in the dark what actually happend.

And yes, nothing has changed with v4 either and the problem persists now for over a year with no visible progress of improvements. Very very annoying. 😞

[sinbai]

FYI. A PR has been submitted to fix this issue and is currently awaiting review.

[J0F3]

Thanks @sinbai. I saw that. But the PR is now also open for over a month without, at least visible, progress.
Further there is another nasty bug which seems to be also related to the PR. I asked questions there and mentioned the PR but got no response at all. So, at my end these all look not very promising. 😢

For approximately over a year now these bugs hurt us almost on a daily basis and we have then to do manual fixes etc. Definitely something that should just not be the case when using Terraform/IaC!
And every time, we hear here: "there is a PR" and "it is almost fixed". But the PRs and the "almost fixed" does not help at the end.
So, I am really tired of it, and I only really believe it when I see it that these bugs are finally fixed, and the azurerm resources are finally working as intended.

[didaskein]

On our side in order to avoid to wait a fix we use a bypass : Retry pattern with the use of Terragrunt on top of Terraform.

First we detect that we need to do : jsonencode(jsondecode()) on the content of the open api file to avoid whitespace encoding issue which create a new deployement action for nothing (so many call in parallel for nothing)

# Creation of API inside the APIM
resource "azurerm_api_management_api" "public" {
  name                = var.my_application.technicalName
  resource_group_name = var.apim_resource_group_name
  api_management_name = var.apim_management_name
  #revision            = var.build_id
  revision              = "1"
  display_name          = "${var.my_application.friendlyName} API"
  description           = "${var.my_application.friendlyName} API deploy through Terraform"
  path                  = var.my_application.urlPath
  protocols             = ["https"]
  subscription_required = false

  # openapi are downloaded during build time on ADO, and generated in prehook locally with terragrunt
  import {
    content_format = "openapi+json"
    content_value  = jsonencode(jsondecode(var.public_openapi_content)) # Encode & Decode in order to avoid whitepace issues & redeploy open api for nothing
  }
}

Then on Terragrunt hcl file we have added the retry pattern :

retry_max_attempts       = 5
retry_sleep_interval_sec = 60
retryable_errors = [
  "(?s).*polling after CreateOrUpdate: executing request: unexpected status 404 .* with error: ResourceNotFound: Api not found.*",
  "(?s).*unexpected status 409 (409 Conflict) with error.*",
  "(?s).*checking for existing Container.*DomainSuffix.*executing request: unexpected status 403.*",
] 

In order to avoid an error during the retry if the api was finaly deployed on APIM you need to import them in terraform, for that you need this powershell script : testApiExistence.ps1 and this terraform file : apim_import.tf :

testApiExistence.ps1 :

[CmdletBinding()]
param (
  [Parameter(Mandatory = $true)] [string] $apimResourceGroupName,
  [Parameter(Mandatory = $true)] [string] $apimServiceName,
  [Parameter(Mandatory = $true)] [string] $apiId
)

$apiToBeImported = az apim api show --service-name $apimServiceName --api-id $apiId --resource-group $apimResourceGroupName --query "name" -o tsv

if ($apiToBeImported) {
  Write-Output @{ "exists" = "true"} | ConvertTo-Json
  }
else {
  Write-Output @{ "exists" = "false"} | ConvertTo-Json
}

apim_import.tf :

# During the first run, we have an issue with the provider. We add the import as a workaround to fix the issue.
# polling after CreateOrUpdate: executing request: unexpected status 404 with error: ResourceNotFound: Api not found

data "external" "apim_import_public" {
  program = ["pwsh", "${path.module}/testApiExistence.ps1", "${data.azurerm_api_management.apim.resource_group_name}", "${data.azurerm_api_management.apim.name}", "${local.my_application.technicalName};rev=1"]
}

import {
  for_each = data.external.apim_import_public.result.exists == "true" ? [1] : []
  to       = module.apim_config.azurerm_api_management_api.public
  id       = "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${data.azurerm_api_management.apim.resource_group_name}/providers/Microsoft.ApiManagement/service/${data.azurerm_api_management.apim.name}/apis/${local.my_application.technicalName};rev=1"
}

Terragrunt is also very helpfull if you need to run prehook or afterhook script when Terraform cannot solve your need...

[Khayoann2]

On our side in order to avoid to wait a fix we use a bypass : Retry pattern with the use of Terragrunt on top of Terraform.

First we detect that we need to do : jsonencode(jsondecode()) on the content of the open api file to avoid whitespace encoding issue which create a new deployement action for nothing (so many call in parallel for nothing)

# Creation of API inside the APIM
resource "azurerm_api_management_api" "public" {
  name                = var.my_application.technicalName
  resource_group_name = var.apim_resource_group_name
  api_management_name = var.apim_management_name
  #revision            = var.build_id
  revision              = "1"
  display_name          = "${var.my_application.friendlyName} API"
  description           = "${var.my_application.friendlyName} API deploy through Terraform"
  path                  = var.my_application.urlPath
  protocols             = ["https"]
  subscription_required = false

  # openapi are downloaded during build time on ADO, and generated in prehook locally with terragrunt
  import {
    content_format = "openapi+json"
    content_value  = jsonencode(jsondecode(var.public_openapi_content)) # Encode & Decode in order to avoid whitepace issues & redeploy open api for nothing
  }
}

Then on Terragrunt hcl file we have added the retry pattern :

retry_max_attempts       = 5
retry_sleep_interval_sec = 60
retryable_errors = [
  "(?s).*polling after CreateOrUpdate: executing request: unexpected status 404 .* with error: ResourceNotFound: Api not found.*",
  "(?s).*unexpected status 409 (409 Conflict) with error.*",
  "(?s).*checking for existing Container.*DomainSuffix.*executing request: unexpected status 403.*",
] 

In order to avoid an error during the retry if the api was finaly deployed on APIM you need to import them in terraform, for that you need this powershell script : testApiExistence.ps1 and this terraform file : apim_import.tf :

testApiExistence.ps1 :

[CmdletBinding()]
param (
  [Parameter(Mandatory = $true)] [string] $apimResourceGroupName,
  [Parameter(Mandatory = $true)] [string] $apimServiceName,
  [Parameter(Mandatory = $true)] [string] $apiId
)

$apiToBeImported = az apim api show --service-name $apimServiceName --api-id $apiId --resource-group $apimResourceGroupName --query "name" -o tsv

if ($apiToBeImported) {
  Write-Output @{ "exists" = "true"} | ConvertTo-Json
  }
else {
  Write-Output @{ "exists" = "false"} | ConvertTo-Json
}

apim_import.tf :

# During the first run, we have an issue with the provider. We add the import as a workaround to fix the issue.
# polling after CreateOrUpdate: executing request: unexpected status 404 with error: ResourceNotFound: Api not found

data "external" "apim_import_public" {
  program = ["pwsh", "${path.module}/testApiExistence.ps1", "${data.azurerm_api_management.apim.resource_group_name}", "${data.azurerm_api_management.apim.name}", "${local.my_application.technicalName};rev=1"]
}

import {
  for_each = data.external.apim_import_public.result.exists == "true" ? [1] : []
  to       = module.apim_config.azurerm_api_management_api.public
  id       = "/subscriptions/${data.azurerm_client_config.current.subscription_id}/resourceGroups/${data.azurerm_api_management.apim.resource_group_name}/providers/Microsoft.ApiManagement/service/${data.azurerm_api_management.apim.name}/apis/${local.my_application.technicalName};rev=1"
}

Terragrunt is also very helpfull if you need to run prehook or afterhook script when Terraform cannot solve your need...

Well thought out, thank you very much for your contribution.