Check if a container or table exists before trying to create it

[liliankasem]

Issue describing the changes in this PR

Resolves multiple GH issues where customers are seeing an excessive amount of "409 The specified container already exists" errors in their logs:

warn: Azure.Core[8]
      Error response [d35f480a-744d-4871-ac64-20693baf387a] 409 The specified container already exists. (00.0s)
      Server:Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
      x-ms-request-id:6ee968fc-c01e-00a4-5f07-ec787b000000
      x-ms-client-request-id:d35f480a-744d-4871-ac64-20693baf387a
      x-ms-version:2021-08-06
      x-ms-error-code:ContainerAlreadyExists
      Date:Wed, 20 Sep 2023 21:15:31 GMT
      Content-Length:230
      Content-Type:application/xml

CreateIfNotExistsAsync throws an exception if the container exists. An issue was opened for this in the sdk repo but was closed unresolved. This was an intention change and not a bug (more details). However, it still does lead to excessive logs.

The change may only need to happen for the blob container but figured better safe and do it for all usage of CreateIfNotExistsAsync.

Relevant issues:

• "409 The specified container already exists."  Azure-Functions#2166
• Microsoft.Azure.WebJobs.Extensions.Storage version 5.1.2 and Azure Functions v. 4.2.0, error: "The specified container already exists", status 409 Azure-Functions#2370
• https://stackoverflow.com/questions/71202908/azure-function-works-but-reports-409-the-specified-container-already-exists

Pull request checklist

• My changes do not require documentation changes
  • Otherwise: Documentation issue linked to PR
• My changes should not be added to the release notes for the next release
  • Otherwise: I've added my notes to release_notes.md
• My changes do not need to be backported to a previous version
  • Otherwise: Backport tracked by issue/PR #issue_or_pr
• My changes do not require diagnostic events changes
  • Otherwise: I have added/updated all related diagnostic events and their documentation (Documentation issue linked to PR)
• I have added all required tests (Unit tests, E2E tests)

Additional information

Additional PR information

[mathewc]

Is the issue you referenced for 409s talking about conflicts coming from storage operations performed by the Host (i.e. code in this repo), or by code in lower levels? I ask because most of the storage operations performed by the Host are done in the WebJobs SDK, and looking through all the CreateIfNotExistsAsync calls (e.g. here) they all seem to be using the existing pattern of just relying on the storage SDK method.

So I'm wondering whether the changes made in this PR will actually address the customer issue? I.e. we're using CreateIfNotExistsAsync all over the place, down through transitive dependencies.

[liliankasem]

Is the issue you referenced for 409s talking about conflicts coming from storage operations performed by the Host (i.e. code in this repo), or by code in lower levels? I ask because most of the storage operations performed by the Host are done in the WebJobs SDK, and looking through all the CreateIfNotExistsAsync calls (e.g. here) they all seem to be using the existing pattern of just relying on the storage SDK method.

So I'm wondering whether the changes made in this PR will actually address the customer issue? I.e. we're using CreateIfNotExistsAsync all over the place, down through transitive dependencies.

That's a good point, a couple years ago we made a similar change but for the lease acquisition for a similar reason, and I think we missed the container creation flow. I think we should change this in the WebJobs SDK too if we can. I did some testing locally and this did reduce some of the 409 noise.

[liliankasem]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).