[BUG]: AzureFileCopy task fails when using Workload Identity Federation

[Sharparam]

Task name

AzureFileCopy

Task version

5

Environment type (Please select at least one enviroment where you face this issue)

• Self-Hosted
• Microsoft Hosted
• VMSS Pool
• Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

windows-2022

Task log

Too long to be pasted: https://gist.github.com/Sharparam/c3bd654b31a9e6fa87473d89f71d2bdf

Relevant log output

##[error]Upload to container: 'redacted' in storage account: 'redacted' with blob prefix: '' failed with error: 'Unsupported authentication scheme 'WorkloadIdentityFederation' for endpoint.' For more info please refer to https://aka.ms/azurefilecopyreadme

Aditional info

v5 is the latest version of the task.

[anish714]

Running into the same issue. Apparently it's the same issue for dacpacdeployment and it was fixed there: #18992

[anish714]

Running into the same issue.

[amitjaura]

Having same issue. Reverted back from Workload Identity Federation and it worked fine

[geekzter]

azcopy itself does not yet support Workload identity federation, see issue in the azure-storage-azcopy repo. Once it does we will update the task as well.

[DekCode]

Has anyone got it working or any work around for this?

[Sharparam]

Has anyone got it working or any work around for this?

We just switched back to the old authentication method for the time being.

[VassilisM]

Hi, same for me. Thing is... I get that "WorkloadIdentityFederation" some kind of authentication is, but I do not know how this is setup and therefore how to revert. So "switching back to old authentication" does not apply just yet, because it was never there. What kind of authentication? With keys and such? Any specific keyword to ask google? (shared access signature?)
UPDATE
Ok, got something. The service connection for the project is described as: "Azure Resource Manager using Workload Identity federation with OpenID Connect (automatic)" so I will ask them to create a new one as "Service Principal" and we go on from there. If this is not the right path, any tips, would be welcome. :-)

[arsh3186]

Hi
I am running into same issue, any updates on this? Is it working for anyone

[zee-cdp]

We are having same issue

[gsimardnet]

Workload Identity Federation is the recommended method to create a service connection. This task should be able to support it.

[stevenoneill]

Would love to see this updated.

[MattJeanes]

We've just been bitten by this as well. Workload identity federation is now the recommended way with prompts in the UI to upgrade so it was very much a surprise for us to see Microsoft maintained tasks still not supporting this on their newest task version.

[pilcherd]

Yep, us too. Set up a new "Recommended" connection and our build script didn't run and copy. Back to the old connection until there is a path forward.

[stevenoneill]

I followed #2112 through to Azure/azure-storage-azcopy#2545. 2545 seems to have some good active updates -- keeping an eye on that.

[benjiecov]

The latest update on that azcopy issue says it's not currently a priority. But Azure DevOps now only creates/renews the secrets for service principal service connections for 3 months (used to be 2 years). So Azure DevOps is pushing us to use workload identity federation to avoid frequent secret management, while azcopy (and by extension this task) don't support it.

[dfederm]

Policies are now being enforced which require workload identities, so the AzureFileCopy pipeline task is now effectively entirely unuseable.

[dfederm]

I was able to work around this by ditching AzureFileCopy in favor of the Azure CLI task (which properly supports workload identities) with the "az storage blob upload-batch" command:

az storage blob upload-batch --account-name <blobAccountName> -d <containerName> -s <sourceDirectory> --overwrite true

[eriksteinebach]

@dfederm Same for us, it didn't allow me to update the service connection, so had to upgrade. Now this task is broken. I will give your fix a try, but the Create SAS Token and AzureFileCopy task now has become unusable for us

[vihanga-bandara]

We are getting the same issue and we use AzureFileCopy@3. As everyone else has mentioned this seems to be due to service connection supporting Workload Identity Federation but the task itself does not support it. Looking at the Support for AAD Workload Identity, they might not fix it anytime soon.

[eriksteinebach]

I was able to solve it by using an AzureCLI task, maybe this will help some people:

steps:
- task: AzureCLI@2
  displayName: 'Copy ARM templates to artifact storage'
  inputs:
    azureSubscription: '<SUBSCRIPTION>'
    scriptType: pscore
    scriptLocation: inlineScript
    inlineScript: |
     az storage blob upload-batch --account-name "xartifacts" -d "artifacts" -s "$(System.DefaultWorkingDirectory)/_accountname/drop/Infrastructure/" --overwrite true
     
     $expiretime=$(date -u -d '30 minutes' +%Y-%m-%dT%H:%MZ)
     
     $connection=$(az storage account show-connection-string --resource-group "resourcegroupname" --name "xartifacts" --query connectionString)
     
     echo "##vso[task.setvariable variable=storageToken]?$(az storage container generate-sas --name artifacts --expiry $expiretime --permissions r --output tsv --connection-string $connection)"
     
     echo "##vso[task.setvariable variable=storageUri]https://xartifacts.blob.core.windows.net/artifacts"

This sets the two variables storageToken and storageUri to call the specific ARM template files I need

[mwiemer-microsoft]

v6 was released yesterday and resolves this issue for us :)

[geekzter]

The AzureFileCopyV6 task resolves this issue. It also uses Azure RBAC to access blob storage instead. This requires the identity of the service connection used to have the appropriate RBAC role e.g. Storage Blob Data Contributor. See Assign an Azure role for access to blob data.

[Veera-kambham]

SASToken is not getting generated in this task. The options fields are missing

[aya-bjoseph]

FYI - For anyone who might have stumbled upon this issue, there appears to be an ongoing regression bug in AzureFileCopyV6 that is blocking use of WIF service connections. See issue #19566 and fix PR #19863 which hasn't been merged as yet.

cc: @geekzter

[v-schhabra]

FYI - For anyone who might have stumbled upon this issue, there appears to be an ongoing regression bug in AzureFileCopyV6 that is blocking use of WIF service connections. See issue #19566 and fix PR #19863 which hasn't been merged as yet.

cc: @geekzter

#19867
This PR has been merged after incorporating the fixes.