Add support for different tenant and/or cloud in Azure Workload Identity

[sbebrys]

Proposal

Current implementation of Azure Workload Identity credentials has possibility to override keda clientId to other via AuthPodIdentity, it will be useful to has the same regarding to tenantId and cloud. I this way we could scale based on resources located in different tenant and even cloud (via registered federated credentials near used managed identity).

Use-Case

My use case is not to wide, but is related only for azure-pipelines-scaler (but could be spread to different cases).
Within Azure DevOps I wanna run and scale azure self hosted release agents in target tenants (including also for example China cloud) to be more near network of deployed resources, but ADO is sticked to dedicated single CI/CD tenant and we cannot read job queue of pool for the scaling from other tenants as ADO has configured. Extend configuration of AuthPodIdentity for tenantId and Cloud will solve that challenge.

Is this a feature you are interested in implementing yourself?

No

Anything else?

No response

[JorTurFer]

@tomkerkhove @v-shenoy , do you know if this is possible in Azure side? I mean, as this can be potentially used for managed identities, can a MSI from one tenant be assigned in other tenant? I guess that for service principal should work, right?

[tomkerkhove]

Azure as such supports different clouds + MI, but I am not sure about Azure DevOps.

I do not know about using multiple Entra tenants, did you check if workload identity supports this @sbebrys?

[sbebrys]

Yes, MSI supports multiple tenants via federation credentials for kubernetes, because we need register there just issuer and subject from workload id_token, which in aks is just oidc url of cluster and service account name + namespace (without any cloud nature informations).

On aks side workload identity addon injects to the container “local“ tenantId and authorizedHost (where we could exchange that id_token to MSI access_token). But if we manualy change that to different tenantId, cloud (authorizedHost) and clientId we could jump to MSI in different organization (if just MSI federated credentials contains our id_token details). ADO will be also accessible in this way, because we will be "visible" as MSI defined in target tenant connected to ADO and source cloud trace gets lost during exchanging. MSI is just added to ADO as normal user but for getting access_token we not play with that IAM, but with pure EntraId - finally we need just jwt for magic aud (499b84ac-1321-427f-aa17-267ca6975798) and known by ADO subject (MSI clientId).

If you look to the azure identity sdk all three informations (by default readed from envs) could be overrided via workload identity credentials options WorkloadIdentityCredentialOptions.

But it's not just interpretation of code, I use that on production 😉
I made service which sync data between tenants. Service is working in aks of tenant A where use local Azure resources via MSI and workload identity. But source of data is in resources of tenant B where we go by MSI defined there, but with federated credentials of issuer aks A and service account of running service. The same data is also sync to the China resources in tenant C, where the same federated credentials are attached to Chinese MSI. Switching authentication of source and targets is made via changing in WorkloadIdentityCredentialOptions tenantId, authorizedHost and clientId, only location of assertion token is the same. Service works smoothly.

[JorTurFer]

Nice research! Thanks ❤️
It totally makes sense. Are you willing to add support for it?

[sbebrys]

Maybe as just observer, golang is not my main stack 😉
Thanks.

[pauldotyu]

Hi all, I'm pretty familiar with workload identity and want to learn KEDA dev a bit more so I'd like to take a stab at this.

[JorTurFer]

Thanks @pauldotyu !

[pauldotyu]

Hey @JorTurFer quick question on this particular line of code in the triggerauthentication_types.go file.

After a bit of testing, I noticed that the RoleArn field is of type string while others are *string. Any particular reason why that is? One thing I noticed when creating a new ta resource is that as soon as I apply the ta manifest, a second generation is created with an empty RoleArn value even though it was not included in my ta manifest. Looks like changing this field's type to *string will resolve that. Is this something I should address here or should I open up a separate issue to track?

[JorTurFer]

Probably it was a mistake, you can change it to pointer string (remember to update the usages to check nil pointers and not panic)