feat(event-hubs-scaler): Add pod/workload identity support to checkpoint in Azure Blob Storage

[andyatwork]

Add support for pod identity based authentication to blob storage accounts in Event Hub scaler

Checklist

• Commits are signed with Developer Certificate of Origin (DCO - learn more)
• Tests have been added
• A PR is opened to update the documentation on (repo) (if applicable)
• Changelog has been updated and is aligned with our changelog requirements

Relates to #3569

Relates to kedacore/keda-docs#905

[v-shenoy]

I checked this PR out and tested it. Confirmed that it works with Workload Identity as well @tomkerkhove @andyatwork

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: eventhub-ta
spec:
  podIdentity:
    provider: azure-workload
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: eventhub-so
spec:
  scaleTargetRef:
    name: nginx-deployment
  minReplicaCount: 0
  maxReplicaCount: 5
  triggers:
  - type: azure-eventhub
    metadata:
      storageAccountName: testhubstorage
      checkpointStrategy: azureFunction
      eventHubNamespace: test-hub-ns
      eventHubName: hub-1
    authenticationRef:
      name: eventhub-ta

[andyatwork]

I checked this PR out and tested it. Confirmed that it works with Workload Identity as well @tomkerkhove @andyatwork

apiVersion: apps/v1
kind: Deployment
metadata:

...

Excellent, thanks for testing this!

[andyatwork]

@tomkerkhove , can you follow up on the change request above?

[tomkerkhove]

Not from my side

[JorTurFer]

Looking good, some comments inline.
Should we add an e2e test checking this?
Thanks for this contribution! ❤️

[andyatwork]

Not from my side

Sorry, I had responded to your comment inline above and realized I hadn't published my comments yet. They remained "pending", so I guess you hadn't actually seen my response. Hope the replies addresses your concerns.

[v-shenoy]

@JorTurFer Any changes here after upgrading the event hubs SDK?

[JorTurFer]

I didn't touch the eventhub sdk, only the service-bus sdk which I'd say is different

[JorTurFer]

I have checked the SDK for this and, oh surprise, uses other authentication mechanism different from the service-bus SDK...
Maybe in the future they unify the SDK in a single SDK for azure or at least reuse the authentication part, but atm I think it's a mess

[v-shenoy]

I didn't touch the eventhub sdk, only the service-bus sdk which I'd say is different

I thought you upgraded it to v3, which is why I was wondering if anything broke over there that would require changes here.

[JorTurFer]

I didn't touch the eventhub sdk, only the service-bus sdk which I'd say is different

I thought you upgraded it to v3, which is why I was wondering if anything broke over there that would require changes here.

Lol, let me check because I didn't notice that

[JorTurFer]

did you mean with this PR?
I only changed the service-bus sdk

others are minor versions

[v-shenoy]

did you mean with this PR? I only changed the service-bus sdk others are minor versions

Oh, I just saw the title for this issue, #2986 and I assumed it was upgrading TO v3. If it's minor versions, then shouldn't cause any issues.

[JorTurFer]

No no, it's a minor version, but that minor version created conflicts I think because it uses generics somewhere. That's why I created the issue to remember bumping it after update golang version

[JorTurFer]

BTW; I have checked and there is a beta version for event-hub in the azure-sdk-for-go so eventually we could migrate to it once it's stable and simplify the pod identities reusing the approach of service-bus 🤞

[JorTurFer]

I have to apologize @andyatwork because I missed this PR...
Let's check if @tomkerkhove has anything to say

[tomkerkhove]

LGTM, @v-shenoy any concerns from your side?

[JorTurFer]

@andyatwork , could you rebase your branch? 🙏 We added e2e test for event hub some weeks ago and rebasing the branch we can execute them before merging the PR

[andyatwork]

Yes, will rebase and update the PR.

[JorTurFer]

/run-e2e event_hub*
Update: You can check the progress here