[EPIC] Support external oauth2 token endpoint to allow `oauth2client` removal #16803

k15r · 2023-02-13T13:54:33Z

Description

Allow the customer to specify a custom token endpoint with matching client credentials to support eventmesh after removal of the oauth2client cr.

Background
The oauth2client CR will be removed from kyma in the near future. As the eventmesh backend requires a client_credential flow to be able to access the webhook endpoint we need to allow replacing this kyma internal token-enpoint with a customer provided external solution.

Tasks

a working apirule for IAS:

- apiVersion: gateway.kyma-project.io/v1beta1
  kind: APIRule
  metadata:
    annotations:
      kubectl.kubernetes.io/last-applied-configuration: |
        {"apiVersion":"gateway.kyma-project.io/v1beta1","kind":"APIRule","metadata":{"annotations":{},"labels":{"beb":"webhook","service":"loadtest-subscriber-0"},"name":"meins","namespace":"tunas"},"spec":{"gateway":"kyma-gateway.kyma-system.svc.cluster.local","host":"meins.k15r-sandbox.kymatunas.shoot.canary.k8s-hana.ondemand.com","rules":[{"accessStrategies":[{"config":{"jwks_urls":["https://kymagoattest.accounts400.ondemand.com/oauth2/certs"]},"handler":"jwt"}],"methods":["POST","OPTIONS"],"path":"/.*","service":{"name":"loadtest-subscriber-0","port":80}}],"service":{"name":"loadtest-subscriber-0","port":80}}}
    creationTimestamp: "2023-02-10T11:56:06Z"
    generation: 1
    labels:
      beb: webhook
      service: loadtest-subscriber-0
    name: meins
    namespace: tunas
    resourceVersion: "31403764"
    uid: a43eb2a0-a851-4dcb-bc6e-133dabf199b1
  spec:
    gateway: kyma-gateway.kyma-system.svc.cluster.local
    host: meins.k15r-sandbox.kymatunas.shoot.canary.k8s-hana.ondemand.com
    rules:
    - accessStrategies:
      - config:
          jwks_urls:
          - https://kymagoattest.accounts400.ondemand.com/oauth2/certs
        handler: jwt
      methods:
      - POST
      - OPTIONS
      path: /.*
      service:
        name: loadtest-subscriber-0
        port: 80
    service:
      name: loadtest-subscriber-0
      port: 80
  status:
    APIRuleStatus:
      code: OK
    accessRuleStatus:
      code: OK
    lastProcessedTime: "2023-02-10T11:56:06Z"
    observedGeneration: 1
    virtualServiceStatus:
      code: OK

https://www.ory.sh/docs/oathkeeper/pipeline/authn

The text was updated successfully, but these errors were encountered:

mfaizanse · 2023-02-14T10:15:33Z

Options for OAuth2Clients:

IAS: Identity Authentication (Service)
xsuaa: eXtended Service for UAA

mfaizanse · 2023-02-14T12:24:19Z

Existing APIRule created by EC

╰─ k get apirules -n tunas-testing webhook-xt2qf -o yaml | yh
apiVersion: gateway.kyma-project.io/v1beta1
kind: APIRule
metadata:
  creationTimestamp: "2023-02-13T15:23:09Z"
  finalizers:
  - gateway.kyma-project.io/subresources
  generateName: webhook-
  generation: 1
  labels:
    beb: webhook
    service: test
  name: webhook-xt2qf
  namespace: tunas-testing
  ownerReferences:
  - apiVersion: eventing.kyma-project.io/v1alpha2
    blockOwnerDeletion: true
    kind: Subscription
    name: test-noapp
    uid: 9866eadc-cc1c-41f9-a9e6-0efecf8dd9b4
  resourceVersion: "292584"
  uid: 6382b29f-ac0e-4b33-8a4a-c547d920ad21
spec:
  gateway: kyma-gateway.kyma-system.svc.cluster.local
  host: web-d685nq2yhv.fzn-s1.kymatunas.shoot.canary.k8s-hana.ondemand.com
  rules:
  - accessStrategies:
    - handler: oauth2_introspection
    methods:
    - POST
    - OPTIONS
    path: /
    service:
      external: true
      name: test
      port: 80
  service:
    external: true
    name: test
    port: 80
status:
  APIRuleStatus:
    code: OK
  accessRuleStatus:
    code: OK
  lastProcessedTime: "2023-02-13T15:23:10Z"
  observedGeneration: 1
  virtualServiceStatus:
    code: OK

In New APIRule, the difference will be:

   - accessStrategies:
      - config:
          jwks_urls:
          - https://kymagoattest.accounts400.ondemand.com/oauth2/certs
        handler: jwt

mfaizanse · 2023-02-14T12:27:01Z

How to allow users to specify custom oauth2client credentials?

Option 1: Use the existing secret which we use to switch backend to Eventmesh

apiVersion: v1
kind: Secret
metadata:
  labels:
    kyma-project.io/eventing-backend: beb
  name: eventing-backend
  namespace: default
type: Opaque
data:
  management: WyAghYjEwMTAiLCAgICAgIHM6Ly90dW5hcyhbmQuY29tIiAgIH0gXQ==
  messaging: WyAgIHsgICAgICJiiTTZfRVJ2a04wMnNBWE1W50aWNhdGlvbi5zYXAkZW1hbmQuY29tIiAgIH0gXQ==
  namespace: ZGVmYXVsddsadsadsadsaS90dW5hcy1kZXZlbG9w
  serviceinstanceid: MWI1ODhjZdsadsadsadsadsadDktMjA5NWUxYWIwNjJl
  xsappname: ZGVmYXVsdsadsadsadasdsadsadsaddahYjMzNjk4fHhiZW0tc2VydmljZS1icm9rZXItIWIxMDEw
  oauth2_client_id: OGIwOTRlMzgdasd5Ni00ZTM4LWFmNTUtNzI4NGIyYmYxNWQ3      ### <<<<- New config
  oauth2_client_secret: Xzg1eFhWeFdsadsa2X1BKUjRmN3g1Tl9TdVk=      ### <<<<- New config
  oauth2_auth_url: https://example.com     ### <<<<- New config

Pros:

Single point to configure EventMesh backend, because in future without these configs, EventMesh flow would not work.
In future (Multiple Active Backends), different oauth2clients can be configured per backend.

Cons:

EventMesh credentials and oauth2 client credentials will be coupled in a single secret.
In future (Multiple Active Backends), the EventMesh secret may be changed.

Option 2: Create and define a new secret `eventing-oauth2-config`

apiVersion: v1
kind: Secret
metadata:
  name: eventing-oauth2-config    ### <<<<<- Defined name
  namespace: kyma-system           ### <<<<<- Defined namespace, or `Kube-public`
type: Opaque
data:
  client_id: OGIwOTRlMzgdasd5Ni00ZTM4LWFmNTUtNzI4NGIyYmYxNWQ3
  client_secret: Xzg1eFhWeFdsadsa2X1BKUjRmN3g1Tl9TdVk=
  auth_url: https://example.com

Pros:

Decoupled configs for OAuth2 and EventMesh.

Cons:

To switch to EventMesh backend, user has to manage two separate secrets.
We have to watch for changes from two secrets i.e. EventMesh secret and oauth2-secret.

Questions:

Do users have access to create secret in kyma-system namespace?
Is it okay to have user configurations in kyma-system ?

Option 3: Create and define a new secret and label it

apiVersion: v1
kind: Secret
metadata:
  name: eventing-oauth2-name          ### <<<<- ANY name
  namespace: user-namespace          ### <<<<- ANY namespace
  labels:
    kyma-project.io/eventing-oauth2-client: true   ### <<<<- ANY namespace
type: Opaque
data:
  client_id: OGIwOTRlMzgdasd5Ni00ZTM4LWFmNTUtNzI4NGIyYmYxNWQ3
  client_secret: Xzg1eFhWeFdsadsa2X1BKUjRmN3g1Tl9TdVk=
  auth_url: https://example.com

Pros:

Secret is not bound to kyma-system namespace.

Cons:

To switch to EventMesh backend, user has to manage two separate secrets.
What if we have multiple secrets with the same label in the cluster?
We have to watch for changes from two secrets i.e. EventMesh secret and oauth2-secret.

Summary

Right now, it is not decided that whether the OAuth2 client credentials will be provided in the Kyma cluster or will the user have to provide the credentials. Based on the decision, we can choose one of the options from above.

If the credentials will be provided by the kyma, then we can go with Option 2 .
- In this case, if we go with Option 1 then we would have to inject the oauth2 client credentials in user provided event mesh secret using reconciler.
If the credentials will be provided by the user, then we can go with Option 1 or Option 3.

mfaizanse · 2023-02-14T12:29:47Z

Blocked until we know that whether the OAuth2 client credentials will be provided in the Kyma cluster or will the user have to provide the credentials.

k15r · 2023-03-09T12:20:56Z

kubectl create -f - <<EOF
apiVersion: services.cloud.sap.com/v1alpha1
kind: ServiceInstance
metadata:
  name: xsuaa-mst
  namespace: mst
spec:
  serviceOfferingName: xsuaa
  servicePlanName: application
  externalName: xsuaa-mst
EOF

kubectl get serviceinstances.services.cloud.sap.com xsuaa-instance -n mst -o yaml

kubectl create -f - <<EOF
apiVersion: services.cloud.sap.com/v1alpha1
kind: ServiceBinding
metadata:
  name: xsuaa-binding
  namespace: mst
spec:
  serviceInstanceName: xsuaa-mst
  externalName: xsuaa-binding
  secretName: xsuaa-binding
EOF

kubectl get servicebindings.services.cloud.sap.com xsuaa-binding -n mst -o yaml

kubectl get secret xsuaa-binding -n mst -o yaml

export CLIENT_ID="$(kubectl get secret -n mst xsuaa-binding -o jsonpath='{.data.clientid}' | base64 --decode)"
export CLIENT_SECRET="$(kubectl get secret -n mst xsuaa-binding -o jsonpath='{.data.clientsecret}' | base64 --decode)"

export ENCODED_CREDENTIALS=$(echo -n "$CLIENT_ID:$CLIENT_SECRET" | base64)

curl --location --request POST "https://mst-xsz69e4u.authentication.eu20.hana.ondemand.com/oauth/token?grant_type=client_credentials&client_id=$CLIENT_ID" --header "Content-Type: application/x-www-form-urlencoded" --header "Authorization: Basic $ENCODED_CREDENTIALS"

https://mst-xsz69e4u.authentication.eu20.hana.ondemand.com/.well-known/openid-configuration

jwksuri: https://mst-xsz69e4u.authentication.eu20.hana.ondemand.com/token_keys
token: https://mst-xsz69e4u.authentication.eu20.hana.ondemand.com/oauth/token

mfaizanse · 2023-03-15T13:26:28Z

With XSUAA:

PR: [Do-not-merge] [POC] Support external oauth2 token endpoint in Eventing #17075
POC doc is added to External link in this ticket.

mfaizanse · 2023-03-16T10:24:45Z

Questions:

Upgrade flow? There should be no event loss. Hint: use the patch request for updating EventMesh subscriptions.

k15r · 2023-04-04T11:30:11Z

here is the documentation for the PUT request to update the credentials on eventmesh: https://api.sap.com/api/SAPEventMeshDefaultCredentialsAPIs/path/put_webhook_credentials__credential_name_

k15r · 2023-04-17T07:35:47Z

Synchronize with: kyma-project/eventing-auth-manager#2

k15r added the area/eventing Issues or PRs related to eventing label Feb 13, 2023

mfaizanse self-assigned this Feb 13, 2023

mfaizanse removed their assignment Feb 14, 2023

mfaizanse self-assigned this Mar 13, 2023

mfaizanse mentioned this issue Mar 14, 2023

[Do-not-merge] [POC] Support external oauth2 token endpoint in Eventing #17075

Closed

mfaizanse assigned k15r and unassigned mfaizanse Mar 16, 2023

k15r changed the title ~~Support external oauth2 token endpoint to allow oauth2client removal~~ [EPIC] Support external oauth2 token endpoint to allow oauth2client removal Apr 21, 2023

mfaizanse mentioned this issue Apr 26, 2023

Update the APIRule and webhook configs for EventMesh to allow validating the presented token. #17371

Closed

5 tasks

k15r added the Epic label May 15, 2023

k15r added this to the 2.18 milestone Sep 11, 2023

k15r closed this as completed Sep 11, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[EPIC] Support external oauth2 token endpoint to allow `oauth2client` removal #16803

[EPIC] Support external oauth2 token endpoint to allow `oauth2client` removal #16803

k15r commented Feb 13, 2023 •

edited by marcobebway

Loading

mfaizanse commented Feb 14, 2023

mfaizanse commented Feb 14, 2023

mfaizanse commented Feb 14, 2023

mfaizanse commented Feb 14, 2023

k15r commented Mar 9, 2023

mfaizanse commented Mar 15, 2023 •

edited

Loading

mfaizanse commented Mar 16, 2023

k15r commented Apr 4, 2023

k15r commented Apr 17, 2023

[EPIC] Support external oauth2 token endpoint to allow oauth2client removal #16803

[EPIC] Support external oauth2 token endpoint to allow oauth2client removal #16803

Comments

k15r commented Feb 13, 2023 • edited by marcobebway Loading

mfaizanse commented Feb 14, 2023

mfaizanse commented Feb 14, 2023

Existing APIRule created by EC

In New APIRule, the difference will be:

mfaizanse commented Feb 14, 2023

How to allow users to specify custom oauth2client credentials?

Option 1: Use the existing secret which we use to switch backend to Eventmesh

Option 2: Create and define a new secret eventing-oauth2-config

Questions:

Option 3: Create and define a new secret and label it

Summary

mfaizanse commented Feb 14, 2023

k15r commented Mar 9, 2023

mfaizanse commented Mar 15, 2023 • edited Loading

mfaizanse commented Mar 16, 2023

k15r commented Apr 4, 2023

k15r commented Apr 17, 2023

[EPIC] Support external oauth2 token endpoint to allow `oauth2client` removal #16803

[EPIC] Support external oauth2 token endpoint to allow `oauth2client` removal #16803

k15r commented Feb 13, 2023 •

edited by marcobebway

Loading

Option 2: Create and define a new secret `eventing-oauth2-config`

mfaizanse commented Mar 15, 2023 •

edited

Loading