Skip to content

Commit

Permalink
add apiserver client config to keycloak (#235)
Browse files Browse the repository at this point in the history
Signed-off-by: akihikokuroda <akihikokuroda2020@gmail.com>
  • Loading branch information
akihikokuroda authored Mar 9, 2023
1 parent 782da91 commit 4421796
Show file tree
Hide file tree
Showing 4 changed files with 270 additions and 59 deletions.
3 changes: 3 additions & 0 deletions infrastructure/helm/hacks/apiserver-patch.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#!/bin/bash
LOCAL_IP=$1
kubectl patch deployments kuberay-apiserver --type=json -p='[{"op": "add", "path": "/spec/template/spec/containers/-","value":{"image": "quay.io/gogatekeeper/gatekeeper:2.1.1","imagePullPolicy": "IfNotPresent","name": "gatekeeper","args":["--no-redirects=true","--forwarding-grant-type=client_credentials","--listen=0.0.0.0:4180","--client-id=rayapiserver","--client-secret=APISERVERSECRET-CHANGEME","--discovery-url=http://'$LOCAL_IP':31059/realms/quantumserverless","--enable-logging=true","--verbose=true","--upstream-url=http://kuberay-apiserver-service:8888/"]}}]'
36 changes: 36 additions & 0 deletions infrastructure/helm/quantumserverless/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ helm dependency build
Update values.yaml file. Find and replace the following strings

- **CLIENTSECRET-CHANGEME**: string used as the secret for a OIDC protocol
- **APISERVERSECRET-CHANGEME**: string used as the secret for a OIDC protocol for apiserver
- **SECRET-CHANGEME**: string used as the secret for a OIDC protocol
- **HELM-RELEASE**: release name used in the helm install command
- **LOCAL-IP**: IP address that can be accessed from both outside of the cluster and inside of the cluster.

Expand Down Expand Up @@ -42,6 +44,12 @@ eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
TX packets 5373197 bytes 774842996 (774.8 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
```
Kind - kubectl output (**172.18.0.2**)
```
# kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
kind-control-plane Ready control-plane 5d6h v1.25.3 172.18.0.2 <none> Ubuntu 22.04.1 LTS 5.4.0-139-generic containerd://1.6.9
```

Install from the default values file
```shell
Expand All @@ -59,6 +67,12 @@ Install from specific values file
kubectl patch svc -n ray kuberay-apiserver-service --type json --patch '[{"op" : "replace" ,"path" : "/spec/selector" ,"value" : {"app.kubernetes.io/component": "kuberay-apiserver"}}]'
```

(temporary) Patch the kuberay-apiserver deployment

```shell
./hack/apisesrver/patch.sh <LOCAL-IP>
```

## Helm chart versions

The Quantum Serverless Chart has several internal and external dependencies. If you are interested to know what versions the project is using you can check them in the [Chart.lock file](./Chart.lock).
Expand Down Expand Up @@ -114,3 +128,25 @@ For our Ray Charts dependencies we are using the configuration created by the Ra
- The initial user ID and password for both keycload console(adminUser/adminPassword) and Ray dashboard(keycloakUserID/keycloakPassword) can be changed in the values.yaml file. It is good to change them before apply the helm.
- Keycloak console can be accessed at http://LOCAL-IP:31059/. Its initial user ID and password are "admin" and "passw0rd".
- Ray dashboard can be accessed at http://localhost/. Its initial user ID and password are "user" and "passw0rd".

## Usage

- Ray Api Server access needs the access token issued by the keycloak. Here is the example to obtain the access token and send request to the Ray API Server

```
#!/bin/bash
API=$1
RESPONSE=$(curl --request POST \
--url 'http://<LOCAL-IP>:31059/realms/quantumserverless/protocol/openid-connect/token' \
--header 'content-type: application/x-www-form-urlencoded' \
--data grant_type=client_credentials \
--data client_id=rayapiserver \
--data client_secret=APISERVERSECRET-CHANGEME \
--data audience=rayapiserver | jq .access_token)
TOKEN=${RESPONSE//'"'/}
curl --request GET -k --proxy http://<LOCAL-IP>:30634/ \
--header "authorization: Bearer $TOKEN" \
--header 'content-type: application/json' \
--url "http://kuberay-apiserver-service:8888/$API"
```
Loading

0 comments on commit 4421796

Please sign in to comment.