Support for OSSM v2.6

[adam-cattermole]

Changes

Updates to support OSSM v2.6, primarily due to changes in types

Verification

Verified on OpenShift v4.16 - Steps adapted from existing guide

• Set CLUSTER_DOMAIN to the domain of your cluster as follows:

kubectl cluster-info
https://api.$CLUSTER_DOMAIN:6443

• Install Gateway API

make gateway-api-install

• Install OSSM v2.6.1 through the OperatorHub

• Create a namespace for istio

kubectl create namespace istio-system

• Create a ServiceMeshControlPlane in ClusterWide mode

kubectl apply -n istio-system -f -<<EOF
apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: istiocontrolplane
spec:
  version: v2.6
  mode: ClusterWide
  policy:
    type: Istiod
  profiles:
    - default
  telemetry:
    type: Istiod
  addons:
    prometheus:
      enabled: false
    kiali:
      enabled: false
    grafana:
      enabled: false
EOF

• Create a gateway targeting existing istio-ingressgateway:

kubectl apply -n istio-system -f -<<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  labels:
    istio: ingressgateway
  name: istio-ingressgateway
spec:
  gatewayClassName: istio
  listeners:
  - name: http
    port: 80
    protocol: HTTP
    allowedRoutes:
      namespaces:
        from: All
  addresses:
  - value: istio-ingressgateway.istio-system.svc.cluster.local
    type: Hostname
EOF

• Add a label to the default ingressgateway to tell istio which gateway API Gateway it is for:

kubectl label -n istio-system pods --selector="app=istio-ingressgateway" "istio.io/gateway-name=istio-ingressgateway"

• Create a namespace for toystore

kubectl create namespace toystore

• Deploy toystore

kubectl apply -n toystore -f https://raw.githubusercontent.com/Kuadrant/kuadrant-operator/main/examples/toystore/toystore.yaml

• Add the toystore namespace as a ServiceMeshMember

kubectl apply -n toystore -f -<<EOF
apiVersion: maistra.io/v1
kind: ServiceMeshMember
metadata:
  name: default
spec:
  controlPlaneRef:
    name: istiocontrolplane
    namespace: istio-system
EOF

• Create the httproute

kubectl apply -n toystore -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: toystore
  labels:
    app: toystore
spec:
  parentRefs:
    - name: istio-ingressgateway
      namespace: istio-system
  hostnames:
    - "*.toystore.apps.$CLUSTER_DOMAIN"
  rules:
    - matches:
        - method: GET
          path:
            type: PathPrefix
            value: "/toy"
      backendRefs:
        - name: toystore
          port: 80
EOF

• Expose the gateway

oc expose -n istio-system service/istio-ingressgateway --port 8080 --name toystore --hostname=api.toystore.apps.$CLUSTER_DOMAIN

• Curl toystore

curl http://api.toystore.apps.$CLUSTER_DOMAIN/toy -i
# 200 OK

• Deploy kuadrant from this branch

make install-cert-manager
make deploy-dependencies
IMG=quay.io/acatterm/kuadrant-operator:ossm-v2.6 make install
IMG=quay.io/acatterm/kuadrant-operator:ossm-v2.6 make deploy

• Request an instance of Kuadrant

kubectl -n kuadrant-system apply -f - <<EOF
apiVersion: kuadrant.io/v1beta1
kind: Kuadrant
metadata:
  name: kuadrant
spec: {}
EOF

• Create an APIKEY AuthPolicy:

kubectl apply -n toystore -f - <<EOF
apiVersion: kuadrant.io/v1beta2
kind: AuthPolicy
metadata:
  name: toystore
spec:
  targetRef:
    group: gateway.networking.k8s.io
    kind: HTTPRoute
    name: toystore
  rules:
    authentication:
      "api-key-users":
        apiKey:
          selector:
            matchLabels:
              app: toystore
          allNamespaces: true
        credentials:
          authorizationHeader:
            prefix: APIKEY
    response:
      success:
        dynamicMetadata:
          "identity":
            json:
              properties:
                "userid":
                  selector: auth.identity.metadata.annotations.secret\.kuadrant\.io/user-id
EOF

kubectl apply -n toystore -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: bob-key
  labels:
    authorino.kuadrant.io/managed-by: authorino
    app: toystore
  annotations:
    secret.kuadrant.io/user-id: bob
stringData:
  api_key: IAMBOB
type: Opaque
---
apiVersion: v1
kind: Secret
metadata:
  name: alice-key
  labels:
    authorino.kuadrant.io/managed-by: authorino
    app: toystore
  annotations:
    secret.kuadrant.io/user-id: alice
stringData:
  api_key: IAMALICE
type: Opaque
EOF

• Curl toystore:

curl http://api.toystore.apps.$CLUSTER_DOMAIN/toy -i
# 401 Unauthorized

curl -H 'Authorization: APIKEY IAMALICE' http://api.toystore.apps.$CLUSTER_DOMAIN/toy -i
# 200 OK

[codecov]

Codecov Report

Attention: Patch coverage is 64.51613% with 11 lines in your changes missing coverage. Please review.

Project coverage is 78.67%. Comparing base (ece13e8) to head (aec0c22).
Report is 195 commits behind head on main.

Files with missing lines	Patch %	Lines
pkg/istio/mesh_config.go	64.51%	5 Missing and 6 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #857      +/-   ##
==========================================
- Coverage   80.20%   78.67%   -1.54%     
==========================================
  Files          64       91      +27     
  Lines        4492     7066    +2574     
==========================================
+ Hits         3603     5559    +1956     
- Misses        600     1092     +492     
- Partials      289      415     +126     

Flag	Coverage Δ
bare-k8s-integration	5.10% <0.00%> (?)
controllers-integration	70.65% <0.00%> (?)
envoygateway-integration	46.67% <0.00%> (?)
gatewayapi-integration	10.71% <0.00%> (?)
integration	?
istio-integration	50.14% <0.00%> (?)
unit	27.90% <64.51%> (-2.13%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
api/v1beta1 (u)	38.46% <0.00%> (-32.97%)	⬇️
api/v1beta2 (u)	74.10% <63.82%> (-17.33%)	⬇️
pkg/common (u)	88.13% <ø> (-0.70%)	⬇️
pkg/istio (u)	71.51% <ø> (-2.40%)	⬇️
pkg/log (u)	94.73% <ø> (ø)
pkg/reconcilers (u)	∅ <ø> (∅)
pkg/rlptools (u)	84.26% <ø> (+4.81%)	⬆️
controllers (i)	80.48% <82.82%> (+3.68%)	⬆️

Files with missing lines	Coverage Δ
pkg/istio/mesh_config.go	68.92% <64.51%> (+3.21%)	⬆️

... and 44 files with indirect coverage changes

[maleck13]

@adam-cattermole in the verification we use the existing ingress pod? Do you know if it is possible to set up a new Gateway using Gateway API in another ns for example?

[adam-cattermole]

@maleck13 I didn't test that gateway with kuadrant, but did try deploying a gateway in a diff ns using gateway API successfully

[maleck13]

@trepel @jsmolar wondering if you want to use this for some of the verification with OSSM 2.6?

[maleck13]

changes look fine to me. I am upgrading my OCP cluster to give the verification a try.

[maleck13]

@adam-cattermole @trepel @jsmolar

Note I tested this on my hosted OCP cluster using 4.16.x

I followed @adam-cattermole guide but instead of using the internal service name as the address I allowed an AWS LB to be spun up to expose the gateway externally.
I then created a gateway in a separate namespace (so not istio-system) ingress-gateway and setup the toystore app and HTTProute. I did not use the ServiceMeshMember resource as traffic coming to this is from external source so I don't believe that is needed.
By default OSSM will set up networkpolicies to block external traffic and traffic from NS not mesh members. To allow external traffic I set the setting to do this to false to allow external traffic the simplest way possible:

spec.security.manageNetworkPolicy: false

apiVersion: maistra.io/v2
kind: ServiceMeshControlPlane
metadata:
  name: istiocontrolplane
spec:
  security:
    manageNetworkPolicy: false
  version: v2.6
  mode: ClusterWide
  policy:
    type: Istiod
  profiles:
    - default
  telemetry:
    type: Istiod
  addons:
    prometheus:
      enabled: false
    kiali:
      enabled: false
    grafana:
      enabled: false

After that everything worked. I tested DNSPolicy in this context also

[maleck13]

verfied all works with 2.6 slightly modified instructions to allow external traffic

@adam-cattermole we should probably have an issue for documenting setting up Kuadrant with OSSM 2.6 as a follow on