Skip to content

Commit

Permalink
fix KServe ingress gateway (#311)
Browse files Browse the repository at this point in the history
**Description of your changes:**
Fixing incomplete changes from #266

Changes in #266 are not sufficient because KServe is using 2 different configmaps(`kserve-config`,` inferenceservice-config`) to configure the gateway depending on the component. I am not certain why but here is what going on:

When an inferenceservice(isvc) is created, there are multiple virtualservices (VS) created. There is one VS for the inferenceservice and for each component inside ivsc like predictor, transformer etc. The ingress specified in the `inferenceservice-config` is configuring gateway used by the Virtual Service of the inference service itself and the ingress specified in `kserve-config`, is configuring the gateways used by the individual components inside the inferenceservice, e.g. predictor, transformer. E.g.

```
(kf) ubuntu@ip-172-31-0-119:~/kubeflow/temp/kubeflow-manifests/tests/e2e$ k get virtualservice -n kubeflow-user-example-com
NAME                                       GATEWAYS                                                                              HOSTS                                                                                                                                                                                                                                                                                                         AGE
sklearn-irisv2                             ["knative-serving/knative-local-gateway","knative-serving/knative-ingress-gateway"]   ["sklearn-irisv2.kubeflow-user-example-com.svc.cluster.local","sklearn-irisv2.kubeflow-user-example-com.cxdemo2.surakota.people.aws.dev"]                                                                                                                                                                     6h39m
sklearn-irisv2-predictor-default-ingress   ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2-predictor-default.kubeflow-user-example-com","sklearn-irisv2-predictor-default.kubeflow-user-example-com.cxdemo2.surakota.people.aws.dev","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc.cluster.local"]   6h39m
sklearn-irisv2-predictor-default-mesh      ["mesh"]                                                                              ["sklearn-irisv2-predictor-default.kubeflow-user-example-com","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc.cluster.local"] 
```
Notice the difference in gateway used by sklearn-irisv2 and sklearn-irisv2-predictor-default-ingress

The patch in #266, I missed the inferenceservice-config (I thought I patched both configs but there was a bug). Following is the output with this PR:

```
(kf) ubuntu@ip-172-31-0-119:~/kubeflow/kubeflow-manifests/tests/e2e$ k get virtualservice -n serving
NAME                                       GATEWAYS                                                                              HOSTS                                                                                                                                                                                                                                 AGE
sklearn-irisv2                             ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2.serving.svc.cluster.local","sklearn-irisv2.serving.cxdemo3.surakota.people.aws.dev"]                                                                                                                                 128m
sklearn-irisv2-predictor-default-ingress   ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2-predictor-default.serving","sklearn-irisv2-predictor-default.serving.cxdemo3.surakota.people.aws.dev","sklearn-irisv2-predictor-default.serving.svc","sklearn-irisv2-predictor-default.serving.svc.cluster.local"]   128m
sklearn-irisv2-predictor-default-mesh      ["mesh"]                                                                              ["sklearn-irisv2-predictor-default.serving","sklearn-irisv2-predictor-default.serving.svc","sklearn-irisv2-predictor-default.serving.svc.cluster.local"]
```

**Testing:**
- WIP: Manually testing the change on new EKS cluster for both Dex and Cognito setups

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
  • Loading branch information
surajkota authored Aug 6, 2022
1 parent fd77059 commit bac6429
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 1 deletion.
7 changes: 7 additions & 0 deletions awsconfigs/apps/kserve/ingress
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"ingressGateway" : "kubeflow/kubeflow-gateway",
"ingressService" : "istio-ingressgateway.istio-system.svc.cluster.local",
"localGateway" : "knative-serving/knative-local-gateway",
"localGatewayService" : "knative-local-gateway.istio-system.svc.cluster.local",
"ingressDomain" : "example.com"
}
7 changes: 6 additions & 1 deletion awsconfigs/apps/kserve/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,9 @@ configMapGenerator:
namespace: kserve
behavior: merge
envs:
- params.env
- params.env
- name: inferenceservice-config
namespace: kserve
behavior: merge
files:
- ingress

0 comments on commit bac6429

Please sign in to comment.