Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update KServe component, Kserve guide and fix incorrect paths #266

Merged
merged 3 commits into from
Jun 22, 2022
Merged

Update KServe component, Kserve guide and fix incorrect paths #266

merged 3 commits into from
Jun 22, 2022

Conversation

surajkota
Copy link
Contributor

Description of your changes:

  • Kserve offers 2 installation options for serverless: standalone installation (in its own namespace) and a Kubeflow one. In KF-1.5 since both KFServing and KServe are included, Kserve standalone installation was used to avoid conflicts in resource names. But, in doing so the default ingress gateway configured became kntaive-serving/knative-ingress-gateway which does not exist in Kubeflow installation. Kubeflow integrated installation should use kubeflow/kubeflow-gateway. Overlay under awsconfig/apps/kserve fixes this and should eventually be merged upstream
  • Previous Kserve example fails to load the model because of incompatible libraries in container and hence had to update the sample used in guide
  • Fixed a bunch of path issues and outdated content and commands

Testing:

  • Manually tested Kserve guide on both dex and cognito setup
  • Tested all deployments build
kf) ubuntu@ip-172-31-0-119:~/kubeflow/kubeflow-manifests$ kustomize build deployments/vanilla/ -o /tmp/kf.yaml
2022/06/21 06:33:45 nil value at `valueFrom.configMapKeyRef.name` ignored in mutation attempt
2022/06/21 06:33:45 nil value at `valueFrom.secretKeyRef.name` ignored in mutation attempt
2022/06/21 06:33:45 well-defined vars that were never replaced: kfp-app-name,kfp-app-version
(kf) ubuntu@ip-172-31-0-119:~/kubeflow/kubeflow-manifests$ kustomize build deployments/cognito/ -o /tmp/kf.yaml
2022/06/21 06:33:55 nil value at `valueFrom.configMapKeyRef.name` ignored in mutation attempt
2022/06/21 06:33:55 nil value at `valueFrom.secretKeyRef.name` ignored in mutation attempt
2022/06/21 06:33:55 well-defined vars that were never replaced: kfp-app-version,clusterName,kfp-app-name
(kf) ubuntu@ip-172-31-0-119:~/kubeflow/kubeflow-manifests$ kustomize build deployments/cognito-rds-s3/ -o /tmp/kf.yaml
2022/06/21 06:34:03 nil value at `valueFrom.configMapKeyRef.name` ignored in mutation attempt
2022/06/21 06:34:03 nil value at `valueFrom.secretKeyRef.name` ignored in mutation attempt
2022/06/21 06:34:03 well-defined vars that were never replaced: clusterName,kfp-app-version,kfp-app-name
(kf) ubuntu@ip-172-31-0-119:~/kubeflow/kubeflow-manifests$ kustomize build deployments/rds-s3/ -o /tmp/kf.yaml
2022/06/21 06:35:28 nil value at `valueFrom.configMapKeyRef.name` ignored in mutation attempt
2022/06/21 06:35:28 nil value at `valueFrom.secretKeyRef.name` ignored in mutation attempt
2022/06/21 06:35:28 well-defined vars that were never replaced: kfp-app-name,kfp-app-version

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@surajkota surajkota merged commit eebda1c into awslabs:main Jun 22, 2022
@surajkota surajkota mentioned this pull request Aug 6, 2022
Merged
surajkota added a commit that referenced this pull request Aug 6, 2022
@surajkota
fix KServe ingress gateway (#311)
bac6429 
**Description of your changes:**
Fixing incomplete changes from #266

Changes in #266 are not sufficient because KServe is using 2 different configmaps(`kserve-config`,` inferenceservice-config`) to configure the gateway depending on the component. I am not certain why but here is what going on:

When an inferenceservice(isvc) is created, there are multiple virtualservices (VS) created. There is one VS for the inferenceservice and for each component inside ivsc like predictor, transformer etc. The ingress specified in the `inferenceservice-config` is configuring gateway used by the Virtual Service of the inference service itself and the ingress specified in `kserve-config`, is configuring the gateways used by the individual components inside the inferenceservice, e.g. predictor, transformer. E.g.

```
(kf) ubuntu@ip-172-31-0-119:~/kubeflow/temp/kubeflow-manifests/tests/e2e$ k get virtualservice -n kubeflow-user-example-com
NAME                                       GATEWAYS                                                                              HOSTS                                                                                                                                                                                                                                                                                                         AGE
sklearn-irisv2                             ["knative-serving/knative-local-gateway","knative-serving/knative-ingress-gateway"]   ["sklearn-irisv2.kubeflow-user-example-com.svc.cluster.local","sklearn-irisv2.kubeflow-user-example-com.cxdemo2.surakota.people.aws.dev"]                                                                                                                                                                     6h39m
sklearn-irisv2-predictor-default-ingress   ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2-predictor-default.kubeflow-user-example-com","sklearn-irisv2-predictor-default.kubeflow-user-example-com.cxdemo2.surakota.people.aws.dev","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc.cluster.local"]   6h39m
sklearn-irisv2-predictor-default-mesh      ["mesh"]                                                                              ["sklearn-irisv2-predictor-default.kubeflow-user-example-com","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc.cluster.local"] 
```
Notice the difference in gateway used by sklearn-irisv2 and sklearn-irisv2-predictor-default-ingress

The patch in #266, I missed the inferenceservice-config (I thought I patched both configs but there was a bug). Following is the output with this PR:

```
(kf) ubuntu@ip-172-31-0-119:~/kubeflow/kubeflow-manifests/tests/e2e$ k get virtualservice -n serving
NAME                                       GATEWAYS                                                                              HOSTS                                                                                                                                                                                                                                 AGE
sklearn-irisv2                             ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2.serving.svc.cluster.local","sklearn-irisv2.serving.cxdemo3.surakota.people.aws.dev"]                                                                                                                                 128m
sklearn-irisv2-predictor-default-ingress   ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2-predictor-default.serving","sklearn-irisv2-predictor-default.serving.cxdemo3.surakota.people.aws.dev","sklearn-irisv2-predictor-default.serving.svc","sklearn-irisv2-predictor-default.serving.svc.cluster.local"]   128m
sklearn-irisv2-predictor-default-mesh      ["mesh"]                                                                              ["sklearn-irisv2-predictor-default.serving","sklearn-irisv2-predictor-default.serving.svc","sklearn-irisv2-predictor-default.serving.svc.cluster.local"]
```

**Testing:**
- WIP: Manually testing the change on new EKS cluster for both Dex and Cognito setups

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rrrkharse rrrkharse rrrkharse approved these changes

@ryansteakley ryansteakley ryansteakley approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@surajkota @ryansteakley @rrrkharse