fix predictors ingress gateway

[surajkota]

Description of your changes:
Fixing incomplete changes from #266

Changes in #266 are not sufficient because KServe is using 2 different configmaps(kserve-config, inferenceservice-config) to configure the gateway depending on the component. I am not certain why but here is what going on:

When an inferenceservice(isvc) is created, there are multiple virtualservices (VS) created. There is one VS for the inferenceservice and for each component inside ivsc like predictor, transformer etc. The ingress specified in the inferenceservice-config is configuring gateway used by the Virtual Service of the inference service itself and the ingress specified in kserve-config, is configuring the gateways used by the individual components inside the inferenceservice, e.g. predictor, transformer. E.g.

(kf) ubuntu@ip-172-31-0-119:~/kubeflow/temp/kubeflow-manifests/tests/e2e$ k get virtualservice -n kubeflow-user-example-com
NAME                                       GATEWAYS                                                                              HOSTS                                                                                                                                                                                                                                                                                                         AGE
sklearn-irisv2                             ["knative-serving/knative-local-gateway","knative-serving/knative-ingress-gateway"]   ["sklearn-irisv2.kubeflow-user-example-com.svc.cluster.local","sklearn-irisv2.kubeflow-user-example-com.cxdemo2.surakota.people.aws.dev"]                                                                                                                                                                     6h39m
sklearn-irisv2-predictor-default-ingress   ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2-predictor-default.kubeflow-user-example-com","sklearn-irisv2-predictor-default.kubeflow-user-example-com.cxdemo2.surakota.people.aws.dev","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc.cluster.local"]   6h39m
sklearn-irisv2-predictor-default-mesh      ["mesh"]                                                                              ["sklearn-irisv2-predictor-default.kubeflow-user-example-com","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc","sklearn-irisv2-predictor-default.kubeflow-user-example-com.svc.cluster.local"] 

Notice the difference in gateway used by sklearn-irisv2 and sklearn-irisv2-predictor-default-ingress

The patch in #266, I missed the inferenceservice-config (I thought I patched both configs but there was a bug). Following is the output with this PR:

(kf) ubuntu@ip-172-31-0-119:~/kubeflow/kubeflow-manifests/tests/e2e$ k get virtualservice -n serving
NAME                                       GATEWAYS                                                                              HOSTS                                                                                                                                                                                                                                 AGE
sklearn-irisv2                             ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2.serving.svc.cluster.local","sklearn-irisv2.serving.cxdemo3.surakota.people.aws.dev"]                                                                                                                                 128m
sklearn-irisv2-predictor-default-ingress   ["knative-serving/knative-local-gateway","kubeflow/kubeflow-gateway"]                 ["sklearn-irisv2-predictor-default.serving","sklearn-irisv2-predictor-default.serving.cxdemo3.surakota.people.aws.dev","sklearn-irisv2-predictor-default.serving.svc","sklearn-irisv2-predictor-default.serving.svc.cluster.local"]   128m
sklearn-irisv2-predictor-default-mesh      ["mesh"]                                                                              ["sklearn-irisv2-predictor-default.serving","sklearn-irisv2-predictor-default.serving.svc","sklearn-irisv2-predictor-default.serving.svc.cluster.local"]

Testing:

• Manually testing the change on new EKS cluster for both Dex and Cognito setups

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[rrrkharse]

Tried kustomize build to verify the config map gets modified correctly and it worked for me.