[RHOAIENG-5073] - Routing and Headless Service Support in KServe Raw …

[spolti]

…Mode Deployment

What this PR does / why we need it:

Raw Deployments need to be fronted by a routing component. Currently, the FMaaS/Rust router (and Caikit) client-side load balance and proxy requests across a model deployment's pods/replica's. To do so they utilizes a Headless service that sits in between itself and the replica's, queries addresses to the physical pods, and round robins requests.

Type of changes
Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Release note:

NONE

[spolti]

The raw succeeded here #272
Wondering what might be happening.
Does the logging behaves differently when using ClusterIP: None?

@terrytangyuan @Jooho

[spolti]

/retest e2e-raw

[openshift-ci]

@spolti: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

• /test e2e-fast
• /test e2e-raw
• /test e2e-slow
• /test images
• /test pr-image-mirror-kserve-agent
• /test pr-image-mirror-kserve-controller
• /test pr-image-mirror-kserve-router
• /test pr-image-mirror-kserve-storage-initializer

Use /test all to run all jobs.

In response to this:

/retest e2e-raw

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[spolti]

/test e2e-raw

[terrytangyuan]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: spolti, terrytangyuan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [spolti,terrytangyuan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment