Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add AI plugin #10627

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open

Add AI plugin #10627

wants to merge 29 commits into from

Conversation

npolshakova
Copy link
Contributor

@npolshakova npolshakova commented Feb 14, 2025

Description

Adds Upstream and RoutePolicy plugin implementations. Requires #10493 and #10495 to merge first.

API changes

NONE (All API changes are in #10493)

Code changes

  • Adds AI Upstream translation to upstream plugin
  • Adds RoutePolicy translation to route policy plugin
  • Introduces new ApplyBackend() which will apply route/typed filter config to backend regardless if policy is attached.

CI changes

NONE

Docs changes

Tracked separately in #10631

Context

See EP: #10495

Interesting decisions

  • For the initial scope, limited support to extensionRef filter reference (no targetRef support) to simplify the assertion the backendRef the route policy is being attached to is an AI Upstream.

Testing steps

  • Adds new ggv2setup_tests for passthrough, inline token, and secretRef (with different providers). These test cases also cover configuring route policies with defaults, prompt enrichment, prompt guard and streaming.

Notes for reviewers

TODO: add steps for reproducing locally

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works

@andy-fong
Copy link
Contributor

It will be easier to read if this is set to merge to npolshakova:port-ai-apis-to-kgateway

@npolshakova
Copy link
Contributor Author

It will be easier to read if this is set to merge to npolshakova:port-ai-apis-to-kgateway

Yep, rebased but the PRs are both in my fork. Here's the cleaner diff: npolshakova#1

@npolshakova npolshakova marked this pull request as ready for review February 19, 2025 20:04
@npolshakova npolshakova added the work in progress signals bulldozer to keep pr open (don't auto-merge) label Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
work in progress signals bulldozer to keep pr open (don't auto-merge)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants