Skip to content

LCORE-1335: Parametrized RHAIIS port and added library mode tests #1164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tisnik merged 1 commit into from
Feb 18, 2026
Merged

LCORE-1335: Parametrized RHAIIS port and added library mode tests #1164

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 60 additions & 43 deletions .github/workflows/e2e_tests_rhaiis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,18 @@ jobs:
e2e_tests:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
mode: ["server", "library"]
environment: [ "rhaiis" ]

name: "RHAIIS E2E: ${{ matrix.mode }} mode / ${{ matrix.environment }}"

env:
E2E_DEPLOYMENT_MODE: ${{ matrix.mode }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
RHAIIS_URL: ${{ secrets.RHAIIS_URL }}
RHAIIS_PORT: ${{ secrets.RHAIIS_PORT }}
RHAIIS_API_KEY: ${{ secrets.RHAIIS_API_KEY }}
RHAIIS_MODEL: ${{ vars.RHAIIS_MODEL }}
FAISS_VECTOR_STORE_ID: ${{ vars.FAISS_VECTOR_STORE_ID }}
Expand Down Expand Up @@ -47,37 +54,18 @@ jobs:
echo "=== Recent commits (should show setup-metrics commits) ==="
git log --oneline -5

- uses: 1arp/create-a-file-action@0.4.5
with:
path: '.'
isAbsolutePath: false
file: 'lightspeed-stack.yaml'
content: |
name: Lightspeed Core Service (LCS)
service:
host: 0.0.0.0
port: 8080
auth_enabled: false
workers: 1
color_log: true
access_log: true
llama_stack:
# Uses a remote llama-stack service
# The instance would have already been started with a llama-stack-run.yaml file
use_as_library_client: false
# Alternative for "as library use"
# use_as_library_client: true
# library_client_config_path: <path-to-llama-stack-run.yaml-file>
url: http://llama-stack:8321
api_key: xyzzy
user_data_collection:
feedback_enabled: true
feedback_storage: "/tmp/data/feedback"
transcripts_enabled: true
transcripts_storage: "/tmp/data/transcripts"

authentication:
module: "noop"
- name: Load lightspeed-stack.yaml configuration
run: |
MODE="${{ matrix.mode }}"
CONFIG_FILE="tests/e2e/configuration/${MODE}-mode/lightspeed-stack.yaml"
echo "Loading configuration for ${MODE} mode from ${CONFIG_FILE}"

if [ ! -f "${CONFIG_FILE}" ]; then
echo "Configuration file not found: ${CONFIG_FILE}"
exit 1
fi

cp "${CONFIG_FILE}" lightspeed-stack.yaml

- name: Select and configure run.yaml
env:
Expand Down Expand Up @@ -121,20 +109,22 @@ jobs:

- name: Test RHAIIS connectivity
run: |
curl -f ${RHAIIS_URL}:8000/v1/models -H "Authorization: Bearer ${RHAIIS_API_KEY}"
curl -f ${RHAIIS_URL}:${RHAIIS_PORT}/v1/models -H "Authorization: Bearer ${RHAIIS_API_KEY}"
Copy link
Contributor

@coderabbitai coderabbitai bot Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Quote the URL in the connectivity check to prevent shell word-splitting.

If RHAIIS_URL includes a protocol prefix (e.g., https://host) or any special characters, the unquoted expansion could cause unexpected behavior.

Proposed fix 
-          curl -f ${RHAIIS_URL}:${RHAIIS_PORT}/v1/models   -H "Authorization: Bearer ${RHAIIS_API_KEY}"  
+          curl -f "${RHAIIS_URL}:${RHAIIS_PORT}/v1/models" -H "Authorization: Bearer ${RHAIIS_API_KEY}"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
curl -f ${RHAIIS_URL}:${RHAIIS_PORT}/v1/models -H "Authorization: Bearer ${RHAIIS_API_KEY}"
curl -f "${RHAIIS_URL}:${RHAIIS_PORT}/v1/models" -H "Authorization: Bearer ${RHAIIS_API_KEY}"
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/e2e_tests_rhaiis.yaml at line 112, The curl connectivity
check currently uses unquoted variable expansion which can cause shell
word-splitting when RHAIIS_URL contains special characters; update the command
to quote the URL expansion (e.g., replace curl -f
${RHAIIS_URL}:${RHAIIS_PORT}/v1/models with curl -f
"${RHAIIS_URL}:${RHAIIS_PORT}/v1/models") and keep the Authorization header
using ${RHAIIS_API_KEY} as before to ensure the full URL is passed as a single
argument.


- name: Docker Login for quay access
if: matrix.mode == 'server'
env:
QUAY_ROBOT_USERNAME: ${{ secrets.QUAY_DOWNSTREAM_USERNAME }}
QUAY_ROBOT_TOKEN: ${{ secrets.QUAY_DOWNSTREAM_TOKEN }}
run: |
echo $QUAY_ROBOT_TOKEN | docker login quay.io -u=$QUAY_ROBOT_USERNAME --password-stdin

- name: Run service manually
run: |
- name: Run services (Server Mode)
if: matrix.mode == 'server'
run: |
docker compose version
docker compose up -d

# Check for errors and show logs if any services failed
if docker compose ps | grep -E 'Exit|exited|stopped'; then
echo "Some services failed to start - showing logs:"
Expand All @@ -144,6 +134,20 @@ jobs:
echo "All services started successfully"
fi

- name: Run services (Library Mode)
if: matrix.mode == 'library'
run: |
echo "Starting service in library mode (1 container)"
docker compose -f docker-compose-library.yaml up -d

if docker compose -f docker-compose-library.yaml ps | grep -E 'Exit|exited|stopped'; then
echo "Service failed to start - showing logs:"
docker compose -f docker-compose-library.yaml logs
exit 1
else
echo "Service started successfully"
fi

- name: Wait for services
run: |
echo "Waiting for services to be healthy..."
Expand All @@ -153,12 +157,20 @@ jobs:
run: |
echo "Testing basic connectivity before full test suite..."
curl -f http://localhost:8080/v1/models || {
echo "❌ Basic connectivity failed - showing logs before running full tests"
docker compose logs --tail=30
echo "Basic connectivity failed - showing logs"
if [ "${{ matrix.mode }}" == "server" ]; then
docker compose logs --tail=30
else
docker compose -f docker-compose-library.yaml logs --tail=30
fi
exit 1
}

- name: Run e2e tests
env:
TERM: xterm-256color
FORCE_COLOR: 1
E2E_DEPLOYMENT_MODE: ${{ matrix.mode }}
run: |
echo "Installing test dependencies..."
pip install uv
Expand All @@ -171,9 +183,14 @@ jobs:
if: failure()
run: |
echo "=== Test failure logs ==="
echo "=== llama-stack logs ==="
docker compose logs llama-stack

echo ""
echo "=== lightspeed-stack logs ==="
docker compose logs lightspeed-stack

if [ "${{ matrix.mode }}" == "server" ]; then
echo "=== llama-stack logs ==="
docker compose logs llama-stack
echo ""
echo "=== lightspeed-stack logs ==="
docker compose logs lightspeed-stack
else
echo "=== lightspeed-stack (library mode) logs ==="
docker compose -f docker-compose-library.yaml logs lightspeed-stack
fi
1 change: 1 addition & 0 deletions docker-compose-library.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ services:
- CLIENT_SECRET=${CLIENT_SECRET:-}
# RHAIIS
- RHAIIS_URL=${RHAIIS_URL:-}
- RHAIIS_PORT=${RHAIIS_PORT:-}
- RHAIIS_API_KEY=${RHAIIS_API_KEY:-}
- RHAIIS_MODEL=${RHAIIS_MODEL:-}
# RHEL AI
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ services:
- CLIENT_SECRET=${CLIENT_SECRET:-}
# RHAIIS
- RHAIIS_URL=${RHAIIS_URL}
- RHAIIS_PORT=${RHAIIS_PORT}
- RHAIIS_API_KEY=${RHAIIS_API_KEY}
- RHAIIS_MODEL=${RHAIIS_MODEL}
# RHEL AI
Expand Down
11 changes: 5 additions & 6 deletions tests/e2e/configs/run-rhaiis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,10 @@ providers:
- provider_id: vllm
provider_type: remote::vllm
config:
url: http://${env.RHAIIS_URL}:8000/v1/
base_url: http://${env.RHAIIS_URL}:${env.RHAIIS_PORT}/v1/
Copy link
Contributor

@coderabbitai coderabbitai bot Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check how base_url is used in remote::vllm provider configuration across the codebase
rg -n "base_url" --type yaml -C2 --glob '**/run-*.yaml'
echo "---"
# Check if other config files use 'url' or 'base_url' for vllm
rg -n "provider_type: remote::vllm" -A5 --type yaml

Repository: lightspeed-core/lightspeed-stack

Length of output: 2727

🏁 Script executed:

#!/bin/bash
# Search for vllm provider implementation or config schema
fd -type f -name "*.py" | xargs rg -l "remote::vllm" 2>/dev/null | head -10
echo "---"
# Also search for vllm config handling
rg -n "vllm" --type py -B2 -A5 | grep -E "(base_url|url|config)" | head -30

Repository: lightspeed-core/lightspeed-stack

Length of output: 497

🏁 Script executed:

#!/bin/bash
# Search for vllm config handling in Python files
rg -n "url|base_url" --type py -B3 -A3 | grep -i vllm -B5 -A5 | head -50

echo "---"
# Look for config class definitions or dataclass patterns
rg -n "class.*vllm|@dataclass" --type py -i | head -20

echo "---"
# Search for any schema or type definitions
fd -type f -name "*.py" -path "*vllm*" 2>/dev/null

echo "---"
# Check for llama-stack requirements or dependencies
cat requirements.txt 2>/dev/null | grep -i llama
cat pyproject.toml 2>/dev/null | grep -i llama

Repository: lightspeed-core/lightspeed-stack

Length of output: 719

🏁 Script executed:

#!/bin/bash
# Search for any documentation about vllm provider config in the repo
rg -n "vllm" -i docs/ --type md -B3 -A3 2>/dev/null | head -50

echo "---"
# Check for any schema or type hints in the repo
rg -n "url.*vllm|vllm.*url" -i --type py -B2 -A2 | head -40

Repository: lightspeed-core/lightspeed-stack

Length of output: 2809

🌐 Web query:

llama-stack 0.4.3 vllm provider base_url vs url configuration schema

💡 Result:

In llama-stack 0.4.3, the remote::vllm inference provider config schema uses url (not base_url).

remote::vllm config fields (schema)

  • url: str | None — vLLM serving endpoint URL
  • max_tokens: int (default 4096)
  • api_token: str | None (default "fake")
  • tls_verify: bool | str (default True)
  • refresh_models: bool (default False)  [1]

Example run.yaml

providers:
  inference:
    - provider_id: vllm0
      provider_type: remote::vllm
      config:
        url: ${env.VLLM_URL:=http://localhost:8000/v1}

The official examples also use config.url. [1], [2]

Where base_url does show up

  • Client-side (e.g., LlamaStackClient(base_url="http://localhost:8321")) for talking to the Llama Stack server, not for configuring the vLLM provider. [3]

So: if you’re seeing a base_url vs url mismatch, for provider_type: remote::vllm you should configure config.url. [1]

Sources: [1] [2] [3]

Change base_url to url — this is the wrong field name for the vllm provider schema.

llama-stack 0.4.3's remote::vllm provider config expects url, not base_url. All other vllm configurations in the repository (run-rhelai.yaml, examples/vllm-*.yaml, e2e-prow/rhoai/configs/run.yaml) consistently use url. The base_url field does not exist in the vllm provider schema and will cause configuration parsing to fail.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@tests/e2e/configs/run-rhaiis.yaml` at line 25, The YAML for the vllm provider
uses the wrong field name; in the remote::vllm provider config replace the key
base_url with url so the provider schema matches llama-stack 0.4.3 (i.e., change
the base_url entry to url in the run-rhaiis.yaml vllm config to match other
configs like run-rhelai.yaml and examples/vllm-*.yaml).

api_token: ${env.RHAIIS_API_KEY}
tls_verify: false
max_tokens: 2048
- provider_id: openai
provider_type: remote::openai
config:
api_key: ${env.OPENAI_API_KEY}
- config: {}
provider_id: sentence-transformers
provider_type: inline::sentence-transformers
Expand All @@ -54,6 +50,9 @@ providers:
- config: {}
provider_id: rag-runtime
provider_type: inline::rag-runtime
- config: {} # Enable MCP (Model Context Protocol) support
provider_id: model-context-protocol
provider_type: remote::model-context-protocol
vector_io:
- config:
persistence:
Expand Down Expand Up @@ -143,7 +142,7 @@ registered_resources:
shields:
- shield_id: llama-guard
provider_id: llama-guard
provider_shield_id: openai/gpt-4o-mini
provider_shield_id: vllm/${env.RHAIIS_MODEL}
vector_stores:
- embedding_dimension: 768
embedding_model: sentence-transformers/all-mpnet-base-v2
Expand Down
Loading