-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
🚀 Describe the new functionality needed
The meta-reference agents provider currently has a hard dependency on the Safety API during initialization, even though safety is only used at runtime when guardrails are explicitly configured. This prevents using the agents provider in configurations that don't include a safety provider (e.g., pure Ollama-only setups).
Current Behavior:
# src/llama_stack/providers/inline/agents/meta_reference/__init__.py:26
async def get_provider_impl(
config: MetaReferenceAgentsImplConfig,
deps: dict[Api, Any],
...
):
impl = MetaReferenceAgentsImpl(
config,
deps[Api.inference],
deps[Api.vector_io],
deps[Api.safety], # Required - KeyError if not present
deps[Api.tool_runtime],
...
)
The provider initialization fails with KeyError if Api.safety is not available in the deps dictionary, even if the user never intends to use guardrails:
# src/llama_stack/providers/inline/agents/meta_reference/responses/streaming.py:211
if self.guardrail_ids: # Only called when guardrails configured
violation = await run_guardrails(self.safety_api, ...)
Safety is only used conditionally when guardrail_ids is non-empty. If no guardrails are configured, the safety API is never called.
💡 Why is this needed? What if we don't build it?
This hard dependency blocks several use cases:
- Local-only LLM deployments (Ollama, llama.cpp, etc.) where users want to avoid external dependencies
- Development/testing environments where safety checks aren't needed
- Minimal configurations for resource-constrained environments
- Custom safety implementations that don't fit the standard safety API
We're trying to integrate Ollama support in lightspeed-core/lightspeed-stack#784 and hit this blocker:
Configuration that should work but fails
apis:
- agents
- inference # Using Ollama provider
# - safety # Disabled - not needed for our use case
providers:
agents:
- provider_id: meta-reference
# Fails during initialization with:
# KeyError: <Api.safety: 'safety'>
Result: Server starts successfully, readiness checks pass, but the agents provider fails to initialize because safety API is unavailable.
Other thoughts
Making this optional should probably require the user to explicitly opt-out
eg:
providers:
agents:
- provider_id: meta-reference
provider_type: inline::meta-reference
config:
require_safety_api: false # Explicit acknowledgment
.
.
So that we stick to the principle:
"Make the right thing easy and the wrong thing hard (but still possible with explicit intent)."
ps: require_safety_api is the best I could come up with for the name, so that require_safety_api=true default and require_safety_api=false explicit opt out is very clear in communicating what the user is choosing to do. Maybe there's a better option (since this will be an important API field).