[AI-194] Openrouter

[tschellenbach]

Openrouter support. Uses openAI SDK but disables the usage of openAI conversation storage.

Summary by CodeRabbit

• New Features

  • Added OpenRouter LLM plugin for vision agents with conversation compatibility.
  • Improved conversation handling for OpenAI-based LLMs to ensure consistent conversation setup.

• Documentation

  • Added OpenRouter plugin README and a runnable example demonstrating integration.

• Tests

  • Added tests for OpenRouter LLM covering normalization, streaming, and memory scenarios.

• Chores

  • Added plugin packaging/config files and enabled lint autofix in the development check flow.

[coderabbitai]

Walkthrough

Adds a new OpenRouter LLM plugin with docs, example, tests, and packaging; refactors OpenAI LLM to centralize conversation helpers; and updates the development lint check to run Ruff with --fix.

Changes

Cohort / File(s)	Summary
Development workflow dev.py	Added --fix flag to the Ruff lint command in check() to auto-apply fixes.
OpenAI LLM enhancements plugins/openai/vision_agents/plugins/openai/openai_llm.py	Added create_conversation() and add_conversation_history(kwargs) helper methods; updated create_response() to call these helpers and centralize conversation setup.
OpenRouter plugin core plugins/openrouter/pyproject.toml, plugins/openrouter/vision_agents/plugins/openrouter/__init__.py, plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py	New plugin packaging, public LLM export, and OpenRouterLLM class (env-driven api_key, base_url, model defaults, manual conversation-history handling).
OpenRouter docs & example plugins/openrouter/README.md, plugins/openrouter/example/openrouter_example.py, plugins/openrouter/example/pyproject.toml	Added README, a runnable example start_agent() demonstrating OpenRouter usage with streaming/TTS/STT/turn-detection, and example project config for local editable installs.
OpenRouter tests plugins/openrouter/tests/test_openrouter_llm.py	New test suite covering message normalization (including images), streaming, memory behaviors, and integration guarded by OPENROUTER_API_KEY.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Caller
    participant OpenAILLM
    participant OpenRouterLLM
    participant RemoteAPI
    rect rgb(240,248,255)
    Note over OpenAILLM,OpenRouterLLM: Conversation setup helpers (new)
    end
    Caller->>OpenAILLM: create_response(kwargs)
    OpenAILLM->>OpenAILLM: create_conversation() (ensure)
    OpenAILLM->>OpenAILLM: add_conversation_history(kwargs)
    alt Using OpenRouterLLM
        Caller->>OpenRouterLLM: create_response(kwargs)
        OpenRouterLLM->>OpenRouterLLM: create_conversation() (no-op)
        OpenRouterLLM->>OpenRouterLLM: add_conversation_history(kwargs) (normalize/store)
        OpenRouterLLM->>RemoteAPI: POST request with updated kwargs
        RemoteAPI-->>OpenRouterLLM: response
        OpenRouterLLM-->>Caller: return response
    else Using OpenAILLM (legacy backend)
        OpenAILLM->>RemoteAPI: POST request with conversation id in kwargs
        RemoteAPI-->>OpenAILLM: response/events
        OpenAILLM-->>Caller: return/stream response
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• fix: Agent Example and TURN detection #70 — Modifies create_response in openai_llm.py; shares overlap with the conversation helper refactor and may conflict or complement event/response handling.

Suggested reviewers

• maxkahan

Poem

The room tightens like a throat at dusk,
Two small hands build a ledger of speech,
Threads of memory stitched into a white mouth,
OpenRouter knocks with a bright, brittle key,
The agent answers; the bell keeps counting.

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings, 1 inconclusive)

Check name	Status	Explanation	Resolution
Linked Issues Check	⚠️ Warning	The pull request successfully integrates OpenRouter support by adding the plugin, packaging, example, and tests, but it does not include any demonstration or configuration for the Qwen VL model referenced in issue AI-194; the example script is configured to use “openai/gpt-4o” rather than a Qwen VL model. Since the linked issue explicitly calls for both Openrouter support and a Qwen VL demo, the absence of the Qwen VL component means the objectives are only partially met.	Add a demonstration of the Qwen VL model (e.g., “qwen3-vl-235b-a22b-thinking”) in the example script and update tests or documentation to showcase the specific Qwen VL workflow as required by issue AI-194.
Out of Scope Changes Check	⚠️ Warning	Changes to dev.py’s lint command and additions to the OpenAILLM helper methods are unrelated to integrating Openrouter support or providing a Qwen VL demo, indicating that development workflow refinements and OpenAI-specific conversation logic have been mixed into this feature-focused pull request. These updates fall outside the scope defined by issue AI-194 and introduce unrelated modifications in the same branch.	Separate the lint configuration change and the OpenAILLM helpers into distinct pull requests or branches so that this PR remains focused solely on the Openrouter plugin integration and Qwen VL demonstration.
Title Check	❓ Inconclusive	The title “[AI-194] Openrouter” includes the linked issue identifier and the plugin name but does not describe the change being introduced or provide context on the integration or demo added, making it too generic to convey the primary purpose of the pull request. It is related to the changeset but lacks clarity about what was implemented or improved. A more descriptive title would help teammates immediately understand the scope without inspecting the diff.	Update the title to clearly summarize the main change, for example “Integrate OpenRouter LLM support and add Qwen VL demo (AI-194),” so that it concisely reflects both the new plugin integration and the example usage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	Docstring coverage is 90.91% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch openrouter

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

dev.py (1)

11-12: Fix run() return typing and docstring to match behavior; keep to Google style.

When check=False and the command fails, the function returns CalledProcessError, but the return type only declares CompletedProcess. Also, add Args/Returns/Raises per Google style.

Apply this diff:

-from typing import Optional
+from typing import Optional, Mapping

@@
-def run(
-    command: str, env: Optional[dict] = None, check: bool = True
-) -> subprocess.CompletedProcess:
-    """Run a shell command with automatic argument parsing."""
+def run(
+    command: str,
+    env: Optional[Mapping[str, str]] = None,
+    check: bool = True,
+) -> subprocess.CompletedProcess | subprocess.CalledProcessError:
+    """Run a shell command with automatic argument parsing.
+
+    Args:
+      command: Command string to execute (parsed with shlex, no shell).
+      env: Extra environment variables to merge into the current environment.
+      check: If True, exit the process on non‑zero return code; otherwise return the error.
+
+    Returns:
+      A CompletedProcess on success, or a CalledProcessError if `check` is False and the command fails.
+
+    Raises:
+      SystemExit: If `check` is True and the command exits with a non‑zero status.
+    """

Also applies to: 16-38

🧹 Nitpick comments (12)

dev.py (2)

103-106: Make ruff auto-fix opt-in for check(); full checks should be non-mutating by default.

ruff check . --fix rewrites files during a "full check" run. That’s surprising and risky if contributors expect a read-only verification step. Recommend gating fixes behind a --fix flag.

Apply this diff:

 @cli.command()
-def check():
-    """Run full check: ruff, mypy, and unit tests."""
-    click.echo("Running full development check...")
+@click.option("--fix/--no-fix", default=False, help="Apply auto-fixes during ruff lint.")
+def check(fix: bool = False):
+    """Run full check: ruff, mypy, and unit tests.
+
+    Args:
+      fix: If True, run ruff with --fix to apply auto-fixes.
+    """
+    click.echo("Running full development check...")

@@
-    run("uv run ruff check . --fix")
+    run(f"uv run ruff check .{' --fix' if fix else ''}")

---

70-74: Use ruff format for the formatter; current command runs the linter.

The function is named/echoed as “format” but executes ruff check --fix. Switch to the dedicated formatter for clarity.

Apply this diff:

 def format():
-    """Run ruff formatting with auto-fix."""
+    """Run ruff formatter."""
     click.echo("Running ruff format...")
-    run("uv run ruff check --fix")
+    run("uv run ruff format .")

plugins/openrouter/README.md (1)

34-34: Explicit API key guidance and inline example.

Spell out the exact env var name and show direct-constructor usage so users don’t hunt around.

Apply:

-This plugin uses the OpenAI-compatible API provided by OpenRouter. You'll need to set your OpenRouter API key as an environment variable or pass it directly to the LLM.
+This plugin uses the OpenAI‑compatible API provided by OpenRouter. Set your API key as OPENROUTER_API_KEY or pass it directly:
+
+```bash
+export OPENROUTER_API_KEY=sk-or-...
+```
+
+```python
+llm = openrouter.LLM(model="anthropic/claude-haiku-4.5", api_key="sk-or-...")
+```

plugins/openrouter/example/pyproject.toml (1)

6-15: Pin examples via a lockfile; keep pyproject lean.

Examples pull latest of many plugins; reproducibility will vary across machines. Consider adding a uv lock (uv pip compile) and keeping pyproject minimal.

plugins/openrouter/pyproject.toml (2)

26-28: Avoid shipping tests/examples in the wheel.

packages = ["."] will include tests and the example folder. Limit to the actual package path.

Apply:

-[tool.hatch.build.targets.wheel]
-packages = ["."]
+[tool.hatch.build.targets.wheel]
+# Only ship the library code
+packages = ["vision_agents"]
+
+# Optionally exclude non-package dirs defensively
+[tool.hatch.build.targets.wheel.force-include]
+# none

---

12-15: Transitive dependency is fine but consider an explicit lower bound.

vision-agents-plugins-openai is required. Add a minimal compatible version once CI proves it, to prevent accidental breaking upgrades.

plugins/openrouter/example/openrouter_example.py (1)

18-24: Docstring: tighten to Google style.

Add Args/Returns for clarity and to match repo guidelines.

Apply:

-async def start_agent() -> None:
-    """Example agent using OpenRouter LLM.
-    
-    This example demonstrates how to use the OpenRouter plugin with a Vision Agent.
-    OpenRouter provides access to multiple LLM providers through a unified API.
-    
-    Set OPENROUTER_API_KEY environment variable before running.
-    """
+async def start_agent() -> None:
+    """Run an example agent using the OpenRouter LLM.
+
+    Args:
+        None
+
+    Returns:
+        None
+
+    Notes:
+        Set OPENROUTER_API_KEY before running. OpenRouter routes to multiple LLM providers.
+    """

plugins/openrouter/tests/test_openrouter_llm.py (1)

90-107: Name nit: use snake_case for locals.

streamingWorks -> streaming_works.

Apply:

-        streamingWorks = False
+        streaming_works = False
@@
-            nonlocal streamingWorks
-            streamingWorks = True
+            nonlocal streaming_works
+            streaming_works = True
@@
-        assert streamingWorks, "Streaming should have generated chunk events"
+        assert streaming_works, "Streaming should have generated chunk events"

plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (4)

34-41: Fail fast if API key is missing (clear UX).

Without a key, requests will 401 later. Raise early with a helpful message.

Apply:

         if api_key is None:
             api_key = os.environ.get("OPENROUTER_API_KEY")
+        if not api_key:
+            raise ValueError(
+                "OpenRouter API key not found. Set OPENROUTER_API_KEY or pass api_key."
+            )

---

43-46: Add a short docstring and spell “don’t”.

Minor polish; also clarifies why we no-op.

Apply:

-    async def create_conversation(self):
-        # Do nothing, dont call super
+    async def create_conversation(self):
+        """OpenRouter does not use provider-side conversation IDs."""
+        # Do nothing, don't call super()
         pass

---

47-61: Conversation growth can explode token usage; cap and include assistant turns.

• Each call prepends the entire history, which will grow unbounded.
• Ensure assistant replies are also stored (if base class doesn’t already) so the model sees both sides.
• Optionally set OpenRouter headers for better routing/quotas.

Apply:

-    def add_conversation_history(self, kwargs):
+    def add_conversation_history(self, kwargs):
         # Use the manual storage
         # ensure the AI remembers the past conversation
         # TODO: there are additional formats to support here.
         new_messages = kwargs["input"]
         if not isinstance(new_messages, list):
             new_messages = [dict(content=new_messages, role="user", type="message")]
-        if hasattr(self, '_conversation') and self._conversation:
-            old_messages = [m.original for m in self._conversation.messages]
-            kwargs["input"] = old_messages + new_messages
+        if hasattr(self, "_conversation") and self._conversation:
+            # Optional: cap history to last N messages to control token cost
+            HISTORY_LIMIT = 40
+            old_messages = [m.original for m in self._conversation.messages][-HISTORY_LIMIT:]
+            kwargs["input"] = old_messages + new_messages
             # Add messages to conversation
             normalized_messages = self._normalize_message(new_messages)
             for msg in normalized_messages:
                 self._conversation.messages.append(msg)

If assistant replies aren’t appended elsewhere, consider appending them after create_response completes. I can sketch that hook if needed. Based on learnings.

---

18-25: Optional: set OpenRouter identifying headers.

OpenRouter recommends HTTP-Referer and X-Title for better routing/quota accounting.

Follow-up (in OpenAILLM init path) after super().init:

# Example:
if hasattr(self, "client"):
    self.client.default_headers = {
        **getattr(self.client, "default_headers", {}) or {},
        "HTTP-Referer": os.getenv("OPENROUTER_REFERRER", "https://visionagents.ai"),
        "X-Title": os.getenv("OPENROUTER_TITLE", "Vision Agents"),
    }

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 3d06446 and ba6c027.

📒 Files selected for processing (9)

• dev.py (1 hunks)
• plugins/openai/vision_agents/plugins/openai/openai_llm.py (2 hunks)
• plugins/openrouter/README.md (1 hunks)
• plugins/openrouter/example/openrouter_example.py (1 hunks)
• plugins/openrouter/example/pyproject.toml (1 hunks)
• plugins/openrouter/pyproject.toml (1 hunks)
• plugins/openrouter/tests/test_openrouter_llm.py (1 hunks)
• plugins/openrouter/vision_agents/plugins/openrouter/__init__.py (1 hunks)
• plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

📄 CodeRabbit inference engine (.cursor/rules/python.mdc)

**/*.py: Do not modify sys.path in Python code
Docstrings must follow the Google style guide

Files:

• plugins/openrouter/vision_agents/plugins/openrouter/__init__.py
• plugins/openai/vision_agents/plugins/openai/openai_llm.py
• plugins/openrouter/tests/test_openrouter_llm.py
• plugins/openrouter/example/openrouter_example.py
• dev.py
• plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py

🧬 Code graph analysis (5)

plugins/openrouter/vision_agents/plugins/openrouter/__init__.py (2)

plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (1)

• OpenRouterLLM (8-60)

agents-core/vision_agents/core/llm/llm.py (1)

• LLM (36-363)

plugins/openai/vision_agents/plugins/openai/openai_llm.py (2)

plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (2)

• create_conversation (43-45)
• add_conversation_history (47-60)

plugins/getstream/vision_agents/plugins/getstream/stream_edge_transport.py (1)

• create_conversation (224-231)

plugins/openrouter/tests/test_openrouter_llm.py (4)

agents-core/vision_agents/core/agents/conversation.py (1)

• InMemoryConversation (230-237)

agents-core/vision_agents/core/llm/events.py (1)

• LLMResponseChunkEvent (90-105)

plugins/openai/vision_agents/plugins/openai/openai_llm.py (3)

• _normalize_message (351-369)
• simple_response (80-108)
• create_response (118-214)

agents-core/vision_agents/core/events/manager.py (1)

• wait (470-484)

plugins/openrouter/example/openrouter_example.py (2)

agents-core/vision_agents/core/edge/types.py (1)

• User (15-18)

agents-core/vision_agents/core/agents/agents.py (2)

• Agent (56-1050)
• finish (410-430)

plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py (2)

agents-core/vision_agents/core/llm/llm.py (1)

• LLM (36-363)

plugins/openai/vision_agents/plugins/openai/openai_llm.py (4)

• OpenAILLM (31-518)
• create_conversation (110-112)
• add_conversation_history (114-116)
• _normalize_message (351-369)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

• GitHub Check: unit / Ruff & mypy
• GitHub Check: unit / Test "not integration"
• GitHub Check: unit / Ruff & mypy

🔇 Additional comments (4)

plugins/openrouter/example/pyproject.toml (1)

17-26: Workspace sources look good.

Editable sources match the repo layout and should speed local dev.

plugins/openrouter/vision_agents/plugins/openrouter/__init__.py (1)

2-4: Clean public surface.

Alias export is clear and predictable (openrouter.LLM). No nits.

plugins/openrouter/tests/test_openrouter_llm.py (1)

60-69: Asyncio mode already enabled
The project’s pytest.ini sets asyncio_mode = auto, so async fixtures will run correctly without adding module‐level markers. No changes needed.

plugins/openrouter/example/openrouter_example.py (1)

43-46: Ignore finish() inside the join context. The AgentSessionContextManager.__exit__ method schedules only agent.close(), not agent.finish(), so the explicit await agent.finish() call inside the with block is required.

Likely an incorrect or invalid review comment.

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

Restore input before injecting history

create_response now calls add_conversation_history before guaranteeing kwargs["input"] exists. The OpenRouter override of add_conversation_history immediately reads kwargs["input"], so any call like await llm.create_response("hi") (no explicit input=) now crashes with a KeyError. Please populate kwargs["input"] from args (current logic at Lines 146-151) before invoking add_conversation_history, or make the helper resilient to missing input. Moving the input-defaulting block above this call fixes the regression.

         if "stream" not in kwargs:
             kwargs["stream"] = True

+        if "input" not in kwargs:
+            input_content = args[0] if args else "Hello"
+            kwargs["input"] = input_content
+
         # create the conversation if needed and add the required args
         await self.create_conversation()
         self.add_conversation_history(kwargs)
-
-        # Set up input parameter for OpenAI Responses API
-        if "input" not in kwargs:
-            input_content = args[0] if args else "Hello"
-            kwargs["input"] = input_content

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	# create the conversation if needed and add the required args
	await self.create_conversation()
	self.add_conversation_history(kwargs)
	if "stream" not in kwargs:
	kwargs["stream"] = True
	if "input" not in kwargs:
	input_content = args[0] if args else "Hello"
	kwargs["input"] = input_content
	# create the conversation if needed and add the required args
	await self.create_conversation()
	self.add_conversation_history(kwargs)

🤖 Prompt for AI Agents

In plugins/openai/vision_agents/plugins/openai/openai_llm.py around lines 130 to
133, the call to self.add_conversation_history(kwargs) happens before
kwargs["input"] is guaranteed, causing a KeyError in overrides that read
kwargs["input"] immediately; move the existing input-defaulting logic (the block
that copies positional args into kwargs["input"] found at lines ~146-151) to
before await self.create_conversation() and
self.add_conversation_history(kwargs), or alternatively update
add_conversation_history to safely handle a missing "input" key; prefer the
first approach: populate kwargs["input"] from args before calling
add_conversation_history so the history helper always sees an input.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Formatter references call_id but most records won’t have it → KeyError.

With this format, any log without record.call_id will crash formatting. Add a filter to set a default.

Apply:

-import logging
+import logging
@@
-logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s [call_id=%(call_id)s] %(name)s: %(message)s")
+logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s [call_id=%(call_id)s] %(name)s: %(message)s")
+class _CallIdFallback(logging.Filter):
+    def filter(self, record: logging.LogRecord) -> bool:
+        if not hasattr(record, "call_id"):
+            record.call_id = "-"
+        return True
+logging.getLogger().addFilter(_CallIdFallback())

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s [call_id=%(call_id)s] %(name)s: %(message)s")
	logger = logging.getLogger(__name__)
	import logging
	logging.basicConfig(
	level=logging.INFO,
	format="%(asctime)s %(levelname)s [call_id=%(call_id)s] %(name)s: %(message)s"
	)
	class _CallIdFallback(logging.Filter):
	def filter(self, record: logging.LogRecord) -> bool:
	if not hasattr(record, "call_id"):
	record.call_id = "-"
	return True
	logging.getLogger().addFilter(_CallIdFallback())
	logger = logging.getLogger(__name__)

🤖 Prompt for AI Agents

In plugins/openrouter/example/openrouter_example.py around lines 13 to 15, the
logging formatter references record.call_id which will raise KeyError for
records without that attribute; add a logging.Filter (or small callable) that
ensures every LogRecord has a call_id attribute (e.g., set record.call_id =
getattr(record, "call_id", "-")) and attach that filter to the root logger or
the handler used by basicConfig before setting the formatter so logs without
call_id do not crash formatting.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Docstring default model is incorrect.

Says default is openai/gpt-4o, but the code sets "openrouter/andromeda-alpha".

Apply:

-            model: Model to use. Defaults to openai/gpt-4o.
+            model: Model to use. Defaults to openrouter/andromeda-alpha.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	api_key: OpenRouter API key. If not provided, uses OPENROUTER_API_KEY env var.
	base_url: OpenRouter API base URL.
	model: Model to use. Defaults to openai/gpt-4o.
	**kwargs: Additional arguments passed to OpenAI LLM.
	"""
	api_key: OpenRouter API key. If not provided, uses OPENROUTER_API_KEY env var.
	base_url: OpenRouter API base URL.
	model: Model to use. Defaults to openrouter/andromeda-alpha.
	**kwargs: Additional arguments passed to OpenAI LLM.
	"""

🤖 Prompt for AI Agents

In plugins/openrouter/vision_agents/plugins/openrouter/openrouter_llm.py around
lines 29–33 the docstring claims the default model is "openai/gpt-4o" but the
code sets the default to "openrouter/andromeda-alpha"; update the docstring to
reflect the actual default model ("openrouter/andromeda-alpha") or, if the
intended default is "openai/gpt-4o", change the default in the code to that
value; ensure the docstring and any examples/tests remain consistent with the
chosen default.

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (8)

plugins/openrouter/tests/test_openrouter_llm.py (8)

14-16: Mark module for asyncio to avoid runner config coupling

If pytest-asyncio isn’t set to auto in this repo, these async tests will error. Add a module-level marker for safety.

 load_dotenv()
 
-
+# Run all tests in this module with an event loop
+pytestmark = pytest.mark.asyncio

Please confirm whether pytest-asyncio is configured in auto mode; if yes, this change is optional.

---

36-59: Avoid testing private API directly (or clearly mark it as such)

LLM._normalize_message is private; refactors can break this test. Prefer exercising normalization via public surfaces (e.g., add_conversation_history through create_response with stream=False and mocking network), or explicitly name tests to indicate internal coupling.

Example options:

• Use public path: call llm.add_conversation_history({"input": ...}) on a test double conversation.
• Or rename tests to test__normalize_message__basic/with_image to acknowledge internals.

---

80-88: Force non‑streaming to stabilize native API assertions

create_response defaults to stream=True; asserting response.original.id is more reliable with non‑streaming.

-        response = await llm.create_response(
-            input="say hi", instructions="You are a helpful assistant."
-        )
+        response = await llm.create_response(
+            input="say hi",
+            instructions="You are a helpful assistant.",
+            stream=False,
+        )

Please confirm OpenRouter returns an OpenAI‑compatible id for non‑streaming responses in your client version.

---

90-107: Tighten streaming assertion and use snake_case

Use PEP8 naming and assert at least one non‑empty delta chunk to prove streaming occurred.

-        streamingWorks = False
+        streaming_works = False
+        deltas = 0
 
         @llm.events.subscribe
         async def passed(event: LLMResponseChunkEvent):
-            nonlocal streamingWorks
-            streamingWorks = True
+            nonlocal streaming_works, deltas
+            if event.delta:
+                deltas += 1
+            streaming_works = True
@@
-        assert streamingWorks, "Streaming should have generated chunk events"
+        assert streaming_works, "Streaming should have generated chunk events"
+        assert deltas > 0, "Expected at least one non-empty text delta"

---

109-121: Reduce flakiness in memory tests that depend on model behavior

Live LLM answers vary. Consider marking these as flaky (rerun) or xfail when model confidence drops, or strengthen prompts to elicit numeric-only answers.

Options:

• If pytest-rerunfailures is available:

@pytest.mark.flaky(reruns=2, reruns_delay=1)

• Or:

pytest.xfail("Memory reasoning can be non-deterministic across models")

Confirm whether pytest-rerunfailures is part of dev dependencies; if not, I can open a PR to add it.

Also applies to: 124-136

---

139-156: Optional: Convert permanent skip to xfail with condition

Since instruction-following is intermittently successful, consider xfail with a condition rather than unconditional skip to still exercise the path locally when the model cooperates.

-        pytest.skip("instruction following doesnt always work")
+        pytest.xfail("Instruction following is not stable across providers/models")

---

44-59: Add a true multimodal integration test to cover Qwen VL objective

The normalization test builds an image message but doesn’t hit the API. To align with AI-194 (“Qwen VL demo”), add an integration test that sends text+image to a VL-capable model when OPENROUTER_MODEL points to Qwen VL.

Example (guarded by model capability):

@pytest.mark.integration
async def test_multimodal_integration(self, llm: LLM):
    if "qwen" not in llm.model and "vl" not in llm.model.lower():
        pytest.skip("Model not VL-capable")
    img_url = "https://upload.wikimedia.org/wikipedia/commons/d/d5/2023_06_08_Raccoon1.jpg"
    resp = await llm.create_response(
        input=[{"type": "input_text", "text": "What animal is this?"}, {"type": "input_image", "image_url": img_url}],
        stream=False,
    )
    self.assert_response_successful(resp)

---

61-68: Make model configurable; fix fixture docstring mismatch

Docstring still references the wrong model and the fixture hardcodes an invalid ID. Read OPENROUTER_MODEL with a valid default and update the docstring:

 async def llm(self) -> LLM:
-    """Fixture for OpenRouter LLM with z-ai/glm-4.6 model."""
+    """Fixture for OpenRouter LLM.
+
+    Returns:
+        LLM: Configured OpenRouter LLM instance. Uses OPENROUTER_MODEL if set.
+    """
     if not os.environ.get("OPENROUTER_API_KEY"):
         pytest.skip("OPENROUTER_API_KEY environment variable not set")
-    
-    llm = LLM(model="anthropic/claude-haiku-4.5")
+    model = os.getenv("OPENROUTER_MODEL", "anthropic/claude-3.5-sonnet")
+    llm = LLM(model=model)
     llm._conversation = InMemoryConversation("be friendly", [])
     return llm

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

• Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between ba6c027 and a426bc2.

📒 Files selected for processing (2)

• plugins/openrouter/README.md (1 hunks)
• plugins/openrouter/tests/test_openrouter_llm.py (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

• plugins/openrouter/README.md

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

📄 CodeRabbit inference engine (.cursor/rules/python.mdc)

**/*.py: Do not modify sys.path in Python code
Docstrings must follow the Google style guide

Files:

• plugins/openrouter/tests/test_openrouter_llm.py

🧬 Code graph analysis (1)

plugins/openrouter/tests/test_openrouter_llm.py (5)

agents-core/vision_agents/core/agents/conversation.py (1)

• InMemoryConversation (230-237)

agents-core/vision_agents/core/llm/events.py (1)

• LLMResponseChunkEvent (90-105)

agents-core/vision_agents/core/llm/llm.py (2)

• LLM (36-363)
• _set_instructions (164-168)

plugins/openai/vision_agents/plugins/openai/openai_llm.py (3)

• _normalize_message (351-369)
• simple_response (80-108)
• create_response (118-214)

agents-core/vision_agents/core/events/manager.py (1)

• wait (470-484)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)

• GitHub Check: unit / Test "not integration"
• GitHub Check: unit / Ruff & mypy
• GitHub Check: unit / Test "not integration"
• GitHub Check: unit / Ruff & mypy