Hotfix | Rogue-Server | Bedrock support

[yuval-qf]

Description

Motivation and Context

Type of Change

• 🐛 Bug fix (non-breaking change which fixes an issue)
• ✨ New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
• 📝 Documentation update
• 🎨 Code style/refactoring (no functional changes)
• 🧪 Test updates
• 🔧 Configuration/build changes

Changes Made

Screenshots/Examples (if applicable)

Checklist

• I have read the CONTRIBUTING.md guide
• My code follows the code style of this project (PEP 8, type hints, docstrings)
• I have run uv run black . to format my code
• I have run uv run flake8 . and fixed all issues
• I have run uv run mypy --config-file .mypy.ini . and addressed type checking issues
• I have run uv run bandit -c .bandit.yaml -r . for security checks
• I have added tests that prove my fix is effective or that my feature works
• I have run uv run pytest and all tests pass
• I have manually tested my changes
• I have updated the documentation accordingly
• I have added/updated type hints for new/modified functions
• My changes generate no new warnings
• I have checked my code for security issues
• Any dependent changes have been merged and published

Testing

Test Configuration:

• Python version:
• OS:
• Other relevant details:

Test Steps:
1.
2.
3.

Additional Notes

Related Issues/PRs

• Fixes #
• Related to #
• Depends on #

---

[coderabbitai]

Summary by CodeRabbit

• Chores

  • Version bumped to 0.2.3
  • Updated LLM model configuration for evaluations
  • Added AWS integration dependency

• Refactor

  • Improved scenario parsing and validation logic in evaluation process
  • Enhanced documentation for internal messaging functionality

Walkthrough

Configuration updates including version bump (0.2.2→0.2.3), model provider switch from OpenAI to AWS Bedrock, boto3 dependency addition, and refactoring of the _log_evaluation method signature to remove the scenario_type parameter with added JSON parsing logic for scenario handling.

Changes

Cohort / File(s)	Summary
Configuration & Versioning VERSION, .vscode/launch.json	Version bumped to 0.2.3; Rogue Server launch arguments updated to include --debug flag; judge-llm model switched from openai/o4-mini to bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
Dependencies pyproject.toml	Added boto3>=1.40.69 dependency to support AWS Bedrock integration
Evaluator Agent Core Logic rogue/evaluator_agent/base_evaluator_agent.py	Added json import; _log_evaluation method signature updated (removed scenario_type parameter); scenario handling refactored to parse string scenarios as JSON with fallback to raw string dict on parse failure; scenario_type now derived from scenario dict instead of parameter
Evaluator Agent MCP rogue/evaluator_agent/mcp/mcp_evaluator_agent.py	Added docstring to _send_message_to_evaluated_agent documenting purpose, parameters, and return structure

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~30–40 minutes

• _log_evaluation signature change – Removal of scenario_type parameter is a breaking change; requires verifying all callers throughout the codebase are compatible with the new interface and that scenario_type derivation from dict is reliable
• JSON parsing and error handling – New logic for parsing scenarios as JSON with error recovery warrants careful review to ensure edge cases (malformed JSON, empty strings, etc.) are handled safely
• Model provider transition – Bedrock model substitution and boto3 integration; verify configuration correctly references the new endpoint and authentication
• Behavioral implications – Early return on JSON parse failure alters control flow; ensure this doesn't inadvertently skip critical evaluation logging in production scenarios

Possibly related PRs

• FIRE-812 | Rogue-TUI | MCP Support #115: Modifies _log_evaluation signature and handling (prior iteration that kept scenario_type parameter and added **kwargs); understanding the relationship between these changes may clarify the refactoring intent
• Hotfix | Bump version to 0.2.2 #131: Version file bump to 0.2.2; provides context on versioning pattern and release cadence

Suggested reviewers

• drorIvry
• osher-qualifire

Poem

🐰 A rabbit hops through JSON streams,
Bedrock models now grace our dreams,
The evaluator learns to parse and flow,
Version bumps with grace—0.2.3's glow!
Scenarios transform, from string to dict divine,
This refactor's logic, now sublime.

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description is entirely empty/templated with no actual content filled in, missing all required information including motivation, changes made, testing details, and type of change selection.	Fill in all required sections with actual content: Description, Motivation and Context (including issue links), select Type of Change, list specific changes made, provide testing steps, and complete the checklist verification.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title directly addresses the main change: adding Bedrock support to Rogue-Server, which aligns with the boto3 dependency addition and Claude model switch in the changeset.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch hotfix/bedrock-support-fixes

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9a011a1 and b12af94.

⛔ Files ignored due to path filters (1)

• uv.lock is excluded by !**/*.lock

📒 Files selected for processing (5)

• .vscode/launch.json (2 hunks)
• VERSION (1 hunks)
• pyproject.toml (1 hunks)
• rogue/evaluator_agent/base_evaluator_agent.py (3 hunks)
• rogue/evaluator_agent/mcp/mcp_evaluator_agent.py (1 hunks)

🧰 Additional context used

📓 Path-based instructions (2)

**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

**/*.py: Format Python code with Black
Ensure code passes flake8 linting
Run mypy with the repository configuration for static typing
Run Bandit security checks using .bandit.yaml configuration
Use isort import conventions for import ordering
Add type hints to all function signatures
Follow PEP 8 naming (snake_case for variables/functions, PascalCase for classes)
Use try/except around code that may raise exceptions

Files:

• rogue/evaluator_agent/base_evaluator_agent.py
• rogue/evaluator_agent/mcp/mcp_evaluator_agent.py

pyproject.toml

📄 CodeRabbit inference engine (AGENTS.md)

Manage dependencies with uv and declare them in pyproject.toml

Files:

• pyproject.toml

🧠 Learnings (1)

📚 Learning: 2025-10-05T15:28:00.385Z

Learnt from: CR
Repo: qualifire-dev/rogue PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-10-05T15:28:00.385Z
Learning: Applies to pyproject.toml : Manage dependencies with uv and declare them in pyproject.toml

Applied to files:

• pyproject.toml

🧬 Code graph analysis (1)

rogue/evaluator_agent/base_evaluator_agent.py (1)

sdks/python/rogue_sdk/types.py (1)

• ScenarioType (51-55)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)

• GitHub Check: codestyle
• GitHub Check: rogue_sanity