Skip to content

Development

Jump to bottom
Joe Curlee (w4ffl35) edited this page Nov 29, 2025 · 14 revisions

Installation

See installation page for detailed instructions.

Environment Variables

  • All values must be set in your environment or a .env file before running AI Runner.
  • Be careful not to share sensitive tokens or API keys publicly.
  • See Settings documentation for available environment variables

Development Guidelines

Documentation

CRITICAL: The top-level docs/ directory in the main repository is DEPRECATED and git-ignored.

Documentation Location:

  • ALL documentation MUST be written in the wiki repository: ~/Projects/airunner.wiki
  • Never create .md files in the main repository's docs/ folder - they will be ignored by git
  • Component-level README.md files within src/airunner/components/*/ are acceptable for internal documentation

Examples:

  • ✅ Update ~/Projects/airunner.wiki/Architecture.md (feature documentation)
  • ✅ Create src/airunner/components/llm/README.md (component-level internal docs)
  • ❌ Create docs/new_feature.md (WRONG - git-ignored)

AI development tools

The use of AI tools to develop AI Runner is encouraged.

Here are some recommendations and guidelines.

  • Try to keep full agent development limited to things like widgets, tests, styles
  • Point your agent at existing widgets, tell your agent to use that as a reference while creating a new widget and provide detailed instructions
  • Do some manual testing to ensure everything works as expected
  • Your code must still be formatted correctly and follow other expected development standards
  • Agents tend to fail on large features that span multiple modules - although code assistants will be helpful while editing things like the model manager classes, agents will struggle.
  • AI Runner cannot as of yet program itself, but this would be an interesting area to explore.
  • Join the Discord server and share some of the things you create and the prompts and techniques you used to create them

Setting Up the Environment

  1. Clone the repository: 
    git clone https://github.com/Capsize-Games/airunner.git
cd airunner
  2. Install dependencies: 
    pip install -r requirements.txt
  3. Set up environment variables in a .env file or export them directly: 
    export AI_RUNNER_DATABASE_URL="sqlite:///path/to/database.db"
export AIRUNNER_LOG_LEVEL="DEBUG"

Running the Application in Development Mode

Use the following command to start AI Runner in development mode:

python src/airunner/app.py

Testing

CRITICAL: The top-level tests/ directory is DEPRECATED and should NOT be used.

Test File Location:

  • Test files MUST live in component-specific tests/ directories
  • Each component has its own tests/ subdirectory within the component folder
  • Test files follow the naming convention: test_<module_name>.py

Examples:

  • ✅ src/airunner/components/llm/tests/test_llm_manager.py
  • ✅ src/airunner/components/model_management/tests/test_model_resource_manager.py
  • ❌ tests/test_llm_manager.py (WRONG - deprecated location)

Run all unit tests:

pytest src/

Run tests for a specific component:

pytest src/airunner/components/llm/tests/

Run a specific test file:

pytest src/airunner/components/llm/tests/test_llm_manager.py

Code Style

  • Use spaces for indentation
  • Follow PEP 8 guidelines for Python code.
  • Use black for code formatting: 
    black .
    • Check for linting issues with flake8: Note: This one is optional, but recommended.
    flake8 src/airunner

Editor Configuration

If you're using VSCode add this to the settings:

{
   "black-formatter.args": ["--line-length", "79"],
   "[python]": {
        "editor.defaultFormatter": "ms-python.black-formatter",
        "editor.formatOnSave": true,
        "editor.rulers": [
          {
            "column": 72,
            "color": "#222"
          },
          {
            "column": 79,
            "color": "#888"
          },
      ]
    }
}

Contributing

  • Fork the repository and create a feature branch.
  • Ensure all tests pass before submitting a pull request.
  • Include detailed commit messages and documentation for new features.

LLM & Agent Development

Mode-Based Architecture

AI Runner uses a mode-based routing system that automatically directs queries to specialized agents:

  • Author Mode 🖋️ - Writing, documentation, content creation
  • Code Mode 💻 - Programming, debugging, code review
  • Research Mode 🔍 - Information gathering, fact-checking
  • QA Mode ❓ - Question answering, explanations
  • General Mode ⚙️ - Fallback for multi-domain tasks

Quick Start:

from airunner.components.llm.managers.workflow_manager import WorkflowManager

# Enable mode routing
manager = WorkflowManager(use_mode_routing=True)
response = manager.process("Write a function to parse JSON")
# Automatically routed to CODE mode

Documentation:

Creating New Agents

To create a new specialized agent:

  1. Define State Schema:

    from typing import TypedDict, Annotated
from langchain_core.messages import BaseMessage
from langgraph.graph.message import add_messages

class MyAgentState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]
    custom_field: str

  2. Create Agent Class:

    from airunner.components.llm.agents.base_agent import BaseAgent

class MyAgent(BaseAgent):
    def __init__(self, chat_model, tools=None):
        super().__init__(
            state_schema=MyAgentState,
            chat_model=chat_model,
            tools=tools or []
        )

  3. Add Tests:

    • Create src/airunner/components/llm/agents/tests/test_my_agent.py
    • Test agent initialization, compilation, and execution
    • Test trajectory and tool usage

  4. Register in Router:

    • Update mode_router.py to include new mode
    • Add routing logic in parent_graph_builder.py

Testing LLM Components

Unit Tests:

# Test specific agent
pytest src/airunner/components/llm/agents/tests/test_author_agent.py

# Test all agents
pytest src/airunner/components/llm/agents/tests/ -v

# Test with trajectory evaluation
pytest src/airunner/components/llm/evaluators/tests/ -v

Integration Tests:

# Test mode routing
pytest src/airunner/components/llm/managers/tests/test_e2e_mode_routing.py

# Test parent router
pytest src/airunner/components/llm/managers/tests/test_parent_graph_builder.py

Trajectory Evaluation:

from airunner.components.llm.evaluators.utils.trajectory_tracking import track_trajectory_sync
from airunner.components.llm.evaluators.utils.trajectory_evaluator import trajectory_tool_usage

# Track agent execution
trajectory = track_trajectory_sync(
    graph=agent.compile(),
    inputs={"messages": [...]},
    expected_nodes=["agent", "tools", "response"]
)

# Validate tool usage
tools_used = trajectory_tool_usage(trajectory, expected_tools=["my_tool"])
assert tools_used["used_expected"]

Tool Development

Creating New Tools:

  1. Define Tool Function:

    from langchain_core.tools import tool
from airunner.enums import ToolCategory

@tool
def my_custom_tool(param: str) -> str:
    """Tool description for LLM.
    
    Args:
        param: Parameter description
        
    Returns:
        Result description
    """
    return f"Result: {param}"

# Set category metadata
my_custom_tool.category = ToolCategory.CODE  # or AUTHOR, RESEARCH, QA

  2. Register Tool:

    • Add to appropriate file in src/airunner/components/llm/tools/
    • Export from __init__.py
    • Tool automatically available to agents in matching category

  3. Test Tool:

    def test_my_custom_tool():
    result = my_custom_tool.invoke({"param": "test"})
    assert "Result: test" in result

Tool Categories:

  • Mode-Specific: AUTHOR, CODE, RESEARCH, QA
  • Cross-Mode: CHAT, IMAGE, FILE, MATH, SYSTEM, etc.

Docker container maintenance

If you make changes to the build files, be sure to test them locally before doing a PR. For this you can use the act tool to run the GitHub actions locally.

  • Install act
  • Create a github personal access token
  • Add the personal access token to a .secrets file in the root of the project. The file should look like this: 
    CR_PAT=ghp_xxxxxxxx1234567890abcdefg
  • Run the build action locally: 
    act push --secret-file .secrets --verbose

AI Runner Internationalization (i18n)

AI Runner supports internationalization (i18n) using the Qt framework. The translation files are located in ./src/airunner/translations/. The main translation file is english.ts, which contains the English translations. You can create additional translation files for other languages by using the pyside6-linguist tool.

Pyside commands for translation

  • Update translation file: 
    pyside6-lupdate ./src/airunner/ -ts src/airunner/translations/english.ts
  • Edit translation file: 
    pyside6-linguist ./src/airunner/translations/english.ts
  • Generate translation files: 
    pyside6-lrelease ./src/airunner/translations/english.ts -qm ./src/airunner/translations/english.qm

Clone this wiki locally