Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
pip install -e ".[dev,telemetry]"

- name: Run tests
run: |
Expand Down Expand Up @@ -73,7 +73,7 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev]"
pip install -e ".[dev,telemetry]"

- name: Run end-to-end tests with real API
env:
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile.test
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ WORKDIR /app
# Copy the SDK source
COPY . .

# Install SDK with dev dependencies
RUN pip install -e ".[dev]"
# Install SDK with dev + telemetry dependencies
RUN pip install -e ".[dev,telemetry]"

# Verify CLI installation
RUN claude -v
Expand Down
44 changes: 44 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,50 @@ Python SDK for Claude Agent. See the [Claude Agent SDK documentation](https://pl
pip install claude-agent-sdk
```

### Telemetry (OpenTelemetry)

Install the optional OpenTelemetry dependencies to enable tracing/metrics:

```bash
pip install "claude-agent-sdk[telemetry]"
```

Then enable telemetry in options:

```python
from claude_agent_sdk import ClaudeAgentOptions, TelemetryOptions

options = ClaudeAgentOptions(
telemetry=TelemetryOptions(enabled=True),
)
```

### Telemetry span names

Spans use the `claude_agent_sdk.<layer>.<operation>` convention. Examples:

- `claude_agent_sdk.client.connect`, `claude_agent_sdk.client.query`, `claude_agent_sdk.client.disconnect`
- `claude_agent_sdk.query.lifecycle`, `claude_agent_sdk.query.initialize`, `claude_agent_sdk.query.read_messages`, `claude_agent_sdk.query.stream_input`, `claude_agent_sdk.query.close`
- `claude_agent_sdk.transport.connect`, `claude_agent_sdk.transport.read_messages`, `claude_agent_sdk.transport.write`, `claude_agent_sdk.transport.close`
- `claude_agent_sdk.permission.can_use_tool`, `claude_agent_sdk.hooks.callback`
- `claude_agent_sdk.mcp.request`, `claude_agent_sdk.mcp.tool_call`
- `claude_agent_sdk.cli.tool_call`

### Telemetry metrics

When enabled, the SDK emits the following metrics (all prefixed with `claude_agent_sdk.`):

- `messages`, `results`, `errors`
- `invocations`
- `tokens.prompt`, `tokens.completion`, `tokens.total`
- `model.latency_ms`, `model.errors`
- `result.duration_ms`, `result.cost_usd`
- `cost.total_usd`
- `response.size_bytes`
- `throttled` (rate-limit events)

Most metrics include a `session.id` attribute to allow per-session grouping.

**Prerequisites:**

- Python 3.10+
Expand Down
26 changes: 23 additions & 3 deletions e2e-tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,14 @@ export ANTHROPIC_API_KEY="your-api-key-here"

### Dependencies

Install the development dependencies:
Install the development dependencies including telemetry support:

```bash
pip install -e ".[dev]"
pip install -e ".[dev,telemetry]"
```

> **Note**: Telemetry tests (`test_telemetry.py`) require the `telemetry` extra. Without it, these tests will be silently skipped via `pytest.importorskip()`.

## Running the Tests

### Run all e2e tests:
Expand Down Expand Up @@ -52,6 +54,24 @@ python -m pytest e2e-tests/test_mcp_calculator.py::test_basic_addition -v

## Test Coverage

### Telemetry Tests (`test_telemetry.py`)

Tests OpenTelemetry tracing and metrics integration:

- **test_telemetry_tracing_spans_emitted**: Verifies core spans are emitted during SDK session
- **test_telemetry_metrics_emitted**: Verifies core metrics (messages, tokens, cost) are recorded
- **test_telemetry_token_metrics_detailed**: Tests prompt/completion token split metrics
- **test_telemetry_tool_use**: Validates tool use spans for both CLI and SDK MCP tools
- **test_telemetry_disabled_no_crash**: Ensures SDK works when telemetry is disabled
- **test_telemetry_not_provided_no_crash**: Ensures SDK works without telemetry config
- **test_telemetry_enabled_without_tracer_or_meter**: Tests fallback to default tracer/meter
- **test_telemetry_options_invalid_tracer/meter**: Validates type checking on TelemetryOptions
- **test_telemetry_hook_spans**: Verifies hook callback spans are emitted
- **test_telemetry_permission_callback_spans**: Verifies permission callback spans
- **test_telemetry_error_recording_invalid_cwd**: Tests error recording on spans
- **test_telemetry_duration_metrics**: Validates duration-related metrics
- **test_telemetry_invocation_counter**: Verifies invocation counter increments

### MCP Calculator Tests (`test_mcp_calculator.py`)

Tests the MCP (Model Context Protocol) integration with calculator tools:
Expand Down Expand Up @@ -99,4 +119,4 @@ When adding new e2e tests:
2. Use the `api_key` fixture to ensure API key is available
3. Keep prompts simple to minimize costs
4. Verify actual tool execution, not just mocked responses
5. Document any special setup requirements in this README
5. Document any special setup requirements in this README
4 changes: 3 additions & 1 deletion e2e-tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,6 @@ def event_loop_policy():

def pytest_configure(config):
"""Add e2e marker."""
config.addinivalue_line("markers", "e2e: marks tests as e2e tests requiring API key")
config.addinivalue_line(
"markers", "e2e: marks tests as e2e tests requiring API key"
)
Loading