refactor: Move AdCP schemas from tests/e2e/ to project root (#614)

PROBLEM:
The official AdCP JSON schemas were buried in tests/e2e/schemas/v1/, creating confusion about their role as the source of truth for the entire project. This location suggested they were test-specific artifacts rather than core project infrastructure.

SOLUTION:
Moved schemas to schemas/v1/ at project root to clarify their purpose:
- Source of truth for entire project (not just tests)
- Used by schema generator (src/core/schemas_generated/)
- Used by schema validator (tests/e2e/adcp_schema_validator.py)
- Used by pre-commit hooks for compliance checking

CHANGES:
- Moved tests/e2e/schemas/ → schemas/
- Updated all script references:
  - scripts/generate_schemas.py
  - scripts/refresh_schemas.py
  - scripts/validate_pydantic_against_adcp_schemas.py
  - scripts/check_schema_sync.py
- Updated test file paths:
  - tests/unit/test_adapter_schema_compliance.py
  - tests/e2e/adcp_schema_validator.py
- Updated documentation:
  - docs/schema-updates.md
  - docs/schema-caching-strategy.md
  - docs/testing/adapter-schema-compliance.md
  - docs/development/schema-auto-generation.md
  - CLAUDE.md
- Updated .pre-commit-config.yaml file patterns
- Updated src/core/schemas_generated/__init__.py header
- Added comprehensive schemas/README.md documenting purpose and usage

BENEFITS:
✅ Clear that schemas are project-wide source of truth
✅ Easier to understand: 'these are OUR schemas'
✅ No confusion about test vs production schemas
✅ Generator script reads from obvious location
✅ Better project organization

TESTING:
- Verified schema validator finds new path
- Verified generator script works with new path
- All references updated and validated

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>

Author: bokelley

.pre-commit-config.yaml

@@ -176,7 +176,7 @@ repos:
         name: Validate adapter schemas match AdCP spec
         entry: uv run pytest tests/unit/test_adapter_schema_compliance.py -v --tb=short
         language: system
-        files: '^(src/core/schema_adapters\.py|tests/e2e/schemas/v1/.*\.json)$'
+        files: '^(src/core/schema_adapters\.py|schemas/v1/.*\.json)$'
         pass_filenames: false
         always_run: true
 
@@ -185,7 +185,7 @@ repos:
         name: Pydantic model alignment with AdCP schemas (REGRESSION PREVENTION)
         entry: uv run pytest tests/unit/test_pydantic_schema_alignment.py::TestSpecificFieldValidation -v --tb=short
         language: system
-        files: '^(src/core/schemas\.py|tests/e2e/schemas/v1/.*\.json)$'
+        files: '^(src/core/schemas\.py|schemas/v1/.*\.json)$'
         pass_filenames: false
         always_run: true
 

CLAUDE.md

@@ -7,7 +7,7 @@
 
 **Schema Hierarchy:**
 1. **Official Spec** (https://adcontextprotocol.org/schemas/v1/) - Primary source of truth
-2. **Cached Schemas** (`tests/e2e/schemas/v1/`) - Checked into git for offline validation
+2. **Cached Schemas** (`schemas/v1/`) - Checked into git for offline validation
 3. **Pydantic Schemas** (`src/core/schemas.py`) - MUST match official spec exactly
 
 **Rules:**
@@ -33,7 +33,7 @@ pytest tests/e2e/test_adcp_compliance.py -v
 pytest tests/unit/test_adcp_contract.py -v
 
 # If schemas are out of date, cached files are auto-updated on next run
-# Commit any schema file changes that appear in tests/e2e/schemas/v1/
+# Commit any schema file changes that appear in schemas/v1/
 ```
 
 **Current Schema Version:**

docs/development/schema-auto-generation.md

@@ -9,7 +9,7 @@ This project now supports **automatic generation of Pydantic models** from the o
 ### 1. Schema Resolution
 
 The generation script handles the AdCP schema structure:
-- **Cached Schemas**: Uses locally cached schemas from `tests/e2e/schemas/v1/`
+- **Cached Schemas**: Uses locally cached schemas from `schemas/v1/`
 - **Flattened Naming**: Handles the flattened file naming (e.g., `_schemas_v1_core_budget_json.json`)
 - **$ref Resolution**: Automatically resolves JSON Schema `$ref` references
 - **Auto-Download**: Downloads missing schemas from https://adcontextprotocol.org/schemas/v1/
@@ -24,7 +24,7 @@ python scripts/generate_schemas.py
 ```
 
 The script:
-1. Loads all JSON schemas from `tests/e2e/schemas/v1/`
+1. Loads all JSON schemas from `schemas/v1/`
 2. Downloads any missing referenced schemas from AdCP website
 3. Resolves all `$ref` references recursively
 4. Generates clean Pydantic v2 models using `datamodel-code-generator`

docs/schema-caching-strategy.md

@@ -86,7 +86,7 @@ else:
 ### File Organization
 
 ```
-tests/e2e/schemas/v1/
+schemas/v1/
 ├── index.json                    # Schema registry
 ├── index.json.meta              # ETag metadata for index
 ├── _schemas_v1_core_package_json.json       # Schema content
@@ -264,10 +264,10 @@ pytest tests/e2e/ --offline-schemas
 ### Cache inconsistencies
 ```bash
 # Check metadata files
-ls -la tests/e2e/schemas/v1/*.meta
+ls -la schemas/v1/*.meta
 
 # View ETag for specific schema
-cat tests/e2e/schemas/v1/index.json.meta
+cat schemas/v1/index.json.meta
 ```
 
 ### Debugging ETag behavior

docs/schema-updates.md

@@ -4,7 +4,7 @@ This document describes how to keep our locally cached AdCP schemas synchronized
 
 ## Overview
 
-We maintain local cached copies of AdCP schemas in `tests/e2e/schemas/v1/` for:
+We maintain local cached copies of AdCP schemas in `schemas/v1/` for:
 - Offline development and testing
 - Schema validation in CI
 - Pydantic model generation via `scripts/generate_schemas.py`
@@ -60,7 +60,7 @@ uv run python scripts/check_schema_sync.py
 uv run python scripts/generate_schemas.py
 
 # Commit changes
-git add tests/e2e/schemas/ src/core/schemas_generated/
+git add schemas/ src/core/schemas_generated/
 git commit -m "Update AdCP schemas to latest from registry"
 ```
 
@@ -71,14 +71,14 @@ If you need to update a specific schema:
 ```bash
 # Download specific schema
 curl -s https://adcontextprotocol.org/schemas/v1/media-buy/package-request.json > \
-  tests/e2e/schemas/v1/_schemas_v1_media-buy_package-request_json.json
+  schemas/v1/_schemas_v1_media-buy_package-request_json.json
 
 # Download dependencies (e.g., format-id.json)
 curl -s https://adcontextprotocol.org/schemas/v1/core/format-id.json > \
-  tests/e2e/schemas/v1/_schemas_v1_core_format-id_json.json
+  schemas/v1/_schemas_v1_core_format-id_json.json
 
 # Verify schema structure
-cat tests/e2e/schemas/v1/_schemas_v1_media-buy_package-request_json.json | \
+cat schemas/v1/_schemas_v1_media-buy_package-request_json.json | \
   jq '.properties.format_ids.items'
 
 # Should show: {"$ref": "/schemas/v1/core/format-id.json"}
@@ -137,7 +137,7 @@ uv run python scripts/check_schema_sync.py --update
 uv run python scripts/generate_schemas.py
 
 # Commit both schema cache and generated code
-git add tests/e2e/schemas/ src/core/schemas_generated/
+git add schemas/ src/core/schemas_generated/
 git commit -m "Update AdCP schemas to v{version}"
 ```
 
@@ -150,7 +150,7 @@ git commit -m "Update AdCP schemas to v{version}"
 **Fix:**
 ```bash
 # 1. Verify cached schema has $ref
-cat tests/e2e/schemas/v1/_schemas_v1_media-buy_package-request_json.json | \
+cat schemas/v1/_schemas_v1_media-buy_package-request_json.json | \
   jq '.properties.format_ids.items'
 
 # Should output: {"$ref": "/schemas/v1/core/format-id.json"}
@@ -172,11 +172,11 @@ This means the cached `package-request.json` is outdated.
 ```bash
 # Download latest schema
 curl -s https://adcontextprotocol.org/schemas/v1/media-buy/package-request.json > \
-  tests/e2e/schemas/v1/_schemas_v1_media-buy_package-request_json.json
+  schemas/v1/_schemas_v1_media-buy_package-request_json.json
 
 # Download format-id definition
 curl -s https://adcontextprotocol.org/schemas/v1/core/format-id.json > \
-  tests/e2e/schemas/v1/_schemas_v1_core_format-id_json.json
+  schemas/v1/_schemas_v1_core_format-id_json.json
 
 # Regenerate models
 uv run python scripts/generate_schemas.py
@@ -213,7 +213,7 @@ graph TD
 - **AdCP GitHub:** https://github.com/adcontextprotocol/adcp
 - **Schema Checker:** `scripts/check_schema_sync.py`
 - **Schema Generator:** `scripts/generate_schemas.py`
-- **Cached Schemas:** `tests/e2e/schemas/v1/`
+- **Cached Schemas:** `schemas/v1/`
 - **Generated Models:** `src/core/schemas_generated/`
 
 ## Recent Changes

docs/testing/adapter-schema-compliance.md

@@ -8,7 +8,7 @@ This document explains how we ensure `schema_adapters.py` stays in sync with the
 
 We have three schema layers:
 
-1. **Official AdCP JSON Schemas** (`tests/e2e/schemas/v1/*.json`) - Source of truth from https://adcontextprotocol.org
+1. **Official AdCP JSON Schemas** (`schemas/v1/*.json`) - Source of truth from https://adcontextprotocol.org
 2. **Base Pydantic Schemas** (`src/core/schemas.py`) - Generated from JSON schemas, domain data only
 3. **Adapter Schemas** (`src/core/schema_adapters.py`) - Wrap base schemas, add `__str__()` for protocol abstraction
 
@@ -66,7 +66,7 @@ def test_list_authorized_properties_response_matches_spec(self):
   name: Validate adapter schemas match AdCP spec
   entry: uv run pytest tests/unit/test_adapter_schema_compliance.py -v --tb=short
   language: system
-  files: '^(src/core/schema_adapters\.py|tests/e2e/schemas/v1/.*\.json)$'
+  files: '^(src/core/schema_adapters\.py|schemas/v1/.*\.json)$'
   pass_filenames: false
   always_run: true
 ```
@@ -251,7 +251,7 @@ class MyModel(Base):
 1. **Verify field exists in official AdCP spec first:**
    ```bash
    # Check cached schema
-   cat tests/e2e/schemas/v1/_schemas_v1_media-buy_list-authorized-properties-response_json.json
+   cat schemas/v1/_schemas_v1_media-buy_list-authorized-properties-response_json.json
    ```
 
 2. **Add field to adapter schema:**
@@ -290,7 +290,7 @@ Missing: implementation_date (required=False)
 ```
 
 **How to fix:**
-1. Check official schema: `tests/e2e/schemas/v1/_schemas_v1_media-buy_update-media-buy-response_json.json`
+1. Check official schema: `schemas/v1/_schemas_v1_media-buy_update-media-buy-response_json.json`
 2. Verify field is in spec (lines 60-65 show `implementation_date`)
 3. Add field to `schema_adapters.py`:
    ```python
@@ -323,7 +323,7 @@ Generate adapter schemas from JSON schemas:
 ```bash
 # Generate adapter schema from official spec
 python scripts/generate_adapter_schema.py \
-    --spec tests/e2e/schemas/v1/_schemas_v1_media-buy_list-authorized-properties-response_json.json \
+    --spec schemas/v1/_schemas_v1_media-buy_list-authorized-properties-response_json.json \
     --output src/core/schema_adapters.py \
     --class-name ListAuthorizedPropertiesResponse
 ```

schemas/README.md

@@ -0,0 +1,150 @@
+# AdCP Schema Cache
+
+This directory contains cached copies of official AdCP JSON schemas from https://adcontextprotocol.org/schemas/
+
+## Purpose
+
+These cached schemas serve as the **source of truth** for the entire project:
+
+1. **Offline Development** - Work without internet access
+2. **Schema Validation** - Validate requests/responses in tests (see `tests/e2e/adcp_schema_validator.py`)
+3. **Pydantic Generation** - Auto-generate type-safe Pydantic models (see `src/core/schemas_generated/`)
+4. **CI/CD Stability** - Ensure consistent schemas across all environments
+
+## Directory Structure
+
+```
+schemas/
+├── README.md                    # This file
+├── v1/                         # AdCP v1 schemas
+│   ├── *.json                  # Cached schema files
+│   ├── *.json.meta             # Metadata (ETag, Last-Modified)
+│   └── SCHEMAS_INFO.md         # Schema registry info
+└── compliance_report.json      # Last compliance check results
+```
+
+## Schema Naming Convention
+
+Schema files use a flattened naming convention to avoid directory nesting:
+
+```
+/schemas/v1/media-buy/create-media-buy-request.json
+→ _schemas_v1_media-buy_create-media-buy-request_json.json
+```
+
+This allows simpler file management while preserving the logical structure.
+
+## Updating Schemas
+
+### Automatic Update (Recommended)
+
+```bash
+# Download all schemas from official registry
+uv run python scripts/refresh_schemas.py
+
+# Regenerate Pydantic models
+uv run python scripts/generate_schemas.py
+
+# Commit changes
+git add schemas/ src/core/schemas_generated/
+git commit -m "Update AdCP schemas to latest from registry"
+```
+
+### Manual Update (Specific Schema)
+
+```bash
+# Download specific schema
+curl -s https://adcontextprotocol.org/schemas/v1/media-buy/create-media-buy-request.json > \
+  schemas/v1/_schemas_v1_media-buy_create-media-buy-request_json.json
+
+# Regenerate Pydantic models
+uv run python scripts/generate_schemas.py
+```
+
+## Schema Validation
+
+The `tests/e2e/adcp_schema_validator.py` module uses these cached schemas to validate:
+
+- Request payloads before sending to external systems
+- Response payloads from external AdCP agents
+- E2E test data for compliance
+
+## ETag-Based Caching
+
+Each schema file has a corresponding `.meta` file containing:
+
+```json
+{
+  "etag": "W/\"abc123\"",
+  "last-modified": "Mon, 01 Jan 2024 12:00:00 GMT",
+  "downloaded_at": "2024-01-01T12:00:00",
+  "schema_ref": "/schemas/v1/media-buy/create-media-buy-request.json"
+}
+```
+
+The schema validator uses ETags for conditional GET requests:
+- If schema hasn't changed on server (304 Not Modified), use cached version
+- If schema updated, download and cache new version
+- Falls back to cache if server unavailable
+
+## Related Files
+
+- **Schema Generator**: `scripts/generate_schemas.py`
+- **Schema Validator**: `tests/e2e/adcp_schema_validator.py`
+- **Schema Sync Checker**: `scripts/check_schema_sync.py`
+- **Generated Pydantic Models**: `src/core/schemas_generated/`
+- **Manual Pydantic Schemas**: `src/core/schemas.py` (hand-written)
+
+## Pre-commit Hooks
+
+Pre-commit hooks automatically check:
+
+1. **Schema Sync** - Ensures cached schemas match official registry
+2. **Pydantic Alignment** - Ensures hand-written schemas match official spec
+3. **Adapter Compliance** - Ensures adapter schemas use correct structure
+
+If schemas are out of sync, the commit is blocked with instructions to update.
+
+## Version History
+
+- **v1** - Initial AdCP specification (current)
+- **v2** - Future version (when released)
+
+## Official Sources
+
+- **Registry**: https://adcontextprotocol.org/schemas/v1/
+- **Documentation**: https://adcontextprotocol.org/docs/
+- **GitHub**: https://github.com/adcontextprotocol/adcp
+
+## FAQ
+
+### Why cache schemas locally?
+
+1. **Offline Development** - Work without internet
+2. **Deterministic Builds** - CI doesn't depend on external services
+3. **Fast Tests** - No network overhead during test runs
+4. **Version Control** - Track schema changes in git
+
+### How often should I update schemas?
+
+- **Development**: When you need new features or notice missing fields
+- **Production**: Before each major release
+- **CI**: Pre-commit hooks check automatically
+
+### What if schemas are out of sync?
+
+Pre-commit hooks will block commits and show:
+
+```
+❌ Schema out of sync: create-media-buy-request.json
+   Run: uv run python scripts/refresh_schemas.py
+```
+
+Just run the suggested command and commit the changes.
+
+### Why both cached schemas AND generated Pydantic models?
+
+- **Cached JSON**: Source of truth for validation (runtime)
+- **Generated Pydantic**: Type-safe models for development (compile-time)
+
+Both serve different purposes and are generated from the same official schemas.

tests/e2e/schemas/compliance_report.json schemas/compliance_report.jsontests/e2e/schemas/compliance_report.json renamed to schemas/compliance_report.json

@@ -72,4 +72,4 @@
       "last_updated": "2025-01-10T15:30:00Z"
     }
   ]
-}
+}