[Feature Request]: Epic: Secure JWT Token Catalog with Per-User Expiry and Revocation

[crivetimihai]

Epic: Secure JWT Token Catalog with Per-User Expiry and Revocation

🎯 Overview

Summary

Implement a comprehensive JWT token management system with enhanced security features including per-user token issuance, individual revocation, CSRF protection, claims validation, and structured logging.

Problem Statement

Currently, all bearer tokens share a global signing secret with several limitations:

• If one token is compromised, rotating the secret invalidates ALL tokens
• No CSRF protection for cookie-based authentication
• Limited JWT claims validation
• No audit trail for authentication events
• Single user basic authentication limits scalability

Solution

Create a secure token catalog system that:

• Each token has a unique identifier (jti) for individual revocation
• Comprehensive security validations (expiration, claims, algorithms)
• CSRF protection for cookie-based authentication
• Structured logging for security monitoring
• Token metadata stored securely with audit trail
• Multi-user support with secure password policies

Dependencies

• Depends on: [SECURITY FEATURE]: Make JWT Token Expiration Mandatory when REQUIRE_TOKEN_EXPIRATION=true (depends on #87) #425 - Configurable JWT Token Expiration Requirement
  • Must have expiration enforcement before implementing token catalog
• Complements: [SECURITY FEATURE]: Gateway-Level Rate Limiting, DDoS Protection & Abuse Detection #257 - Gateway-Level Rate Limiting, DDoS Protection & Abuse Detection
  • Rate limiting for auth endpoints will be handled by [SECURITY FEATURE]: Gateway-Level Rate Limiting, DDoS Protection & Abuse Detection #257

👥 User Stories

Story 1: API Token Management Endpoints

As a developer or automation client
I want RESTful API endpoints to manage tokens
So that I can programmatically create, list, and revoke tokens

Acceptance Criteria:

Scenario: Create token via API
  Given I authenticate with Basic Auth
  When I POST to /auth/tokens with {"description": "CI/CD Token", "expires_in_days": 30}
  Then I receive a 201 response with:
    - Raw JWT token (one-time only)
    - Token metadata (id, description, expires_at)
    - Token includes all required claims (exp, iat, sub, jti, iss, aud)

Scenario: List my tokens
  Given I authenticate with my credentials
  When I GET /auth/tokens
  Then I see all my active tokens with:
    - Metadata only (no raw tokens)
    - Last used timestamp
    - Revocation status

Scenario: Revoke token
  Given I own a token with id "123"
  When I DELETE /auth/tokens/123
  Then the token is immediately revoked
  And subsequent use returns 401 Unauthorized

Story 2: Admin UI Token Management

As a platform administrator
I want a web interface to manage API tokens
So that I can easily issue and revoke tokens without using CLI

Acceptance Criteria:

Scenario: Generate token from Admin UI
  Given I am logged into the Admin UI
  When I navigate to "Admin > API Keys"
  And I click "Generate New Token"
  And I enter description "Production API" and expiry "30 days"
  Then I see the raw JWT once with a copy button
  And it appears in the token table

Scenario: View and manage tokens
  Given I am on the API Keys page
  Then I see a table with columns:
    - Description
    - Created
    - Expires
    - Last Used
    - Status
    - Actions
  And I can filter by active/revoked status
  And I can search by description

UI Mockup:

┌─────────────────────────────────────────────────────────────────────┐
│ Admin > API Keys                                      [+ New Token] │
├─────────────────────────────────────────────────────────────────────┤
│ Filter: [All ▼] [Active] [Revoked]    Search: [_______________] 🔍  │
│                                                                     │
│ Your API Tokens (3 active, 2 revoked)                               │
│ ┌────────────────┬──────────┬──────────┬───────────┬──────┬──────┐  │
│ │ Description    │ Created  │ Expires  │ Last Used │Status│Action│  │
│ ├────────────────┼──────────┼──────────┼───────────┼──────┼──────┤  │
│ │ 🔑 Production  │ Jan 15   │ Feb 15   │ 2 min ago │ ✅   │ 🗑️  │  │
│ │    API Server  │ 10:30 AM │ 10:30 AM │           │      │      │  │
│ ├────────────────┼──────────┼──────────┼───────────┼──────┼──────┤  │
│ │ 🔑 CI/CD       │ Jan 10   │ Jul 10   │ 1 hr ago  │ ✅   │ 🗑️  │  │
│ │    Pipeline    │ 2:15 PM  │ 2:15 PM  │           │      │      │  │
│ ├────────────────┼──────────┼──────────┼───────────┼──────┼──────┤  │
│ │ 🔑 Dev Testing │ Dec 20   │ Jan 20   │ Yesterday │ ⚠️   │ 🗑️  │ │
│ │                │ 9:00 AM  │ 9:00 AM  │           │      │      │  │
│ └────────────────┴──────────┴──────────┴───────────┴──────┴──────┘  │
│                                                                     │
│ ⚠️ = Expiring soon  ❌ = Revoked  ✅ = Active                     │
└─────────────────────────────────────────────────────────────────────┘

[New Token Dialog]
┌─────────────────────────────────────────────────────────────────────┐
│ Generate New API Token                                          [X] │
├─────────────────────────────────────────────────────────────────────┤
│ Description: [_____________________________________]                │
│                                                                     │
│ Expires in:  [30 ▼] days                                            │
│              ├─ 1 day                                               │
│              ├─ 7 days                                              │
│              ├─ 30 days                                             │
│              ├─ 90 days                                             │
│              └─ 365 days                                            │
│                                                                     │
│ ⚠️ This token will have full API access for your account             
│                                                                     │
│ [Cancel]                                           [Generate Token] │
└─────────────────────────────────────────────────────────────────────┘

[Token Generated]
┌─────────────────────────────────────────────────────────────────────┐
│ ✅ API Token Generated                                          [X] │
├─────────────────────────────────────────────────────────────────────┤
│ ⚠️ Copy this token now. You won't see it again!                    │
│                                                                     │
│ ┌─────────────────────────────────────────────────────────┐         │
│ │ eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhZG1p │ [📋]      │
│ │ biIsImp0aSI6IjU1MGU4NDBjLWY1YmUtNDcxNS04OGY5LTJjNmY5    │         │
│ │ ZjQ3ZDYzNCIsImV4cCI6MTcwNzk5NjYzMCwiaWF0IjoxNzA1NDA0Nj │          │
│ └─────────────────────────────────────────────────────────┘         │
│                                                                     │
│ Token ID: 550e840c-f5be-4715-88f9-2c6f9f47d634                      │
│ Expires: February 15, 2024 at 10:30 AM                              │
│                                                                     │
│                                                    [Done]           │
└─────────────────────────────────────────────────────────────────────┘

Story 3: CSRF-Protected Authentication

As a security administrator
I want CSRF protection for cookie-based authentication
So that the API is protected from cross-site request forgery attacks

Acceptance Criteria:

Scenario: Cookie auth requires CSRF token
  Given I have a valid JWT in a cookie
  When I make an API request without CSRF token
  Then the request returns 403 Forbidden
  
Scenario: Valid CSRF token allows request
  Given I have a valid JWT in a cookie
  And I obtained a CSRF token from /auth/csrf-token
  When I include the CSRF token in X-CSRF-Token header
  Then the request succeeds
  
Scenario: Bearer auth bypasses CSRF
  Given I have a valid JWT
  When I use it in Authorization: Bearer header
  Then the request succeeds without CSRF token

Story 4: Enhanced Security Validation

As a security engineer
I want comprehensive token validation
So that only properly formatted, secure tokens are accepted

Acceptance Criteria:

Scenario: Validate all required claims
  Given a JWT token is presented
  Then the system validates:
    - Algorithm is in allowed list (not "none")
    - Required claims present: exp, iat, sub, jti, iss, aud
    - Issuer matches expected value
    - Audience matches expected value
    - Token age (iat) is within acceptable range
    
Scenario: Reject insecure tokens
  Given a token with "none" algorithm
  When I use it for authentication
  Then I receive 401 with generic error message
  And the attempt is logged as security event

Story 5: Multi-User Support with Password Policy

As a platform administrator
I want support for multiple users with secure passwords
So that the system can scale beyond single-user authentication

Acceptance Criteria:

Scenario: Create user with secure password
  Given I am an admin
  When I create a new user
  Then the password must meet policy:
    - Minimum 12 characters
    - Mix of uppercase, lowercase, numbers, symbols
    - Not in common password list
    - Stored using bcrypt with cost factor 12
    
Scenario: Each user manages own tokens
  Given multiple users exist
  When user A creates tokens
  Then only user A can see and revoke them
  And user B cannot access user A's tokens

Story 6: Security Audit Trail

As a compliance officer
I want comprehensive logging of all authentication events
So that I can audit access and detect anomalies

Acceptance Criteria:

Scenario: Log all auth events
  Given authentication logging is enabled
  Then the system logs:
    - Successful logins with timestamp, user, IP
    - Failed attempts with reason
    - Token creation/revocation events
    - CSRF validation failures
    - Algorithm security violations
    
Scenario: View audit logs
  Given I have audit permissions
  When I access "Admin > Audit Logs"
  Then I can search and filter by:
    - Date range
    - User
    - Event type
    - Success/failure

📊 Architecture

flowchart TB
    subgraph "Token Creation Flow"
        U[User/Admin] -->|Generate Token| API[POST /auth/tokens]
        API -->|Create JWT with jti| JWT[JWT Service]
        JWT -->|Validate claims| CLAIMS[Claims Validator]
        CLAIMS -->|Store hash + metadata| DB[(api_tokens DB)]
        JWT -->|Log event| LOG[(audit_logs DB)]
        JWT -->|Return once| U
    end
    
    subgraph "Token Validation Flow"
        C[Client] -->|Bearer Token| GW[Gateway API]
        GW -->|1. Rate limit check| RL[Rate Limiter #257]
        RL -->|2. Algorithm check| ALG[Algorithm Validator]
        ALG -->|3. Verify signature| VS[verify_jwt_token]
        VS -->|4. Validate claims| CLAIMS2[iss, aud, exp, iat]
        CLAIMS2 -->|5. Check CSRF if cookie| CSRF[CSRF Protection]
        CSRF -->|6. Lookup jti| DB
        DB -->|7. Check revoked| GW
        GW -->|Log attempt| LOG
        GW -->|Allow/Deny| C
    end
    
    subgraph "User Management"
        ADMIN[Admin] -->|Manage Users| UM[User Management]
        UM -->|Create with policy| UP[Password Policy]
        UP -->|Bcrypt hash| UDB[(users DB)]
    end
    
    style JWT fill:#90EE90
    style DB fill:#87CEEB
    style VS fill:#FFB6C1
    style LOG fill:#DDA0DD
    style CSRF fill:#FFD700
    style RL fill:#FF6B6B

Loading

🏗️ Technical Design

Database Schema

-- Enhanced users table for multi-user support
CREATE TABLE users (
    id INTEGER PRIMARY KEY,
    username VARCHAR(255) UNIQUE NOT NULL,
    password_hash VARCHAR(255) NOT NULL,  -- bcrypt with cost 12
    email VARCHAR(255),
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    last_login TIMESTAMP,
    is_active BOOLEAN DEFAULT TRUE,
    is_admin BOOLEAN DEFAULT FALSE,
    password_changed_at TIMESTAMP,
    failed_login_attempts INTEGER DEFAULT 0,
    locked_until TIMESTAMP,
    INDEX idx_username (username)
);

-- Token catalog
CREATE TABLE api_tokens (
    id INTEGER PRIMARY KEY,
    user_id INTEGER NOT NULL REFERENCES users(id),
    jti UUID UNIQUE NOT NULL,
    token_hash CHAR(64) UNIQUE NOT NULL,  -- SHA-256 hash
    description TEXT,
    issued_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    expires_at TIMESTAMP NOT NULL,
    last_used TIMESTAMP,
    revoked BOOLEAN DEFAULT FALSE,
    revoked_at TIMESTAMP,
    revoked_by INTEGER REFERENCES users(id),
    created_ip VARCHAR(45),
    INDEX idx_jti (jti),
    INDEX idx_user_active (user_id, revoked, expires_at)
);

-- Security audit trail
CREATE TABLE auth_logs (
    id INTEGER PRIMARY KEY,
    timestamp TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    event_type VARCHAR(50) NOT NULL,  -- login, logout, token_created, token_revoked, auth_failed, csrf_failed
    user_id INTEGER REFERENCES users(id),
    username VARCHAR(255),  -- For failed logins where user_id unknown
    ip_address VARCHAR(45),
    user_agent TEXT,
    token_jti UUID,
    success BOOLEAN,
    failure_reason VARCHAR(255),
    additional_data JSON,  -- For extra context
    INDEX idx_timestamp (timestamp),
    INDEX idx_user (user_id),
    INDEX idx_event (event_type),
    INDEX idx_ip (ip_address)
);

Enhanced Security Configuration

# settings.py additions
class Settings(BaseSettings):
    # JWT Security
    jwt_issuer: str = Field(default="mcpgateway", description="JWT issuer claim")
    jwt_audience: str = Field(default="mcpgateway-api", description="JWT audience claim")
    jwt_max_age_hours: int = Field(default=24, description="Maximum token age in hours")
    allowed_algorithms: List[str] = Field(
        default=["HS256", "HS384", "HS512", "RS256", "RS384", "RS512"],
        description="Allowed JWT algorithms (never include 'none')"
    )
    
    # CSRF Protection
    csrf_token_name: str = Field(default="csrf-token", description="CSRF cookie name")
    csrf_header_name: str = Field(default="X-CSRF-Token", description="CSRF header name")
    
    # Password Policy
    password_min_length: int = Field(default=12, description="Minimum password length")
    password_require_uppercase: bool = Field(default=True)
    password_require_lowercase: bool = Field(default=True)
    password_require_numbers: bool = Field(default=True)
    password_require_special: bool = Field(default=True)
    password_bcrypt_rounds: int = Field(default=12, description="Bcrypt cost factor")
    
    # Security Logging
    enable_auth_logging: bool = Field(default=True, description="Enable authentication audit logs")
    auth_log_retention_days: int = Field(default=90, description="Days to retain auth logs")

Security Implementation Details

Password Policy Validator

class PasswordPolicy:
    def validate(self, password: str) -> List[str]:
        errors = []
        
        if len(password) < settings.password_min_length:
            errors.append(f"Password must be at least {settings.password_min_length} characters")
            
        if settings.password_require_uppercase and not re.search(r'[A-Z]', password):
            errors.append("Password must contain uppercase letters")
            
        if settings.password_require_lowercase and not re.search(r'[a-z]', password):
            errors.append("Password must contain lowercase letters")
            
        if settings.password_require_numbers and not re.search(r'\d', password):
            errors.append("Password must contain numbers")
            
        if settings.password_require_special and not re.search(r'[!@#$%^&*(),.?":{}|<>]', password):
            errors.append("Password must contain special characters")
            
        # Check common passwords
        if password.lower() in COMMON_PASSWORDS_SET:
            errors.append("Password is too common")
            
        return errors
    
    def hash_password(self, password: str) -> str:
        return bcrypt.hashpw(
            password.encode('utf-8'),
            bcrypt.gensalt(rounds=settings.password_bcrypt_rounds)
        ).decode('utf-8')

Enhanced JWT Validation

async def verify_jwt_token(token: str, request: Request = None) -> dict:
    try:
        # Pre-validation: Check algorithm
        header = jwt.get_unverified_header(token)
        if header.get("alg") not in settings.allowed_algorithms:
            await log_auth_event("auth_failed", None, "Invalid algorithm", request=request)
            raise HTTPException(status_code=401, detail="Invalid token")
        
        # Decode with full validation
        payload = jwt.decode(
            token,
            settings.jwt_secret_key,
            algorithms=[settings.jwt_algorithm],
            options={"require": ["exp", "iat", "sub", "jti"]},
            audience=settings.jwt_audience,
            issuer=settings.jwt_issuer
        )
        
        # Additional validations
        # 1. Check token age
        token_age = datetime.utcnow() - datetime.fromtimestamp(payload["iat"])
        if token_age > timedelta(hours=settings.jwt_max_age_hours):
            await log_auth_event("auth_failed", payload.get("sub"), "Token too old", request=request)
            raise HTTPException(status_code=401, detail="Invalid token")
        
        # 2. Check revocation status
        if "jti" in payload:
            async with get_db() as db:
                token_record = await db.query(ApiToken).filter_by(jti=payload["jti"]).first()
                if token_record and token_record.revoked:
                    await log_auth_event("auth_failed", payload.get("sub"), "Token revoked", request=request)
                    raise HTTPException(status_code=401, detail="Invalid token")
                
                # Update last used
                if token_record:
                    token_record.last_used = datetime.utcnow()
                    await db.commit()
        
        await log_auth_event("auth_success", payload.get("sub"), "Token validated", request=request)
        return payload
        
    except jwt.ExpiredSignatureError:
        await log_auth_event("auth_failed", None, "Token expired", request=request)
        raise HTTPException(status_code=401, detail="Invalid token")
    except Exception as e:
        await log_auth_event("auth_failed", None, f"Token validation error: {type(e).__name__}", request=request)
        raise HTTPException(status_code=401, detail="Invalid token")

🛠️ Implementation Tasks

Phase 1: Security Foundation

• Add password policy implementation with bcrypt hashing
• Create users table migration with secure schema
• Implement algorithm validation blocking "none"
• Add comprehensive JWT claims validation
• Create auth_logs table for audit trail
• Implement structured logging service

Phase 2: Token Catalog Core

• Create api_tokens table migration
• Implement auth_token_service.py with:
  • create_token() - Generate JWT with all claims
  • list_tokens() - User-scoped token listing
  • revoke_token() - Immediate revocation
  • verify_token() - Check revocation status
• Update verify_jwt_token() to check catalog
• Add backward compatibility for legacy tokens

Phase 3: API Implementation

• Create /auth/tokens router with:
  • POST /auth/tokens - Create with validation
  • GET /auth/tokens - List user's tokens
  • DELETE /auth/tokens/{id} - Revoke token
  • GET /auth/csrf-token - Get CSRF token
• Add request/response schemas
• Implement proper error responses
• Add OpenAPI documentation

Phase 4: CSRF Protection

• Implement double-submit cookie pattern
• Add CSRF validation to require_auth()
• Create /auth/csrf-token endpoint
• Update Admin UI to include CSRF tokens

Phase 5: Admin UI

• Create "API Keys" tab with:
  • Token generation form
  • Token list table with search/filter
  • One-time token display modal
  • Revocation with confirmation
• Create "Users" tab for user management
• Create "Audit Logs" tab with filtering
• Integrate CSRF tokens in all forms

Phase 6: Testing & Documentation

• Unit tests for password policy
• Security tests for algorithm validation
• CSRF protection tests
• Token lifecycle integration tests
• Audit logging verification tests
• Document security architecture
• Create API usage examples

📋 Acceptance Criteria

Security Requirements

• Password policy enforced for all users
• All tokens include required claims: exp, iat, sub, jti, iss, aud
• "none" algorithm always rejected
• CSRF tokens required for cookie auth
• All auth events logged with metadata
• Generic error messages prevent info leakage
• Rate limiting via [SECURITY FEATURE]: Gateway-Level Rate Limiting, DDoS Protection & Abuse Detection #257 integration

Functionality

• Token CRUD operations work correctly
• Multi-user isolation enforced
• Legacy tokens work with warnings
• Audit logs searchable and exportable
• Performance < 50ms overhead

User Experience

• Intuitive token management UI
• Clear password policy feedback
• One-time token display works
• Helpful error messages
• Responsive design

🚫 Out of Scope

• OAuth2/OIDC integration (that is covered in [AUTH FEATURE]: Authentication & Authorization - SSO + Identity-Provider Integration #220)
• Token scoping/permissions
• Automated token rotation
• External identity providers
• Custom password policies per user

📊 Success Metrics

• 100% tokens have security claims
• All auth attempts logged
• Password policy compliance 100%
• < 100ms security overhead

🔗 MCP Standards Check

• ✅ Does not affect MCP protocol
• ✅ No impact on message format
• ✅ Enhanced security without breaking compatibility

📝 Notes

• Depends on [SECURITY FEATURE]: Make JWT Token Expiration Mandatory when REQUIRE_TOKEN_EXPIRATION=true (depends on #87) #425 for expiration configuration
• Rate limiting handled by [SECURITY FEATURE]: Gateway-Level Rate Limiting, DDoS Protection & Abuse Detection #257
• Default settings maintain compatibility
• Update docs/docs/deployment and docs/docs/manage with admin info
• Update helm charts with new settings
• Password policy can be adjusted per deployment

[phmcder]

The MCP spec has now been updated to remove the need for an MCP server to manage their own tokens and have returned that role to an external authrisation service such as Entra/Cognito/Keycloak etc.

The issue with the old standard is laid out here https://blog.christianposta.com/the-updated-mcp-oauth-spec-is-a-mess/

The conversation over the spec changes can be found here modelcontextprotocol/modelcontextprotocol#205

[crivetimihai]

Yup.... but we still have to ensure backwards-compatibility with older specification versions.