MCP Server Catalog Improvements

[crivetimihai]

Pull Request: MCP Server Catalog Improvements

Closes #1143

Summary

This PR comprehensively enhances the MCP Server Catalog feature with 21 critical fixes and improvements, addressing transport detection, authentication, UI/UX, and developer experience issues. All tests passing with perfect quality scores.

Changes Overview

🔧 Core Functionality Fixes (10 fixes)

1. WebSocket Transport Detection - Fixed auto-detection logic to correctly identify ws:// and wss:// URLs as WEBSOCKET transport instead of incorrectly mapping to SSE

2. Transport Field Support - Added optional transport field to CatalogServer schema allowing explicit specification of SSE, STREAMABLEHTTP, or WEBSOCKET

3. Auth Headers Mapping - Fixed authentication headers to use proper list structure [{"key": "X-API-Key", "value": "..."}] instead of individual key/value fields

4. OAuth Server Registration - Implemented ability to register OAuth2.1 servers without immediate credentials, creating them in disabled state until OAuth flow is completed

5. URL Normalization - Verified URL normalization handles trailing slashes, paths, and protocols correctly

6. Server Availability Tracking - Fixed is_available field logic to properly reflect registration and health status

7. Provider Filter Normalization - Normalized provider names for consistent filtering across the catalog

8. Root Path Handling - Standardized window.ROOT_PATH global variable for consistent URL construction in JavaScript

9. Configurable Page Size - Added MCPGATEWAY_CATALOG_PAGE_SIZE environment variable (default: 12) for customizable pagination

10. Import Path Fix - Corrected import from mcpgateway.utils.slug to mcpgateway.utils.create_slug.slugify

🎨 UI/UX Enhancements (7 fixes)

11. Custom Name in Registration Modal - Added visible input field allowing users to specify custom server names during registration

12. Authorization Bearer Header - Added JWT token to fetch requests via Authorization: Bearer header for authenticated API calls

13. HTMX Error Handling - Implemented hx-on::after-request error handling with user-friendly alert messages on registration failures

14. Loading States - Added visual loading indicators during server registration with spinner and "Registering..." text

15. Search Debouncing - Implemented 300ms debounce on search input to prevent excessive DOM manipulation and improve performance

16. Pagination Filter Preservation - Filter parameters (category, auth_type, search) now preserved when navigating between pages

17. Refresh Button - Added manual catalog refresh button to reload server list without full page reload

⚡ Developer Experience (4 fixes)

18. Better Error Messages - User-friendly error mapping for common issues:

• Connection errors: "Server is offline or unreachable"
• SSL errors: "SSL certificate verification failed"
• Timeout errors: "Server took too long to respond"
• Auth errors: "Authentication failed - check API key"
• 404 errors: "Server endpoint not found"
• IPv6 errors: "IPv6 URLs are not supported"

19. Code Deduplication - Consolidated duplicate code paths in registration and filtering logic

20. Environment Documentation - Updated .env.example with MCPGATEWAY_CATALOG_PAGE_SIZE configuration

21. Comprehensive Documentation - Updated docs/docs/manage/catalog.md with:

• Transport field documentation and examples
• WebSocket server configuration
• Recent improvements section
• Transport troubleshooting guide

Files Modified

Code Changes (7 files)

• .env.example - Added catalog page size configuration
• mcpgateway/config.py - Added mcpgateway_catalog_page_size setting
• mcpgateway/schemas.py - Added optional transport field to CatalogServer
• mcpgateway/services/catalog_service.py - Core fixes for transport detection, OAuth registration, error handling
• mcpgateway/admin.py - Filter preservation for pagination
• mcpgateway/templates/mcp_registry_partial.html - UI enhancements, modal improvements, debouncing, refresh button

Documentation Updates (2 files)

• README.md - Added catalog page size config and key features section
• docs/docs/manage/catalog.md - Comprehensive transport field docs, WebSocket examples, improvements section, troubleshooting

Total: +305 net lines across 9 files

Transport Detection Logic

The catalog service now intelligently detects transport types:

# Auto-detection priority (when transport field not explicitly set):
if url.startswith('ws://') or url.startswith('wss://'):
    transport = 'WEBSOCKET'
elif url.endswith('/sse') or '/sse/' in url:
    transport = 'SSE'
elif '/mcp' in url:
    transport = 'STREAMABLEHTTP'
else:
    transport = 'SSE'  # Default fallback

Servers can also explicitly specify transport in catalog YAML:

- id: "websocket-server"
  url: "wss://api.example.com/mcp"
  transport: "WEBSOCKET"  # Explicit specification

Test Results

✅ All Tests Passing

• Doctest: 769 passed, 7 skipped
• Unit Tests: 3377 passed, 45 skipped
• Coverage: 74% overall

✅ Perfect Quality Scores

• Flake8: 0 issues
• Bandit: 0 security issues
• Interrogate: 100% docstrings (2067/2067)
• Pylint: 10.00/10 (perfect score)
• Verify: Package ready to publish

Breaking Changes

None - all changes are backward compatible.

Migration Guide

No migration required. Existing catalog servers will continue to work with auto-detection. Optional: Add explicit transport field to catalog YAML for clarity:

catalog_servers:
  - id: "my-server"
    url: "wss://api.example.com/ws"
    transport: "WEBSOCKET"  # Optional but recommended for WebSocket servers

Related Issues

Closes #1143 - MCP Server Catalog Improvements