Skip to content

Commit 785c607

Browse files
authored
MCP Server Catalog Improvements (#1170)
* Update catalog Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Update catalog Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Update catalog Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Update catalog Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Update catalog Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> --------- Signed-off-by: Mihai Criveti <crivetimihai@gmail.com>
1 parent 3bec931 commit 785c607

File tree

10 files changed

+419
-303
lines changed

10 files changed

+419
-303
lines changed

.env.example

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -395,6 +395,10 @@ MCPGATEWAY_CATALOG_AUTO_HEALTH_CHECK=true
395395
# Default: 3600 (1 hour)
396396
MCPGATEWAY_CATALOG_CACHE_TTL=3600
397397

398+
# Number of catalog servers to display per page
399+
# Default: 100
400+
MCPGATEWAY_CATALOG_PAGE_SIZE=100
401+
398402
#####################################
399403
# Header Passthrough Configuration
400404
#####################################

README.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -461,7 +461,7 @@ When using a MCP Client such as Claude with stdio:
461461

462462
## Quick Start - Containers
463463

464-
Use the official OCI image from GHCR with **Docker** *or* **Podman**.
464+
Use the official OCI image from GHCR with **Docker** *or* **Podman**.
465465
Please note: Currently, arm64 is not supported. If you are e.g. running on MacOS, install via PyPi.
466466

467467
---
@@ -1307,6 +1307,15 @@ ContextForge implements **OAuth 2.0 Dynamic Client Registration (RFC 7591)** and
13071307
| `MCPGATEWAY_CATALOG_FILE` | Path to catalog configuration file | `mcp-catalog.yml` | string |
13081308
| `MCPGATEWAY_CATALOG_AUTO_HEALTH_CHECK` | Automatically health check catalog servers | `true` | bool |
13091309
| `MCPGATEWAY_CATALOG_CACHE_TTL` | Catalog cache TTL in seconds | `3600` | int > 0 |
1310+
| `MCPGATEWAY_CATALOG_PAGE_SIZE` | Number of catalog servers per page | `12` | int > 0 |
1311+
1312+
**Key Features:**
1313+
- 🔄 Refresh Button - Manually refresh catalog without page reload
1314+
- 🔍 Debounced Search - Optimized search with 300ms debounce
1315+
- 📝 Custom Server Names - Specify custom names when registering
1316+
- 🔌 Transport Detection - Auto-detect SSE, WebSocket, or HTTP transports
1317+
- 🔐 OAuth Support - Register OAuth servers and configure later
1318+
- ⚡ Better Error Messages - User-friendly errors for common issues
13101319

13111320
**Documentation:**
13121321
- [MCP Server Catalog Guide](https://ibm.github.io/mcp-context-forge/manage/catalog/) - Complete catalog setup and configuration

docs/docs/manage/catalog.md

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ MCPGATEWAY_CATALOG_AUTO_HEALTH_CHECK=true
3434

3535
# Catalog cache TTL in seconds (default: 3600)
3636
MCPGATEWAY_CATALOG_CACHE_TTL=3600
37+
38+
# Number of catalog servers to display per page (default: 12)
39+
MCPGATEWAY_CATALOG_PAGE_SIZE=12
3740
```
3841

3942
---
@@ -94,6 +97,7 @@ catalog_servers:
9497
name: "Production Time Server"
9598
category: "Utilities"
9699
url: "https://time.api.example.com/sse"
100+
transport: "SSE" # Optional: Explicitly specify transport type
97101
auth_type: "OAuth2.1"
98102
provider: "Internal Platform"
99103
description: "Production time server with geo-replication"
@@ -107,6 +111,21 @@ catalog_servers:
107111
logo_url: "https://static.example.com/time-server-logo.png"
108112
documentation_url: "https://docs.example.com/time-server"
109113

114+
- id: "websocket-server"
115+
name: "WebSocket MCP Server"
116+
category: "Development Tools"
117+
url: "wss://api.example.com/mcp"
118+
transport: "WEBSOCKET" # Specify WebSocket transport
119+
auth_type: "API Key"
120+
provider: "Internal Platform"
121+
description: "Real-time MCP server using WebSocket protocol"
122+
requires_api_key: true
123+
secure: true
124+
tags:
125+
- "production"
126+
- "websocket"
127+
- "real-time"
128+
110129
- id: "database-server"
111130
name: "Database Server"
112131
category: "Database"
@@ -177,6 +196,7 @@ Based on the `CatalogServer` schema (schemas.py:5371-5387):
177196
| `requires_api_key` | boolean | No | Whether API key is required (default: `false`) |
178197
| `secure` | boolean | No | Whether additional security is required (default: `false`) |
179198
| `tags` | array | No | Tags for categorization (default: `[]`) |
199+
| `transport` | string | No | Transport type: `SSE`, `STREAMABLEHTTP`, or `WEBSOCKET` (auto-detected if not specified) |
180200
| `logo_url` | string | No | URL to server logo/icon |
181201
| `documentation_url` | string | No | URL to server documentation |
182202
| `is_registered` | boolean | No | Whether server is already registered (set by system) |
@@ -448,6 +468,57 @@ auth_types:
448468

449469
3. For API Key servers, provide the API key during registration
450470

471+
### Transport Type Issues
472+
473+
**Symptoms:** WebSocket servers fail to connect after registration
474+
475+
**Solutions:**
476+
477+
1. Explicitly specify the `transport` field in your catalog YAML:
478+
```yaml
479+
catalog_servers:
480+
- id: "websocket-server"
481+
url: "wss://api.example.com/mcp"
482+
transport: "WEBSOCKET" # Explicitly set transport
483+
```
484+
485+
2. Verify URL scheme matches transport type:
486+
- WebSocket: `ws://` or `wss://`
487+
- SSE: `http://` or `https://` with `/sse` path
488+
- HTTP: `http://` or `https://` with `/mcp` path
489+
490+
---
491+
492+
## Recent Improvements (v0.7.0)
493+
494+
### Enhanced UI Features
495+
496+
The catalog UI now includes several UX improvements:
497+
498+
- **🔄 Refresh Button**: Manually refresh the catalog without page reload
499+
- **🔍 Debounced Search**: 300ms debounce on search input for better performance
500+
- **📝 Custom Server Names**: Ability to specify custom names when registering servers
501+
- **📄 Pagination with Filters**: Filter parameters preserved when navigating pages
502+
- **⚡ Better Error Messages**: User-friendly error messages for common issues (connection, auth, SSL, etc.)
503+
- **🔐 OAuth Support**: OAuth servers can be registered without credentials and configured later
504+
505+
### Transport Type Detection
506+
507+
The catalog now supports:
508+
509+
- **Explicit Transport**: Specify `transport` field in catalog YAML (`SSE`, `WEBSOCKET`, `STREAMABLEHTTP`)
510+
- **Auto-Detection**: Automatically detects transport from URL if not specified
511+
- `ws://` or `wss://` → `WEBSOCKET`
512+
- URLs ending in `/sse` → `SSE`
513+
- URLs with `/mcp` path → `STREAMABLEHTTP`
514+
- Default fallback → `SSE`
515+
516+
### Authentication Improvements
517+
518+
- **Custom Auth Headers**: Properly mapped as list of header key-value pairs
519+
- **OAuth Registration**: OAuth servers can be registered in "disabled" state until OAuth flow is completed
520+
- **API Key Modal**: Enhanced modal with custom name field and proper authorization headers
521+
451522
---
452523

453524
## See Also

mcpgateway/admin.py

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9801,34 +9801,45 @@ async def catalog_partial(
98019801
root_path = request.scope.get("root_path", "")
98029802

98039803
# Calculate pagination
9804-
page_size = 100
9804+
page_size = settings.mcpgateway_catalog_page_size
98059805
offset = (page - 1) * page_size
98069806

98079807
catalog_request = CatalogListRequest(category=category, auth_type=auth_type, search=search, show_available_only=False, limit=page_size, offset=offset)
98089808

98099809
response = await catalog_service.get_catalog_servers(catalog_request, db)
98109810

9811+
# Get ALL servers (no filters, no pagination) for counting statistics
9812+
all_servers_request = CatalogListRequest(show_available_only=False, limit=1000, offset=0)
9813+
all_servers_response = await catalog_service.get_catalog_servers(all_servers_request, db)
9814+
9815+
# Pass filter parameters to template for pagination links
9816+
filter_params = {
9817+
"category": category,
9818+
"auth_type": auth_type,
9819+
"search": search,
9820+
}
9821+
98119822
# Calculate statistics and pagination info
98129823
total_servers = response.total
98139824
registered_count = sum(1 for s in response.servers if s.is_registered)
98149825
total_pages = (total_servers + page_size - 1) // page_size # Ceiling division
98159826

9816-
# Count servers by category, auth type, and provider
9827+
# Count ALL servers by category, auth type, and provider (not just current page)
98179828
servers_by_category = {}
98189829
servers_by_auth_type = {}
98199830
servers_by_provider = {}
98209831

9821-
for server in response.servers:
9832+
for server in all_servers_response.servers:
98229833
servers_by_category[server.category] = servers_by_category.get(server.category, 0) + 1
98239834
servers_by_auth_type[server.auth_type] = servers_by_auth_type.get(server.auth_type, 0) + 1
98249835
servers_by_provider[server.provider] = servers_by_provider.get(server.provider, 0) + 1
98259836

98269837
stats = {
9827-
"total_servers": total_servers,
9838+
"total_servers": all_servers_response.total, # Use total from all servers
98289839
"registered_servers": registered_count,
9829-
"categories": response.categories,
9830-
"auth_types": response.auth_types,
9831-
"providers": response.providers,
9840+
"categories": all_servers_response.categories,
9841+
"auth_types": all_servers_response.auth_types,
9842+
"providers": all_servers_response.providers,
98329843
"servers_by_category": servers_by_category,
98339844
"servers_by_auth_type": servers_by_auth_type,
98349845
"servers_by_provider": servers_by_provider,
@@ -9842,6 +9853,7 @@ async def catalog_partial(
98429853
"page": page,
98439854
"total_pages": total_pages,
98449855
"page_size": page_size,
9856+
"filter_params": filter_params,
98459857
}
98469858

98479859
return request.app.state.templates.TemplateResponse("mcp_registry_partial.html", context)

mcpgateway/config.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -311,6 +311,7 @@ class Settings(BaseSettings):
311311
mcpgateway_catalog_file: str = Field(default="mcp-catalog.yml", description="Path to catalog configuration file")
312312
mcpgateway_catalog_auto_health_check: bool = Field(default=True, description="Automatically health check catalog servers")
313313
mcpgateway_catalog_cache_ttl: int = Field(default=3600, description="Catalog cache TTL in seconds")
314+
mcpgateway_catalog_page_size: int = Field(default=100, description="Number of catalog servers per page")
314315

315316
# Security
316317
skip_ssl_verify: bool = False

mcpgateway/schemas.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5381,6 +5381,7 @@ class CatalogServer(BaseModel):
53815381
requires_api_key: bool = Field(default=False, description="Whether API key is required")
53825382
secure: bool = Field(default=False, description="Whether additional security is required")
53835383
tags: List[str] = Field(default_factory=list, description="Tags for categorization")
5384+
transport: Optional[str] = Field(None, description="Transport type: SSE, STREAMABLEHTTP, or WEBSOCKET")
53845385
logo_url: Optional[str] = Field(None, description="URL to server logo/icon")
53855386
documentation_url: Optional[str] = Field(None, description="URL to server documentation")
53865387
is_registered: bool = Field(default=False, description="Whether server is already registered")

mcpgateway/services/catalog_service.py

Lines changed: 92 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@
3131
CatalogServerStatusResponse,
3232
)
3333
from mcpgateway.services.gateway_service import GatewayService
34+
from mcpgateway.utils.create_slug import slugify
3435

3536
logger = logging.getLogger(__name__)
3637

@@ -120,6 +121,9 @@ async def get_catalog_servers(self, request: CatalogListRequest, db) -> CatalogL
120121
for server_data in servers:
121122
server = CatalogServer(**server_data)
122123
server.is_registered = server.url in registered_urls
124+
# Set availability based on registration status (registered servers are assumed available)
125+
# Individual health checks can be done via the /status endpoint
126+
server.is_available = server.is_registered or server_data.get("is_available", True)
123127
catalog_servers.append(server)
124128

125129
# Apply filters
@@ -206,18 +210,22 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
206210
# First-Party
207211
from mcpgateway.schemas import GatewayCreate # pylint: disable=import-outside-toplevel
208212

209-
# Detect transport type from URL or use SSE as default
210-
url = server_data["url"].lower()
211-
# Check for SSE patterns (highest priority)
212-
if url.endswith("/sse") or "/sse/" in url:
213-
transport = "SSE" # SSE endpoints or paths containing /sse/
214-
elif url.startswith("ws://") or url.startswith("wss://"):
215-
transport = "SSE" # WebSocket URLs typically use SSE transport
216-
# Then check for HTTP patterns
217-
elif "/mcp" in url or url.endswith("/"):
218-
transport = "STREAMABLEHTTP" # Generic MCP endpoints typically use HTTP
219-
else:
220-
transport = "SSE" # Default to SSE for most catalog servers
213+
# Use explicit transport if provided, otherwise auto-detect from URL
214+
transport = server_data.get("transport")
215+
if not transport:
216+
# Detect transport type from URL or use SSE as default
217+
url = server_data["url"].lower()
218+
# Check for WebSocket patterns (highest priority)
219+
if url.startswith("ws://") or url.startswith("wss://"):
220+
transport = "WEBSOCKET" # WebSocket transport for ws:// and wss:// URLs
221+
# Check for SSE patterns
222+
elif url.endswith("/sse") or "/sse/" in url:
223+
transport = "SSE" # SSE endpoints or paths containing /sse/
224+
# Then check for HTTP patterns
225+
elif "/mcp" in url or url.endswith("/"):
226+
transport = "STREAMABLEHTTP" # Generic MCP endpoints typically use HTTP
227+
else:
228+
transport = "SSE" # Default to SSE for most catalog servers
221229

222230
# Check for IPv6 URLs early to provide a clear error message
223231
url = server_data["url"]
@@ -237,6 +245,8 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
237245

238246
# Set authentication based on server requirements
239247
auth_type = server_data.get("auth_type", "Open")
248+
skip_initialization = False # Flag to skip connection test for OAuth servers without creds
249+
240250
if request and request.api_key and auth_type != "Open":
241251
# Handle all possible auth types from the catalog
242252
if auth_type in ["API Key", "API"]:
@@ -248,10 +258,54 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
248258
gateway_data["auth_type"] = "bearer"
249259
gateway_data["auth_token"] = request.api_key
250260
else:
251-
# For any other auth types, use custom headers
261+
# For any other auth types, use custom headers (as list of dicts)
252262
gateway_data["auth_type"] = "authheaders"
253-
gateway_data["auth_header_key"] = "X-API-Key"
254-
gateway_data["auth_header_value"] = request.api_key
263+
gateway_data["auth_headers"] = [{"key": "X-API-Key", "value": request.api_key}]
264+
elif auth_type in ["OAuth2.1", "OAuth"]:
265+
# OAuth server without credentials - register but skip initialization
266+
# User will need to complete OAuth flow later
267+
skip_initialization = True
268+
logger.info(f"Registering OAuth server {server_data['name']} without credentials - OAuth flow required later")
269+
270+
# For OAuth servers without credentials, register directly without connection test
271+
if skip_initialization:
272+
# Create minimal gateway entry without tool discovery
273+
# First-Party
274+
from mcpgateway.db import Gateway as DbGateway # pylint: disable=import-outside-toplevel
275+
276+
gateway_create = GatewayCreate(**gateway_data)
277+
slug_name = slugify(gateway_data["name"])
278+
279+
db_gateway = DbGateway(
280+
name=gateway_data["name"],
281+
slug=slug_name,
282+
url=gateway_data["url"],
283+
description=gateway_data["description"],
284+
tags=gateway_data.get("tags", []),
285+
transport=gateway_data["transport"],
286+
capabilities={},
287+
auth_type=None, # Will be set during OAuth configuration
288+
enabled=False, # Disabled until OAuth is configured
289+
created_via="catalog",
290+
visibility="public",
291+
version=1,
292+
)
293+
294+
db.add(db_gateway)
295+
db.commit()
296+
db.refresh(db_gateway)
297+
298+
# First-Party
299+
from mcpgateway.schemas import GatewayRead # pylint: disable=import-outside-toplevel
300+
301+
gateway_read = GatewayRead.model_validate(db_gateway)
302+
303+
return CatalogServerRegisterResponse(
304+
success=True,
305+
server_id=str(gateway_read.id),
306+
message=f"Successfully registered {gateway_read.name} - OAuth configuration required before activation",
307+
error=None,
308+
)
255309

256310
gateway_create = GatewayCreate(**gateway_data)
257311

@@ -284,9 +338,31 @@ async def register_catalog_server(self, catalog_id: str, request: Optional[Catal
284338

285339
except Exception as e:
286340
logger.error(f"Failed to register catalog server {catalog_id}: {e}")
341+
342+
# Map common exceptions to user-friendly messages
343+
error_str = str(e)
344+
user_message = "Registration failed"
345+
346+
if "Connection refused" in error_str or "connect" in error_str.lower():
347+
user_message = "Server is offline or unreachable"
348+
elif "SSL" in error_str or "certificate" in error_str.lower():
349+
user_message = "SSL certificate verification failed - check server security settings"
350+
elif "timeout" in error_str.lower() or "timed out" in error_str.lower():
351+
user_message = "Server took too long to respond - it may be slow or unavailable"
352+
elif "401" in error_str or "Unauthorized" in error_str:
353+
user_message = "Authentication failed - check API key or OAuth credentials"
354+
elif "403" in error_str or "Forbidden" in error_str:
355+
user_message = "Access forbidden - check permissions and API key"
356+
elif "404" in error_str or "Not Found" in error_str:
357+
user_message = "Server endpoint not found - check URL is correct"
358+
elif "500" in error_str or "Internal Server Error" in error_str:
359+
user_message = "Remote server error - the MCP server is experiencing issues"
360+
elif "IPv6" in error_str:
361+
user_message = "IPv6 URLs are not supported - please use IPv4 or domain names"
362+
287363
# Don't rollback here - let FastAPI handle it
288364
# db.rollback()
289-
return CatalogServerRegisterResponse(success=False, server_id="", message="Registration failed", error=str(e))
365+
return CatalogServerRegisterResponse(success=False, server_id="", message=user_message, error=error_str)
290366

291367
async def check_server_availability(self, catalog_id: str) -> CatalogServerStatusResponse:
292368
"""Check if a catalog server is available.

mcpgateway/static/admin.js

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4798,7 +4798,7 @@ function showTab(tabName) {
47984798
if (tabName === "mcp-registry") {
47994799
// Load MCP Registry content
48004800
const registryContent = safeGetElement(
4801-
"mcp-registry-content",
4801+
"mcp-registry-servers",
48024802
);
48034803
if (registryContent) {
48044804
// Always load on first visit or if showing loading message
@@ -4820,7 +4820,7 @@ function showTab(tabName) {
48204820
"GET",
48214821
`${rootPath}/admin/mcp-registry/partial`,
48224822
{
4823-
target: "#mcp-registry-content",
4823+
target: "#mcp-registry-servers",
48244824
swap: "innerHTML",
48254825
},
48264826
)

0 commit comments

Comments
 (0)