Skip to content

Commit a492834

Browse files
vk-playgroundcrivetimihai
vk-playground
and
crivetimihai
authored
feat: Bulk Import Tools modal wiring IBM#737 (IBM#739)
* feat: Bulk Import Tools modal wiring and backend implementation - Add modal UI in admin.html with bulk import button and dialog - Implement modal open/close/ESC functionality in admin.js - Add POST /admin/tools/import endpoint with rate limiting - Support both JSON textarea and file upload inputs - Validate JSON structure and enforce 200 tool limit - Return detailed success/failure information per tool - Include loading states and comprehensive error handling Refs IBM#737 Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * fix: Remove duplicate admin_import_tools function and fix HTML formatting - Remove duplicate admin_import_tools function definition - Fix HTML placeholder attribute to use double quotes - Add missing closing div tag - Fix flake8 blank line issues Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * feat: Complete bulk import backend with file upload support and enhanced docs - Add file upload support to admin_import_tools endpoint - Fix response format to match frontend expectations - Add UI usage documentation with modal instructions - Update API docs to show all three input methods - Enhance bulk import guide with UI and API examples Backend improvements: - Support tools_file form field for JSON file uploads - Proper file content parsing with error handling - Response includes imported/failed counts and details - Frontend-compatible response format for UI display Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Bulk import Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * fix: Remove conflicting inline script and fix bulk import functionality - Remove conflicting inline JavaScript that was preventing form submission - Fix indentation in setupBulkImportModal function - Ensure bulk import modal uses proper admin.js implementation - Restore proper form submission handling for bulk import This fixes the issue where bulk import appeared to do nothing. Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * fix: Integrate bulk import setup with main initialization - Add setupBulkImportModal() to main initialization sequence - Remove duplicate DOMContentLoaded listener - Ensure bulk import doesn't interfere with other tab functionality Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * fix: JavaScript formatting issues in bulk import modal - Fix multiline querySelector formatting - Fix multiline Error constructor formatting - Ensure prettier compliance for web linting Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * debug: Temporarily disable bulk import setup to test tabs Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * fix: Remove duplicate setupFormValidation call and delay bulk import setup - Remove duplicate setupFormValidation() call that could cause conflicts - Use setTimeout to delay bulk import modal setup after other initialization - Add better null safety to form element queries - This should fix tab switching issues Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * fix: Restore proper initialization sequence for tab functionality - Remove setTimeout delay for bulk import setup - Keep bulk import setup in main initialization but with error handling - Ensure tab navigation isn't affected by bulk import modal setup Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * fix: Correct HTML structure and restore tab navigation - Move bulk import modal to correct location after tools panel - Remove extra closing div that was breaking HTML structure - Ensure proper page-level modal placement - Restore tab navigation functionality for all tabs This fixes the broken Global Resources, Prompts, Gateways, Roots, and Metrics tabs. Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * feat: Add configurable bulk import settings Configuration additions: - MCPGATEWAY_BULK_IMPORT_MAX_TOOLS (default: 200) - MCPGATEWAY_BULK_IMPORT_RATE_LIMIT (default: 10) Implementation: - config.py: Add new settings with defaults - admin.py: Use configurable rate limit and batch size - .env.example: Document all bulk import environment variables - admin.html: Use dynamic max tools value in UI text - CLAUDE.md: Document configuration options for developers - docs: Update bulk import guide with configuration details This makes bulk import fully configurable for different deployment scenarios. Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> * Update docs Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> --------- Signed-off-by: Mihai Criveti <crivetimihai@gmail.com> Co-authored-by: Mihai Criveti <crivetimihai@gmail.com>
1 parent fa8e3ea commit a492834

File tree

7 files changed

+426
-32
lines changed

7 files changed

+426
-32
lines changed

‎.env.example‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,12 @@ MCPGATEWAY_ADMIN_API_ENABLED=true
131131
# Enable bulk import endpoint for tools (true/false)
132132
MCPGATEWAY_BULK_IMPORT_ENABLED=true
133133

134+
# Maximum number of tools allowed per bulk import request
135+
MCPGATEWAY_BULK_IMPORT_MAX_TOOLS=200
136+
137+
# Rate limiting for bulk import endpoint (requests per minute)
138+
MCPGATEWAY_BULK_IMPORT_RATE_LIMIT=10
139+
134140
#####################################
135141
# Header Passthrough Configuration
136142
#####################################

‎CLAUDE.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,11 @@ AUTH_REQUIRED=true
137137
MCPGATEWAY_UI_ENABLED=true
138138
MCPGATEWAY_ADMIN_API_ENABLED=true
139139

140+
# Bulk Import (Admin UI feature)
141+
MCPGATEWAY_BULK_IMPORT_ENABLED=true # Enable/disable bulk import endpoint
142+
MCPGATEWAY_BULK_IMPORT_MAX_TOOLS=200 # Maximum tools per import batch
143+
MCPGATEWAY_BULK_IMPORT_RATE_LIMIT=10 # Requests per minute limit
144+
140145
# Federation
141146
MCPGATEWAY_ENABLE_MDNS_DISCOVERY=true
142147
MCPGATEWAY_ENABLE_FEDERATION=true

‎docs/docs/manage/bulk-import.md‎

Lines changed: 61 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,34 +2,89 @@
22

33
The MCP Gateway provides a bulk import endpoint for efficiently loading multiple tools in a single request, perfect for migrations, environment setup, and team onboarding.
44

5-
!!! info "Feature Flag Required"
6-
This feature is controlled by the `MCPGATEWAY_BULK_IMPORT_ENABLED` environment variable.
7-
Default: `true` (enabled). Set to `false` to disable this endpoint.
5+
!!! info "Configuration Options"
6+
This feature is controlled by several environment variables:
7+
8+
- `MCPGATEWAY_BULK_IMPORT_ENABLED=true` - Enable/disable the endpoint (default: true)
9+
- `MCPGATEWAY_BULK_IMPORT_MAX_TOOLS=200` - Maximum tools per batch (default: 200)
10+
- `MCPGATEWAY_BULK_IMPORT_RATE_LIMIT=10` - Requests per minute limit (default: 10)
811

912
---
1013

1114
## 🚀 Overview
1215

13-
The `/admin/tools/import` endpoint allows you to register multiple tools at once, providing:
16+
The bulk import feature allows you to register multiple tools at once through both the Admin UI and API, providing:
1417

1518
- **Per-item validation** - One invalid tool won't fail the entire batch
1619
- **Detailed reporting** - Know exactly which tools succeeded or failed
1720
- **Rate limiting** - Protected against abuse (10 requests/minute)
1821
- **Batch size limits** - Maximum 200 tools per request
19-
- **Multiple input formats** - JSON payload or form data
22+
- **Multiple input formats** - JSON payload, form data, or file upload
23+
- **User-friendly UI** - Modal dialog with drag-and-drop file support
24+
25+
---
26+
27+
## 🎨 Admin UI Usage
28+
29+
### Accessing the Bulk Import Modal
30+
31+
1. **Navigate to Admin UI** - Open your gateway's admin interface at `http://localhost:4444/admin`
32+
2. **Go to Tools Tab** - Click on the "Tools" tab in the main navigation
33+
3. **Open Bulk Import** - Click the "+ Bulk Import Tools" button next to "Add New Tool"
34+
35+
### Using the Modal
36+
37+
The bulk import modal provides two ways to input tool data:
38+
39+
#### Option 1: JSON Textarea
40+
1. **Paste JSON directly** into the text area
41+
2. **Validate format** - The modal will check JSON syntax before submission
42+
3. **Click Import Tools** to process
43+
44+
#### Option 2: File Upload
45+
1. **Prepare a JSON file** with your tools array
46+
2. **Click "Choose File"** and select your `.json` file
47+
3. **Click Import Tools** to process
48+
49+
### UI Features
50+
51+
- **Real-time validation** - JSON syntax checking before submission
52+
- **Loading indicators** - Progress spinner during import
53+
- **Detailed results** - Success/failure counts with error details
54+
- **Auto-refresh** - Page reloads automatically after successful import
55+
- **Modal controls** - Close with button, backdrop click, or ESC key
2056

2157
---
2258

2359
## 📡 API Endpoint
2460

25-
### Request
61+
### Request Methods
2662

63+
#### Method 1: JSON Body
2764
```http
2865
POST /admin/tools/import
2966
Authorization: Bearer <jwt_token>
3067
Content-Type: application/json
3168
```
3269

70+
#### Method 2: Form Data (JSON String)
71+
```http
72+
POST /admin/tools/import
73+
Authorization: Bearer <jwt_token>
74+
Content-Type: multipart/form-data
75+
76+
Form field: tools_json=<json_string>
77+
```
78+
79+
#### Method 3: File Upload
80+
```http
81+
POST /admin/tools/import
82+
Authorization: Bearer <jwt_token>
83+
Content-Type: multipart/form-data
84+
85+
Form field: tools_file=<uploaded_json_file>
86+
```
87+
3388
### Payload Structure
3489

3590
```json

‎mcpgateway/admin.py‎

Lines changed: 51 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1568,6 +1568,7 @@ async def admin_ui(
15681568
"root_path": root_path,
15691569
"max_name_length": max_name_length,
15701570
"gateway_tool_name_separator": settings.gateway_tool_name_separator,
1571+
"bulk_import_max_tools": settings.mcpgateway_bulk_import_max_tools,
15711572
},
15721573
)
15731574

@@ -4375,7 +4376,7 @@ async def admin_list_tags(
43754376

43764377
@admin_router.post("/tools/import/")
43774378
@admin_router.post("/tools/import")
4378-
@rate_limit(requests_per_minute=10)
4379+
@rate_limit(requests_per_minute=settings.mcpgateway_bulk_import_rate_limit)
43794380
async def admin_import_tools(
43804381
request: Request,
43814382
db: Session = Depends(get_db),
@@ -4418,19 +4419,33 @@ async def admin_import_tools(
44184419
except Exception as ex:
44194420
LOGGER.exception("Invalid form body")
44204421
return JSONResponse({"success": False, "message": f"Invalid form data: {ex}"}, status_code=422)
4421-
raw = form.get("tools_json") or form.get("json") or form.get("payload")
4422-
if not raw:
4423-
return JSONResponse({"success": False, "message": "Missing tools_json/json/payload form field."}, status_code=422)
4424-
try:
4425-
payload = json.loads(raw)
4426-
except Exception as ex:
4427-
LOGGER.exception("Invalid JSON in form field")
4428-
return JSONResponse({"success": False, "message": f"Invalid JSON: {ex}"}, status_code=422)
4422+
# Check for file upload first
4423+
if "tools_file" in form:
4424+
file = form["tools_file"]
4425+
if hasattr(file, "file"):
4426+
content = await file.read()
4427+
try:
4428+
payload = json.loads(content.decode("utf-8"))
4429+
except (json.JSONDecodeError, UnicodeDecodeError) as ex:
4430+
LOGGER.exception("Invalid JSON file")
4431+
return JSONResponse({"success": False, "message": f"Invalid JSON file: {ex}"}, status_code=422)
4432+
else:
4433+
return JSONResponse({"success": False, "message": "Invalid file upload"}, status_code=422)
4434+
else:
4435+
# Check for JSON in form fields
4436+
raw = form.get("tools") or form.get("tools_json") or form.get("json") or form.get("payload")
4437+
if not raw:
4438+
return JSONResponse({"success": False, "message": "Missing tools/tools_json/json/payload form field."}, status_code=422)
4439+
try:
4440+
payload = json.loads(raw)
4441+
except Exception as ex:
4442+
LOGGER.exception("Invalid JSON in form field")
4443+
return JSONResponse({"success": False, "message": f"Invalid JSON: {ex}"}, status_code=422)
44294444

44304445
if not isinstance(payload, list):
44314446
return JSONResponse({"success": False, "message": "Payload must be a JSON array of tools."}, status_code=422)
44324447

4433-
max_batch = 200
4448+
max_batch = settings.mcpgateway_bulk_import_max_tools
44344449
if len(payload) > max_batch:
44354450
return JSONResponse({"success": False, "message": f"Too many tools ({len(payload)}). Max {max_batch}."}, status_code=413)
44364451

@@ -4463,15 +4478,33 @@ async def admin_import_tools(
44634478
LOGGER.exception("Unexpected error importing tool %r at index %d", name, i)
44644479
errors.append({"index": i, "name": name, "error": {"message": str(ex)}})
44654480

4466-
return JSONResponse(
4467-
{
4468-
"success": len(errors) == 0,
4469-
"created_count": len(created),
4470-
"failed_count": len(errors),
4471-
"created": created,
4472-
"errors": errors,
4481+
# Format response to match both frontend and test expectations
4482+
response_data = {
4483+
"success": len(errors) == 0,
4484+
# New format for frontend
4485+
"imported": len(created),
4486+
"failed": len(errors),
4487+
"total": len(payload),
4488+
# Original format for tests
4489+
"created_count": len(created),
4490+
"failed_count": len(errors),
4491+
"created": created,
4492+
"errors": errors,
4493+
# Detailed format for frontend
4494+
"details": {
4495+
"success": [item["name"] for item in created if item.get("name")],
4496+
"failed": [{"name": item["name"], "error": item["error"].get("message", str(item["error"]))} for item in errors],
44734497
},
4474-
status_code=200,
4498+
}
4499+
4500+
if len(errors) == 0:
4501+
response_data["message"] = f"Successfully imported all {len(created)} tools"
4502+
else:
4503+
response_data["message"] = f"Imported {len(created)} of {len(payload)} tools. {len(errors)} failed."
4504+
4505+
return JSONResponse(
4506+
response_data,
4507+
status_code=200, # Always return 200, success field indicates if all succeeded
44754508
)
44764509

44774510
except HTTPException:

‎mcpgateway/config.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,8 @@ class Settings(BaseSettings):
150150
mcpgateway_ui_enabled: bool = False
151151
mcpgateway_admin_api_enabled: bool = False
152152
mcpgateway_bulk_import_enabled: bool = True
153+
mcpgateway_bulk_import_max_tools: int = 200
154+
mcpgateway_bulk_import_rate_limit: int = 10
153155

154156
# Security
155157
skip_ssl_verify: bool = False

0 commit comments

Comments
 (0)