Add per-serverID log files for easier backend troubleshooting

[Copilot]

Per-ServerID Logging Implementation ✅

This PR implements per-serverID log files to make it easier to troubleshoot individual MCP servers.

Completed Tasks

• Explore the repository and understand current logging infrastructure
• Design per-serverID logging architecture
  • Add new ServerFileLogger that manages per-serverID log files
  • Create log files named {serverID}.log in the log directory
  • Add helper functions to log with serverID context
• Implement per-serverID file logger
  • Create ServerFileLogger struct with map of serverID -> file logger
  • Add LogWithServer functions that route to per-serverID files
  • Handle concurrent access with proper locking
• Update existing log calls to use per-serverID logging
  • Update backend-related log calls with serverID context
  • Updated launcher package to use per-serverID logging
  • Updated connection.go stderr logging
  • Updated unified server tool registration logging
  • Keep gateway-wide logs in mcp-gateway.log
• Add tests for per-serverID logging
  • Test log file creation per serverID
  • Test concurrent logging to different serverID files
  • Test all log levels
  • Test fallback mode
  • Test multiple initialization
  • Test that unified log is preserved ✨
• Documentation
  • Update logger README with per-serverID logging details
  • Update main README to highlight the feature
  • Update AGENTS.md with logging details
  • Document log directory structure
• Verify changes
  • Build successful
  • All unit tests passing
  • Linter passed ✅ (fixed gofmt issue)
  • Code review completed - addressed feedback
  • Security check (CodeQL) - No vulnerabilities found

Implementation Summary

Key Feature - Dual Logging:
✅ Each per-serverID log function writes to BOTH:

1. Server-specific log file (e.g., github.log) - isolated messages
2. Unified mcp-gateway.log - all messages with [serverID] prefix

This ensures single-view files are preserved while adding per-serverID isolation.

Log Directory Structure:

/tmp/gh-aw/mcp-logs/
├── mcp-gateway.log    # ALL messages (single-view) ✅
├── github.log         # Only GitHub server logs
├── slack.log          # Only Slack server logs
├── notion.log         # Only Notion server logs
├── gateway.md         # Markdown format
└── rpc-messages.jsonl # RPC messages

API Usage:

// Writes to both {serverID}.log AND mcp-gateway.log
logger.LogInfoWithServer("github", "backend", "Server started")
logger.LogWarnWithServer("slack", "backend", "Connection timeout")

Testing:

• 8 comprehensive test cases including test for unified log preservation
• All existing tests continue to pass
• Verified single-view and per-serverID logs work correctly
• CodeQL security check passed with no vulnerabilities
• All linting checks pass ✅

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[lpcox]

@copilot make sure that single-view files are preserved as well as the individual per-serverid log files

[Copilot]

@copilot make sure that single-view files are preserved as well as the individual per-serverid log files

Confirmed! Single-view files are preserved. Each LogXWithServer function writes to both:

1. Per-serverID log file (e.g., github.log) - isolated server messages
2. Unified mcp-gateway.log - ALL messages with [serverID] prefix

Added test in commit 7b18f89 that verifies:

• Per-serverID logs contain only their server's messages
• Unified mcp-gateway.log contains messages from all servers
• All messages in unified log include serverID prefix for identification

The implementation maintains backward compatibility - the unified log continues to work exactly as before, with the addition of per-serverID prefixes and optional per-server isolation files.

[lpcox]

@copilot please fix this linting errror

make lint
shell: /usr/bin/bash -e {0}
env:
GOTOOLCHAIN: local
Running linters...
Running gofmt check...
The following files are not formatted:
internal/logger/server_file_logger_test.go
make: *** [Makefile:26: lint] Error 1

[Copilot]

@copilot please fix this linting errror

make lint
...

Fixed in commit 6602744. Removed trailing blank line from server_file_logger_test.go to pass gofmt formatting check.

[Copilot]

Pull request overview

This PR implements per-serverID log files to improve troubleshooting of individual MCP backend servers. The implementation creates separate log files for each serverID (e.g., github.log, slack.log) while preserving a unified mcp-gateway.log with all messages. This dual logging approach maintains backward compatibility while adding the ability to view server-specific logs in isolation.

Changes:

• Added ServerFileLogger implementation with thread-safe per-serverID file management and automatic fallback to unified logging
• Updated backend server logging calls in launcher, unified server, and connection packages to use new LogInfoWithServer, LogWarnWithServer, LogErrorWithServer, and LogDebugWithServer functions
• Enhanced documentation in README files to describe the new logging structure and usage patterns

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
internal/logger/server_file_logger.go	Core implementation of ServerFileLogger with per-serverID log file management, concurrent access handling, and dual logging to both per-server and unified logs
internal/logger/server_file_logger_test.go	Comprehensive test suite covering initialization, concurrent access, fallback mode, all log levels, multiple initialization, and unified log preservation
internal/cmd/root.go	Integration of per-serverID logger initialization during gateway startup with proper cleanup on shutdown
internal/launcher/launcher.go	Updated backend server launch and session management logs to use per-serverID logging functions
internal/launcher/log_helpers.go	Updated all launcher helper logging calls (security warnings, launch start/success/error, timeout errors) to use per-serverID logging
internal/mcp/connection.go	Updated stderr logging to use per-serverID logging with simplified format
internal/server/unified.go	Updated tool registration success and failure logs to use per-serverID logging
internal/logger/README.md	Added comprehensive documentation for per-serverID logging feature including usage examples, log file structure, and benefits
README.md	Updated main documentation to highlight per-serverID logs feature with log directory structure examples
AGENTS.md	Enhanced logging documentation with per-serverID logging details and usage patterns

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Potential race condition: After calling logger.Println on line 115 without holding a lock, another goroutine could call Close() which acquires a write lock, closes all files, and clears the maps. This means logger.Println could write to a file handle that's being closed or has been closed. Consider holding the RLock from before the Println call through the Sync call, or use a different synchronization approach to ensure the logger and file aren't closed while being used.

Suggested change

	logger.Println(logLine)
	// Flush to disk immediately
	sfl.mu.RLock()
	if file, exists := sfl.files[serverID]; exists {
	if err := file.Sync(); err != nil {
	log.Printf("WARNING: Failed to sync log file for server %s: %v", serverID, err)
	}
	}
	sfl.mu.RUnlock()
	sfl.mu.RLock()
	defer sfl.mu.RUnlock()
	logger.Println(logLine)
	// Flush to disk immediately
	if file, exists := sfl.files[serverID]; exists {
	if err := file.Sync(); err != nil {
	log.Printf("WARNING: Failed to sync log file for server %s: %v", serverID, err)
	}
	}