Skip to content

Comments

Pre-validate workflow names in MCP logs tool#15221

Merged
pelikhan merged 5 commits intomainfrom
copilot/improve-error-messaging-logs
Feb 12, 2026
Merged

Pre-validate workflow names in MCP logs tool#15221
pelikhan merged 5 commits intomainfrom
copilot/improve-error-messaging-logs

Conversation

Copy link
Contributor

Copilot AI commented Feb 12, 2026

Summary: Improve error messaging for non-existent workflows in logs command ✅

Adds workflow name validation to the MCP server's logs tool to provide user-friendly error messages with suggestions when an invalid workflow name is provided, instead of generic CLI exit errors.

Changes

1. Added validateWorkflowName() function in pkg/cli/mcp_server.go

  • Validates workflow names before executing the logs command
  • Returns nil for empty workflow names (means "all workflows")
  • Tries to resolve workflow IDs using workflow.ResolveWorkflowName()
  • Checks against known workflows using getAgenticWorkflowNames()
  • Provides fuzzy-matched suggestions using suggestWorkflowNames()

2. Integrated validation in logs tool handler

  • Validates workflow name before building command arguments (line 765-776)
  • Returns MCP error with InvalidParams code on validation failure
  • Includes structured error data with workflow_name and error_type

3. Added comprehensive unit tests

  • TestMCPValidateWorkflowName tests 4 scenarios:
    • Empty workflow name (valid case)
    • Non-existent workflow (error case)
    • Error message includes suggestions
    • Fuzzy match suggestions for similar workflow names

Error Message Improvement

Before: Generic MCP error with exit status 1

McpError: MCP error -32603: calling "tools/call": failed to download workflow logs: exit status 1

After: Structured error with actionable suggestions

McpError: MCP error -32602 (InvalidParams): workflow 'nonexistent-workflow-xyz-12345' not found. 
Did you mean: test-workflow, test-dispatcher-workflow? 
Use the 'status' tool to see all available workflows 
Check for typos in the workflow name 
Use the workflow ID (e.g., 'test-claude') or GitHub Actions workflow name (e.g., 'Test Claude')

Error Data:
{
  "workflow_name": "nonexistent-workflow-xyz-12345",
  "error_type": "workflow_not_found"
}

Benefits

  • ✅ User-friendly error messages with actionable suggestions
  • ✅ Fuzzy matching suggests similar workflow names
  • ✅ Structured error data for programmatic handling
  • ✅ Early validation prevents unnecessary command execution
  • ✅ Consistent with existing patterns in the logs command
  • ✅ Reuses existing validation logic (no code duplication)
  • ✅ References MCP status tool for discovering available workflows

Testing & Validation

  • ✅ All unit tests pass (4 new test cases added)
  • ✅ No regressions in existing logs command tests
  • ✅ Linting passes with no issues
  • ✅ Code formatted correctly
  • ✅ Follows repository patterns and conventions
  • ✅ Security scan passes (no new issues introduced)

Files Changed

  • pkg/cli/mcp_server.go (+57 lines)
  • pkg/cli/mcp_server_workflow_validation_test.go (+57 lines)

Total: 114 lines added across 2 files

Next Steps

This pattern can be applied to the audit command as mentioned in the issue for consistent error messaging across all MCP tools.

Original prompt

This section details on the original issue you should resolve

<issue_title>[cli-tools-test] Improve error messaging for non-existent workflow in logs command</issue_title>
<issue_description>## Problem Description

When querying logs for a workflow that doesn't exist, the MCP server returns a generic MCP error instead of a user-friendly error message.

Command/Tool

  • Tool: logs
  • Command: agenticworkflows-logs with workflow_name: "nonexistent-workflow-xyz-12345"

Steps to Reproduce

  1. Use the agentic-workflows MCP server logs tool
  2. Provide a workflow name that doesn't exist (e.g., "nonexistent-workflow-xyz-12345")
  3. Observe the error response

Expected Behavior

One of:

  • Empty result set with summary indicating 0 runs found
  • Clear error message: "Workflow 'nonexistent-workflow-xyz-12345' not found. Available workflows: ..."
  • Helpful suggestion to check workflow name spelling or use list command

Actual Behavior

Generic MCP error:

McpError: MCP error -32603: calling "tools/call": failed to download workflow logs: exit status 1

Environment

  • Repository: github/gh-aw
  • Run ID: 21958431840
  • Date: 2026-02-12
  • Testing Context: Daily CLI Tools Exploratory Testing

Impact

  • Severity: Medium
  • Frequency: Every time an invalid workflow name is provided
  • Workaround: Users must manually verify workflow names using the list command first

Recommended Fix

Add validation layer in the logs command that:

  1. Pre-validates workflow name against known workflows
  2. Returns structured error with helpful context
  3. Suggests corrections (fuzzy matching) or lists available workflows

Example improved error:

{
  "error": "workflow_not_found",
  "message": "Workflow 'nonexistent-workflow-xyz-12345' not found",
  "suggestions": ["Did you mean: test-workflow, test-dispatcher-workflow?"],
  "help": "Run 'gh aw status' to see all available workflows"
}

Additional Context

This issue was discovered during comprehensive exploratory testing of the agentic-workflows MCP server. The same pattern affects the audit command when provided with invalid run IDs (see related issue).

AI generated by Daily CLI Tools Exploratory Tester

  • expires on Feb 19, 2026, 6:13 PM UTC

Comments on the Issue (you are @copilot in this section)

@pelikhan @copilot add a workflow name validation code in the MCP server to detect invalid name before launching the process. Reuse APIs from logs command (find out which one there is)

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 2 commits February 12, 2026 18:41
- Add validateWorkflowName() function in mcp_server.go
- Validate workflow name before executing logs command
- Return structured MCP error with suggestions on invalid workflow
- Add unit test TestMCPValidateWorkflowName
- Import sliceutil package for workflow name checking

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Improve error messaging for nonexistent workflow in logs command Pre-validate workflow names in MCP logs tool Feb 12, 2026
Copilot AI requested a review from pelikhan February 12, 2026 18:52
Change suggestion from "Run 'gh aw status'" to "Use the 'status' tool"
to reference the MCP status tool available in the server.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review February 12, 2026 19:08
Copilot AI review requested due to automatic review settings February 12, 2026 19:08
@pelikhan pelikhan merged commit e616c16 into main Feb 12, 2026
50 checks passed
@pelikhan pelikhan deleted the copilot/improve-error-messaging-logs branch February 12, 2026 19:08
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves error handling in the MCP server's logs tool by adding pre-validation for workflow names, providing user-friendly error messages with actionable suggestions instead of generic CLI exit errors when invalid workflow names are provided.

Changes:

  • Added validateWorkflowName() function to check workflow existence before command execution
  • Integrated validation into the logs tool handler with structured error responses
  • Added comprehensive unit tests covering valid and invalid workflow name scenarios

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
pkg/cli/mcp_server.go Implements workflow name validation logic and integrates it into the logs tool handler
pkg/cli/mcp_server_workflow_validation_test.go Adds unit tests for workflow name validation covering empty names, non-existent workflows, and error message content

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +175 to +176
mcpLog.Printf("Validating workflow name: %s", workflowName)

Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a debug-level log or removing this log statement. This validation is called for every logs tool invocation and will create noise in production logs.

Suggested change
mcpLog.Printf("Validating workflow name: %s", workflowName)

Copilot uses AI. Check for mistakes.
Comment on lines +178 to +180
resolvedName, err := workflow.ResolveWorkflowName(workflowName)
if err == nil {
mcpLog.Printf("Workflow name resolved successfully: %s -> %s", workflowName, resolvedName)
Copy link

Copilot AI Feb 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resolved workflow name is logged but not used. Either remove the log statement on line 180 or use the resolvedName for further validation if needed.

Suggested change
resolvedName, err := workflow.ResolveWorkflowName(workflowName)
if err == nil {
mcpLog.Printf("Workflow name resolved successfully: %s -> %s", workflowName, resolvedName)
if _, err := workflow.ResolveWorkflowName(workflowName); err == nil {
mcpLog.Printf("Workflow name resolved successfully: %s", workflowName)

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[cli-tools-test] Improve error messaging for non-existent workflow in logs command

2 participants