Skip to content

Add agent version capture to agentic workflows#1112

Merged
pelikhan merged 3 commits intomainfrom
copilot/fix-938ca616-dfee-47dd-9a8c-499d74cbbb2f
Sep 30, 2025
Merged

Add agent version capture to agentic workflows#1112
pelikhan merged 3 commits intomainfrom
copilot/fix-938ca616-dfee-47dd-9a8c-499d74cbbb2f

Conversation

Copy link
Contributor

Copilot AI commented Sep 30, 2025

Overview

This PR adds support for capturing and recording the actual version of the agent binary (e.g., claude, copilot, codex) that executes agentic workflows. The version information is now automatically captured during workflow execution and included in the aw-info.json metadata file as an agent_version field.

Problem

Previously, agentic workflows only recorded the configured version from the frontmatter (e.g., engine.version: beta), but not the actual installed version of the agent binary. This made it difficult to:

  • Debug issues related to specific agent versions
  • Track which agent version was used for a particular workflow run
  • Correlate workflow behavior with agent releases

Solution

Interface Enhancement

Added a new GetVersionCommand() method to the CodingAgentEngine interface that returns the command to retrieve the agent's version:

// GetVersionCommand returns the command to get the version of the agent (e.g., "copilot --version")
// Returns empty string if the engine does not support version reporting
GetVersionCommand() string

Engine Implementations

Each engine now implements the version command appropriate for its CLI:

  • Claude: claude --version
  • Copilot: copilot --version
  • Codex: codex --version
  • Custom: Returns empty string (no version command)

Compiler Updates

The compiler now generates a version capture step before creating aw-info.json:

- name: Capture agent version
  run: |
    VERSION_OUTPUT=$(claude --version 2>&1 || echo "unknown")
    echo "AGENT_VERSION=$VERSION_OUTPUT" >> $GITHUB_ENV
    echo "Agent version: $VERSION_OUTPUT"

The captured version is then included in the metadata:

const awInfo = {
  engine_id: "claude",
  engine_name: "Claude Code",
  model: "",
  version: "",
  agent_version: process.env.AGENT_VERSION || "",
  workflow_name: "Example",
  ...
};

Benefits

  1. Better Debugging: Workflow runs now include the exact agent version, making it easier to identify version-specific issues
  2. Audit Trail: Complete record of which agent version executed each workflow run
  3. Version Correlation: Ability to track behavior changes across agent releases
  4. Graceful Degradation: Falls back to "unknown" if version command fails, ensuring workflows continue to run

Testing

  • Added comprehensive unit tests for all engine implementations
  • Added compiler tests to verify version capture step generation
  • All 44 workflow files successfully recompiled with new functionality
  • All existing tests continue to pass

Example Output

For a workflow using the Claude engine, the aw-info.json now includes:

{
  "engine_id": "claude",
  "engine_name": "Claude Code",
  "model": "",
  "version": "beta",
  "agent_version": "claude 1.2.3",
  "workflow_name": "CI Failure Doctor",
  ...
}

Related

Addresses the requirement to capture and record the actual agent version in addition to the configured engine version.

Original prompt

Update the agentic engine type to support a function that returns the command to get the version of the agent (like "copilot --version").

In the compiler, if the function returns a command, generate a step that executes that command and include the output in the aw-info.json file as "agent-version" field

Update Claude, copilot, codex engines


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits September 30, 2025 15:29
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title Update the agentic engine type to support a function that returns the command to get the version of the agent (like "copilot --version"). In the compiler, if the function returns a command, generate a step that executes that command and include the ou... Add agent version capture to agentic workflows Sep 30, 2025
Copilot AI requested a review from pelikhan September 30, 2025 15:37
@pelikhan pelikhan marked this pull request as ready for review September 30, 2025 15:44
@pelikhan pelikhan merged commit 1d4c023 into main Sep 30, 2025
16 checks passed
@pelikhan pelikhan deleted the copilot/fix-938ca616-dfee-47dd-9a8c-499d74cbbb2f branch September 30, 2025 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants