Behaviour of custom safe outputs `inputs` not matching docs

[dsyme]

I'm looking in to a bug in safe-outputs

Trying to understand if it's a bug in docs, implementation, or test,

I'm following instructions here: https://githubnext.github.io/gh-aw/guides/custom-safe-outputs/

But inputs.message is not being set. Equally I can't quite see who/what should be setting this. Wondering if docs are wrong.

Workflow is here: https://github.com/githubnext/gh-aw-test/blob/main/.github/workflows/test-claude-safe-jobs.md?plain=1

---
on: 
  workflow_dispatch:

permissions: read-all

engine: claude

safe-outputs:
  jobs:
    print:
      #name: "print the message"
      runs-on: ubuntu-latest
      inputs:
        message:
          description: "Message to print"
          required: true
          type: string
      steps:
        - name: print message
          env:
            MESSAGE: "${{ inputs.message }}"
          run: |
              if [ -z "$MESSAGE" ]; then
                echo "Error: message is empty"
                exit 1
              fi
              echo "print: $MESSAGE"
              echo "### Print Step Summary" >> "$GITHUB_STEP_SUMMARY"
              echo "$MESSAGE" >> "$GITHUB_STEP_SUMMARY"    
            
---
Summarize and use print the message using the `print` tool.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.

Lock file here: https://github.com/githubnext/gh-aw-test/blob/main/.github/workflows/test-claude-safe-jobs.lock.yml?plain=1

Looking at other examples uses of inputs: in safe outputs in gh-aw files, they seem to process like this:
const outputContent = process.env.GH_AW_AGENT_OUTPUT;

....
Ian't find an example that goes via inputs like the docs say

I'm asking what the spec is.

Question is whether it is it right to expect this to work, as it says in the docs:
env:
MESSAGE: "${{ inputs.message }}"

This seems unlikely. Inputs is reserved for workflow inputs as far as I can tell. I think the docs have been hallucinated here.

AFAICS all that is available is GH_AW_AGENT_OUTPUT env var and the use code must process that.

I won't fix it just yet, will write it up as an issue for your guidance on how the safe output job gets the agent output.

[dsyme]

Bug: Documentation Incorrectly Shows ${{ inputs.* }} Usage in Custom Safe-Output Jobs

Summary

The Custom Safe Outputs documentation incorrectly suggests that custom safe-output jobs can access agent-provided data via ${{ inputs.message }} or similar expressions. This is misleading - safe-output jobs must instead read from the GH_AW_AGENT_OUTPUT environment variable and parse the JSON file.

Incorrect Documentation

Location: docs/src/content/docs/guides/custom-safe-outputs.md

Issue 1: Environment Variables Table (lines 272-279)

| Variable | Description |
|----------|-------------|
| `GH_AW_AGENT_OUTPUT` | Path to JSON file with agent output |
| `GH_AW_SAFE_OUTPUTS_STAGED` | Set to `"true"` in staged mode |
| `${{ inputs.NAME }}` | Each input as a variable |  ❌ WRONG

Access inputs using `${{ inputs.name }}` in steps or via `process.env` in JavaScript.  ❌ WRONG

Issue 2: Complete Example (lines 282-310)

safe-outputs:
  jobs:
    webhook-notify:
      description: "Send a notification to a webhook URL"
      runs-on: ubuntu-latest
      output: "Notification sent!"
      inputs:
        title:
          description: "Notification title"
          required: true
          type: string
        body:
          description: "Notification body"
          required: true
          type: string
      steps:
        - name: Send webhook
          env:
            WEBHOOK_URL: "${{ secrets.WEBHOOK_URL }}"
            TITLE: "${{ inputs.title }}"           ❌ WRONG
            BODY: "${{ inputs.body }}"             ❌ WRONG
          run: |
            PAYLOAD=$(jq -n --arg title "$TITLE" --arg body "$BODY" \
              '{title: $title, body: $body}')
            curl -X POST "$WEBHOOK_URL" \
              -H "Content-Type: application/json" \
              -d "$PAYLOAD"

Issue 3: Quick Start Example (lines 30-45)

The Quick Start example has the same issue:

- name: Send Slack message
  env:
    SLACK_WEBHOOK: "${{ secrets.SLACK_WEBHOOK }}"
    MESSAGE: "${{ inputs.message }}"                 ❌ WRONG

Correct Implementation

Safe-output jobs must:

1. Read the GH_AW_AGENT_OUTPUT environment variable (path to JSON file)
2. Parse the JSON file which has structure: {"items": [...]}
3. Filter items by type field (job name with dashes → underscores)
4. Extract fields from each matching item

Correct Example (Bash)

steps:
  - name: print message
    run: |
      if [ -f "$GH_AW_AGENT_OUTPUT" ]; then
        MESSAGE=$(cat "$GH_AW_AGENT_OUTPUT" | jq -r '.items[] | select(.type == "print") | .message')
        echo "print: $MESSAGE"
      else
        echo "No agent output found"
      fi

Correct Example (JavaScript)

steps:
  - name: Process agent output
    uses: actions/github-script@v8
    with:
      script: |
        const fs = require('fs');
        const outputFile = process.env.GH_AW_AGENT_OUTPUT;
        
        if (!outputFile) {
          core.info('No GH_AW_AGENT_OUTPUT environment variable found');
          return;
        }
        
        const fileContent = fs.readFileSync(outputFile, 'utf8');
        const agentOutput = JSON.parse(fileContent);
        
        const items = agentOutput.items.filter(item => item.type === 'webhook_notify');
        
        for (const item of items) {
          const title = item.title;
          const body = item.body;
          // Process each item...
        }

Evidence

Working examples in the codebase that use the correct pattern:

1. pkg/cli/workflows/test-copilot-safe-jobs.md:

   MESSAGE=$(cat "$GH_AW_AGENT_OUTPUT" | jq -r '.items[] | select(.type == "print") | .message')

2. pkg/cli/workflows/test-claude-safe-jobs.md:

   MESSAGE=$(cat "$GH_AW_AGENT_OUTPUT" | jq -r '.items[] | select(.type == "print") | .message')

3. pkg/cli/templates/create-shared-agentic-workflow.md:

   const outputContent = process.env.GH_AW_AGENT_OUTPUT;

4. pkg/cli/workflows/shared/mcp/slack.md:

   const outputContent = process.env.GH_AW_AGENT_OUTPUT;

What inputs: Actually Does

The inputs: field defines the MCP tool schema - it tells the AI agent what parameters the tool accepts. It's used for:

• Tool discovery by the AI agent
• Validation of agent output structure
• Documentation of the tool's interface

It does NOT:

• Create GitHub Actions workflow inputs
• Pass data directly to job steps
• Work like ${{ inputs.* }} in workflow_dispatch inputs

Related Files to Fix

1. docs/src/content/docs/guides/custom-safe-outputs.md - Fix all examples and documentation
2. Consider adding a clear explanation of inputs: vs GH_AW_AGENT_OUTPUT

User Impact

Users following the documentation will create non-working workflows because:

• ${{ inputs.message }} will be empty/undefined
• Jobs will fail or skip operations
• No clear error messages explain the issue

Reproduction

See test workflow: https://github.com/githubnext/gh-aw-test/blob/main/.github/workflows/test-claude-safe-jobs.md

The workflow attempts to use ${{ inputs.message }} but it doesn't work. The correct implementation requires reading $GH_AW_AGENT_OUTPUT.

Suggested Fix

1. Update all examples in docs/src/content/docs/guides/custom-safe-outputs.md to show the correct GH_AW_AGENT_OUTPUT pattern
2. Add a clear explanation of what inputs: means (MCP tool schema, not workflow inputs)
3. Add a "Common Mistakes" section warning against ${{ inputs.* }} usage
4. Update the environment variables table to remove the incorrect ${{ inputs.NAME }} row
5. Consider adding validation/linting to detect this pattern and warn users

[dsyme]

@copilot go ahead and do the suggested fix