Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 15, 2025

After setup, users had no way to verify configuration worked or troubleshoot failures. Documentation lacked context on command usage patterns and safe-outputs security model.

Quick Start verification

  • Add gh aw status verification step after token configuration
  • Show expected output to confirm successful setup
  • Include troubleshooting for missing workflows, token errors, and permission issues

CLI command workflows

  • Add "Common Workflows for Beginners" section showing command patterns:
    • Post-creation: compile → run → logs
    • Debugging: status → logs → audit
    • Recovery: secrets bootstrap, enable/disable, fix
  • Document run-id location in GitHub Actions UI

Safe outputs security model

  • Clarify safe-outputs as security feature, not convenience
  • Explain AI executes with read-only permissions; separate job handles writes
  • Document security benefits: least privilege, prompt injection defense, auditability, controlled blast radius

Example verification flow:

# After adding COPILOT_GITHUB_TOKEN
gh aw status

# Expected output confirms:
# ✓ Workflow compiled and enabled
# ✓ Schedule configured
# ✓ Ready to execute
Original prompt

This section details on the original issue you should resolve

<issue_title>[plan] Add verification steps and troubleshooting context</issue_title>
<issue_description>## Objective

Add verification commands after setup steps so users can confirm their configuration is working correctly, and provide troubleshooting context for common issues.

Problem

After adding the COPILOT_GITHUB_TOKEN secret, the guide says "Your repository is now configured!" but doesn't tell users:

  • How to verify the setup worked
  • What to do if something went wrong
  • How to know if the token is valid

Additionally, the CLI commands page lacks context about when to use each command.

Approach

  1. Add verification step after token setup in Quick Start:
# Verify your setup
gh aw status

# Expected output:
# ✓ gh-aw extension: v0.x.x
# ✓ GitHub CLI: authenticated
# ✓ Repository: githubnext/gh-aw
# ✓ Copilot token: configured
  1. Add "Common Workflows" section to CLI Commands page:
### Common Workflows for Beginners

**After creating a new workflow:**
```bash
gh aw compile my-workflow.md  # Check for syntax errors
gh aw run my-workflow         # Test it manually (if workflow_dispatch enabled)
gh aw logs my-workflow        # See what happened

When something goes wrong:

gh aw status                  # Check configuration
gh aw logs my-workflow -f     # Watch live logs
gh aw audit (run-id)          # Analyze specific run (get run-id from Actions tab)
  1. Improve Safe Outputs explanation with security context:
**Safe Outputs are a security feature.** Your AI agent runs with minimal permissions (read-only). When it wants to make changes (like creating an issue), it "requests" that action. A separate, permission-controlled job reviews and executes these requests. This prevents the AI from accidentally or maliciously making unauthorized changes.

Files to Modify

  • docs/src/content/docs/setup/quick-start.mdx - Add verification commands after token setup
  • docs/src/content/docs/setup/cli.mdx - Add "Common Workflows" section with context
  • docs/src/content/docs/index.mdx or relevant concept page - Improve Safe Outputs explanation

Acceptance Criteria

  • Verification command added after token configuration step
  • Expected output shown so users know what success looks like
  • CLI Commands page includes usage context for beginners
  • Run-id explained (where to find it)
  • Safe Outputs explanation clarifies security model
  • Users can self-diagnose basic setup issues

Reference

From noob test report: Critical Issue #3 (Token Creation), Confusing Areas #4 (Safe Outputs), #5 (CLI Commands)
Related to #6505

AI generated by Plan Command for discussion #6502

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: mnkiefer <8320933+mnkiefer@users.noreply.github.com>
Copilot AI changed the title [WIP] Add verification steps and troubleshooting context Add verification steps and troubleshooting guidance to documentation Dec 15, 2025
Copilot AI requested a review from mnkiefer December 15, 2025 19:22
@pelikhan pelikhan marked this pull request as ready for review December 15, 2025 19:28
@pelikhan pelikhan merged commit 778bc0d into main Dec 15, 2025
4 checks passed
@pelikhan pelikhan deleted the copilot/add-verification-steps branch December 15, 2025 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[plan] Add verification steps and troubleshooting context

3 participants