diff --git a/docs/astro.config.mjs b/docs/astro.config.mjs index 158bb131f0..07270afbd4 100644 --- a/docs/astro.config.mjs +++ b/docs/astro.config.mjs @@ -192,11 +192,8 @@ export default defineConfig({ { label: 'Campaigns', items: [ - { label: 'About campaigns', link: '/guides/campaigns/' }, + { label: 'About Campaigns', link: '/guides/campaigns/' }, { label: 'Getting Started', link: '/guides/campaigns/getting-started/' }, - { label: 'Campaign Specs', link: '/guides/campaigns/specs/' }, - { label: 'Campaign Lifecycle', link: '/guides/campaigns/lifecycle/' }, - { label: 'CLI Commands', link: '/guides/campaigns/cli-commands/' }, ], }, { diff --git a/docs/campaign-workers.md b/docs/campaign-workers.md deleted file mode 100644 index e086a2c24e..0000000000 --- a/docs/campaign-workers.md +++ /dev/null @@ -1,615 +0,0 @@ -# Campaign Workers - -Campaign workers are first-class workflows designed to be orchestrated by campaign orchestrators. This document describes the worker pattern, input contract, idempotency requirements, and the bootstrap + worker metadata system. - -## Overview - -Campaign workers follow these principles: - -1. **Dispatch-only**: Workers are triggered via `workflow_dispatch` by orchestrators -2. **Standardized contract**: All workers accept `campaign_id` and `payload` inputs -3. **Idempotent**: Workers use deterministic keys to avoid duplicate work -4. **Orchestration-agnostic**: Workers don't encode orchestration policy -5. **Discoverable**: Workers produce outputs with guaranteed labeling contracts - -## Bootstrap + Planning Model - -When a campaign starts with zero discovered work items (discovery = 0), the orchestrator needs a way to create initial work. The bootstrap configuration provides three strategies: - -### 1. Seeder Worker Mode - -Dispatch a specialized worker to discover and create initial work items: - -```yaml -bootstrap: - mode: seeder-worker - seeder-worker: - workflow-id: security-scanner - payload: - scan-type: full - max-findings: 100 - max-items: 50 -``` - -**Flow**: -1. Orchestrator detects discovery = 0 -2. Orchestrator dispatches the seeder worker with configured payload -3. Seeder worker scans for work and creates issues/PRs with tracker labels -4. Next orchestrator run discovers the seeder's outputs - -### 2. Project Todos Mode - -Read work items from the Project board's "Todo" column: - -```yaml -bootstrap: - mode: project-todos - project-todos: - status-field: Status - todo-value: Backlog - max-items: 10 - require-fields: - - Priority - - Assignee -``` - -**Flow**: -1. Orchestrator detects discovery = 0 -2. Orchestrator queries Project board for items with Status = "Backlog" -3. Orchestrator uses worker metadata to select appropriate worker for each item -4. Orchestrator dispatches workers with payloads built from Project field values - -### 3. Manual Mode - -Wait for manual work item creation: - -```yaml -bootstrap: - mode: manual -``` - -**Flow**: -1. Orchestrator detects discovery = 0 -2. Orchestrator reports waiting for manual work item creation -3. Users manually create issues/PRs with proper tracker labels -4. Next orchestrator run discovers the manual items - -## Worker Metadata - -Worker metadata enables deterministic worker selection and ensures worker outputs are discoverable. Define worker metadata in your campaign spec: - -```yaml -workers: - - id: security-fixer - name: Security Fix Worker - description: Fixes security vulnerabilities - capabilities: - - fix-security-alerts - - create-pull-requests - payload-schema: - repository: - type: string - description: Target repository in owner/repo format - required: true - example: owner/repo - alert_id: - type: string - description: Security alert identifier - required: true - example: alert-123 - severity: - type: string - description: Alert severity level - required: false - example: high - output-labeling: - labels: - - security - - automated - key-in-title: true - key-format: "campaign-{campaign_id}-{repository}-{alert_id}" - metadata-fields: - - Campaign Id - - Worker Workflow - - Alert ID - - Severity - idempotency-strategy: pr-title-based - priority: 10 -``` - -### Worker Metadata Fields - -- **id**: Workflow identifier (basename without .md) -- **name**: Human-readable worker name -- **description**: What the worker does -- **capabilities**: List of work types this worker can handle -- **payload-schema**: Expected payload structure with types and descriptions -- **output-labeling**: Guaranteed labeling contract for worker outputs -- **idempotency-strategy**: How the worker ensures idempotent execution -- **priority**: Worker selection priority (higher = preferred) - -### Deterministic Worker Selection - -When the orchestrator needs to dispatch a worker (e.g., during bootstrap from Project todos): - -1. **Match capabilities**: Find workers whose capabilities match the work item type -2. **Validate payload**: Check if worker's payload schema can be satisfied from available data -3. **Select by priority**: If multiple workers match, select the one with highest priority -4. **Build payload**: Construct payload according to worker's payload schema -5. **Dispatch**: Call worker with campaign_id and constructed payload - -### Output Labeling Contract - -The `output-labeling` section guarantees how worker outputs are labeled and formatted: - -- **labels**: Labels the worker applies to created items (in addition to the campaign's tracker-label) -- **key-in-title**: Whether worker includes a deterministic key in item titles -- **key-format**: Format of the key when `key-in-title` is true -- **metadata-fields**: Project fields the worker populates - -Workers automatically apply the campaign's tracker label (defined at the campaign level) to all created items, ensuring: -- **Discoverable**: Can be found via tracker label searches -- **Attributable**: Can be traced back to the campaign and worker -- **Idempotent**: Can be checked for duplicates via deterministic keys - -## Why Dispatch-Only? - -Making workers dispatch-only (no schedule/push/pull_request triggers) provides several benefits: - -- **Unambiguous ownership**: Workers are clearly orchestrated, not autonomous -- **Prevents duplicate execution**: Avoids conflicts between original triggers and orchestrator -- **Explicit orchestration**: Orchestrator controls when and how workers run -- **Clear responsibility**: Sequential vs parallel execution is orchestrator's concern - -## Input Contract - -All campaign workers MUST accept these inputs: - -```yaml -on: - workflow_dispatch: - inputs: - campaign_id: - description: 'Campaign identifier' - required: true - type: string - payload: - description: 'JSON payload with work item details' - required: true - type: string -``` - -### Campaign ID - -The `campaign_id` identifies the campaign orchestrating this worker. Use it to: - -- Label created items: `z_campaign_${campaign_id}` -- Generate deterministic keys: `campaign-${campaign_id}-${work_item_id}` -- Track work in repo-memory: `memory/campaigns/${campaign_id}/` - -### Payload - -The `payload` is a JSON string containing work-specific data. Parse it to extract: - -- `repository`: Target repository (owner/repo format) -- `work_item_id`: Unique identifier for this work item -- `target_ref`: Target branch/ref (e.g., "main") -- Additional context specific to the worker - -Example payload: -```json -{ - "repository": "owner/repo", - "work_item_id": "alert-123", - "target_ref": "main", - "alert_type": "sql-injection", - "severity": "high", - "file_path": "src/database/query.go", - "line_number": 42 -} -``` - -## Idempotency Requirements - -Workers MUST implement idempotency to prevent duplicate work over repeated orchestrator runs. - -### Deterministic Work Item Keys - -Compute a stable key for each work item: - -``` -campaign-{campaign_id}-{repository}-{work_item_id} -``` - -Example: `campaign-security-q1-2025-myorg-myrepo-alert-123` - -Use this key in: -- Branch names: `fix/campaign-security-q1-2025-myorg-myrepo-alert-123` -- PR titles: `[campaign-security-q1-2025-myorg-myrepo-alert-123] Fix SQL injection` -- Issue titles: `[alert-123] High severity: SQL injection vulnerability` - -### Check Before Create - -Before creating any GitHub resource: - -1. **Search for existing items** with the deterministic key -2. **Filter by campaign tracker label**: `z_campaign_${campaign_id}` -3. **If found**: Skip or update existing item -4. **If not found**: Proceed with creation - -Example: -```javascript -const workKey = `campaign-${campaignId}-${repository}-${workItemId}`; -const searchQuery = `repo:${repository} is:pr is:open "${workKey}" in:title`; - -const existingPRs = await github.search.issuesAndPullRequests({ - q: searchQuery -}); - -if (existingPRs.total_count > 0) { - console.log(`PR already exists: ${existingPRs.items[0].html_url}`); - return; // Skip creation -} -``` - -### Label All Created Items - -Apply the campaign tracker label to all created items: - -- Label format: `z_campaign_${campaign_id}` -- Prevents interference from other workflows -- Enables discovery by orchestrator - -## Worker Template - -```yaml ---- -name: My Campaign Worker -description: Worker workflow for campaign orchestration - -on: - workflow_dispatch: - inputs: - campaign_id: - description: 'Campaign identifier' - required: true - type: string - payload: - description: 'JSON payload with work item details' - required: true - type: string - -tracker-id: my-campaign-worker - -tools: - github: - toolsets: [default] - -safe-outputs: - create-pull-request: - max: 1 - add-comment: - max: 2 ---- - -# My Campaign Worker - -You are a campaign worker that processes work items. - -## Step 1: Parse Input - -Parse the workflow_dispatch inputs: - -```javascript -const campaignId = context.payload.inputs.campaign_id; -const payload = JSON.parse(context.payload.inputs.payload); -``` - -Extract work item details from payload: -- `repository`: Target repository -- `work_item_id`: Unique identifier -- Additional context fields - -## Step 2: Check for Existing Work - -Generate deterministic key: -```javascript -const workKey = `campaign-${campaignId}-${payload.repository}-${payload.work_item_id}`; -``` - -Search for existing PR/issue with this key in title. - -If found: -- Log that work already exists -- Optionally add a comment with status update -- Exit successfully - -## Step 3: Perform Work - -If no existing work found: -1. Create branch with deterministic name -2. Make required changes -3. Create PR with deterministic title -4. Apply labels: `z_campaign_${campaignId}`, [additional labels] - -## Step 4: Report Status - -Report completion: -- Link to created/updated PR or issue -- Whether work was skipped or completed -- Any errors or blockers encountered -``` - -## Idempotency Patterns - -### Pattern 1: Branch-based Idempotency - -```yaml -# In worker prompt -Branch naming pattern: `fix/campaign-${campaignId}-${repo}-${workItemId}` - -Before creating: -1. Check if branch exists in target repo -2. If exists: Checkout and update -3. If not: Create new branch -``` - -### Pattern 2: PR Title-based Idempotency - -```yaml -# In worker prompt -PR title pattern: `[${workKey}] ${description}` - -Before creating PR: -1. Search for PRs with `${workKey}` in title -2. Filter by `z_campaign_${campaignId}` label -3. If found: Update with comment or skip -4. If not: Create new PR -``` - -### Pattern 3: Cursor-based Tracking - -```yaml -# In worker prompt -Track processed items in repo-memory: -- Path: `memory/campaigns/${campaignId}/processed-${workerId}.json` -- Format: `{"processed": ["item-1", "item-2"]}` - -Before processing: -1. Load processed items from repo-memory -2. Check if current work_item_id is in list -3. If in list: Skip -4. If not: Process, add to list, save -``` - -### Pattern 4: Issue Title-based Idempotency - -```yaml -# In worker prompt -Issue title pattern: `[${workItemId}] ${description}` - -Before creating issue: -1. Search for issues with `[${workItemId}]` in title -2. Filter by `z_campaign_${campaignId}` label -3. If found: Update existing issue -4. If not: Create new issue -``` - -## Example: Security Fix Worker - -```yaml ---- -name: Security Fix Worker -on: - workflow_dispatch: - inputs: - campaign_id: - description: 'Campaign identifier' - required: true - type: string - payload: - description: 'JSON with alert details' - required: true - type: string - -tracker-id: security-fix-worker - -tools: - github: - toolsets: [default, code_security] - bash: ["*"] - edit: true - -safe-outputs: - create-pull-request: - max: 1 ---- - -# Security Fix Worker - -Process a code scanning alert and create a fix PR. - -## Parse Input - -```javascript -const campaignId = context.payload.inputs.campaign_id; -const payload = JSON.parse(context.payload.inputs.payload); -// payload: { repository, work_item_id: "alert-123", alert_type, file_path, ... } -``` - -## Idempotency Check - -```javascript -const workKey = `campaign-${campaignId}-alert-${payload.work_item_id}`; -const branchName = `fix/${workKey}`; -const prTitle = `[${workKey}] Fix: ${payload.alert_type} in ${payload.file_path}`; - -// Search for existing PR -const existingPRs = await github.search.issuesAndPullRequests({ - q: `repo:${payload.repository} is:pr is:open "${workKey}" in:title` -}); - -if (existingPRs.total_count > 0) { - console.log(`PR already exists: ${existingPRs.items[0].html_url}`); - // Optionally add comment with update - await github.issues.createComment({ - owner: payload.repository.split('/')[0], - repo: payload.repository.split('/')[1], - issue_number: existingPRs.items[0].number, - body: `Still being tracked by campaign ${campaignId}` - }); - return; -} -``` - -## Create Fix - -```bash -# Clone repo and create branch -git clone https://github.com/${payload.repository}.git -cd $(basename ${payload.repository}) -git checkout -b ${branchName} - -# Make security fix -# ... fix code ... - -# Commit and push -git add . -git commit -m "Fix ${payload.alert_type} in ${payload.file_path}" -git push origin ${branchName} -``` - -## Create PR - -```javascript -const pr = await github.pulls.create({ - owner: payload.repository.split('/')[0], - repo: payload.repository.split('/')[1], - title: prTitle, - body: `Fixes security alert ${payload.work_item_id}\n\n**Campaign**: ${campaignId}\n**Alert Type**: ${payload.alert_type}`, - head: branchName, - base: payload.target_ref || 'main' -}); - -// Apply labels -await github.issues.addLabels({ - owner: payload.repository.split('/')[0], - repo: payload.repository.split('/')[1], - issue_number: pr.number, - labels: [`z_campaign_${campaignId}`, 'security', 'automated'] -}); - -console.log(`Created PR: ${pr.html_url}`); -``` - -## Report Status - -Output: -- PR URL -- Alert ID processed -- Fix applied -- Labels added -``` - -## Best Practices - -### 1. Single Responsibility - -Each worker should have one clear purpose: -- ✅ "Create security fix PRs" -- ✅ "Update dependency versions" -- ❌ "Scan and fix and test and deploy" - -### 2. Deterministic Behavior - -Workers should produce the same output for the same input: -- Use deterministic keys based on input data -- Don't rely on timestamps or random values -- Make work idempotent via existence checks - -### 3. Explicit Errors - -Report errors clearly: -- Log what failed and why -- Include relevant context (repo, work item ID) -- Don't fail silently - -### 4. Minimal Permissions - -Request only needed permissions: -- Use specific GitHub toolsets -- Limit safe-output maxima (start with 1-3) -- Don't request wildcard permissions - -### 5. Clear Completion Status - -Always report what happened: -- "Created PR: [url]" -- "Skipped: PR already exists" -- "Failed: Missing required data" - -## Testing Workers - -Before using a worker in a campaign: - -1. **Test manually** with workflow_dispatch: - ```bash - gh workflow run my-worker.yml \ - -f campaign_id=test-campaign \ - -f payload='{"repository":"owner/repo","work_item_id":"test-1"}' - ``` - -2. **Verify idempotency** by running twice: - - First run should create resources - - Second run should skip/update without errors - -3. **Check labels** on created items: -- Verify `z_campaign_test-campaign` label is applied - - Confirm tracker-id is in description (if applicable) - -4. **Test error cases**: - - Invalid repository - - Missing payload fields - - Duplicate work items - -## Migration from Fusion Approach - -If you have workflows that used the old fusion approach: - -### Before (Fusion): -```yaml -on: - schedule: daily - push: - workflow_dispatch: # Added by fusion -``` - -### After (Dispatch-Only): -```yaml -on: - workflow_dispatch: - inputs: - campaign_id: - description: 'Campaign identifier' - required: true - type: string - payload: - description: 'JSON payload' - required: true - type: string -``` - -### Migration Steps: - -1. **Remove autonomous triggers**: Delete schedule/push/pull_request -2. **Add input contract**: Add campaign_id and payload inputs -3. **Update prompt**: Parse inputs at the start -4. **Add idempotency**: Implement deterministic key checking -5. **Apply campaign label**: Label all created items -6. **Test**: Verify with manual dispatch - -## See Also - -- [Campaign Files Architecture](../scratchpad/campaigns-files.md) -- [Campaign Examples](./src/content/docs/examples/campaigns.md) -- [Safe Outputs Documentation](./src/content/docs/reference/safe-outputs.md) diff --git a/docs/src/content/docs/examples/campaigns.md b/docs/src/content/docs/examples/campaigns.md index 87f01a6fd6..dfd3122b3a 100644 --- a/docs/src/content/docs/examples/campaigns.md +++ b/docs/src/content/docs/examples/campaigns.md @@ -192,6 +192,4 @@ The campaign discovers these via tracker labels without controlling execution. ## Further reading - [Campaign guides](/gh-aw/guides/campaigns/) - Setup and configuration -- [Campaign lifecycle](/gh-aw/guides/campaigns/lifecycle/) - Execution model -- [Campaign specs](/gh-aw/guides/campaigns/specs/) - Configuration reference - [Safe outputs](/gh-aw/reference/safe-outputs/) - dispatch_workflow configuration diff --git a/docs/src/content/docs/guides/campaigns/cli-commands.md b/docs/src/content/docs/guides/campaigns/cli-commands.md deleted file mode 100644 index 682e1328d5..0000000000 --- a/docs/src/content/docs/guides/campaigns/cli-commands.md +++ /dev/null @@ -1,198 +0,0 @@ ---- -title: CLI commands -description: Command reference for campaign management -banner: - content: '⚠️ Deprecated: Campaign CLI commands for .campaign.md files are deprecated. Use the project field in workflow frontmatter instead.' ---- - -:::caution[Commands deprecated] -The `gh aw campaign` commands described here operate on the deprecated `.campaign.md` file format. For project tracking, use the `project` field in workflow frontmatter instead. See [Project Tracking](/gh-aw/reference/frontmatter/#project-tracking-project). -::: - -The GitHub Agentic Workflows CLI provides commands for listing, inspecting, validating, and managing campaigns (deprecated `.campaign.md` format). - -:::note -Use the automated creation flow to create campaigns. These commands are for managing existing campaigns. See [Getting started](/gh-aw/guides/campaigns/getting-started/). -::: - -## Campaign commands - -```bash -gh aw campaign # List all campaigns -gh aw campaign security # Filter by ID or name -gh aw campaign --json # JSON output - -gh aw campaign status # Status for all campaigns -gh aw campaign status incident # Filter status by ID or name -gh aw campaign status --json # JSON status output - -gh aw campaign new my-campaign # Scaffold new spec (advanced) -gh aw campaign new my-campaign --project --owner @me # Create with GitHub Project -gh aw campaign validate # Validate all specs -gh aw campaign validate --no-strict # Report without failing -``` - -## List campaigns - -View all campaign specs in `.github/workflows/*.campaign.md`: - -```bash -gh aw campaign -``` - -Output shows campaign ID, name, state, and file path. - -### Filter by name or ID - -```bash -gh aw campaign security -``` - -Shows campaigns containing "security" in ID or name. - -### JSON output - -```bash -gh aw campaign --json -``` - -Returns structured data for scripting and automation. - -## Check campaign status - -View live status from project boards: - -```bash -gh aw campaign status -``` - -Shows active campaigns with project board statistics, progress, and health indicators. - -### Filter status - -```bash -gh aw campaign status incident -``` - -Shows status for campaigns matching "incident". - -### JSON status - -```bash -gh aw campaign status --json -``` - -Returns structured status data including metrics, KPIs, and item counts. - -## Validate campaigns - -Check all campaign specs for configuration errors: - -```bash -gh aw campaign validate -``` - -Validates: -- Required fields present -- Valid YAML syntax -- Proper KPI configuration -- Discovery scope configured -- Project URLs valid -- Workflow references exist - -Exit code 1 indicates validation failures. - -### Non-failing validation - -```bash -gh aw campaign validate --no-strict -``` - -Reports problems without failing. Useful for CI pipelines during development. - -## Create new campaign (advanced) - -:::caution -Advanced users only. Most users should use the [automated creation flow](/gh-aw/guides/campaigns/getting-started/). -::: - -Scaffold a new campaign spec: - -```bash -gh aw campaign new my-campaign-id -``` - -Creates `.github/workflows/my-campaign-id.campaign.md` with basic structure. You must: -1. Configure all required fields -2. Set up project board manually -3. Compile the spec with `gh aw compile` -4. Test thoroughly before running - -### Create campaign with project board - -Create a campaign spec and automatically generate a GitHub Project with required fields and views: - -```bash -gh aw campaign new my-campaign-id --project --owner @me -``` - -Or for an organization: - -```bash -gh aw campaign new my-campaign-id --project --owner myorg -``` - -This creates: -- Campaign spec file at `.github/workflows/my-campaign-id.campaign.md` -- GitHub Project with standard views (Progress Board, Task Tracker, Campaign Roadmap) -- Required custom fields (Campaign Id, Worker Workflow, Priority, Size, Start Date, End Date) -- Updates the spec file with the project URL - -The automated flow handles all this for you. - -## Common workflows - -### Check campaign health - -```bash -# Quick health check -gh aw campaign status - -# Detailed inspection of specific campaign -gh aw campaign status security-audit --json | jq '.campaigns[0]' -``` - -### Pre-commit validation - -```bash -# In CI or pre-commit hook -gh aw campaign validate --no-strict -``` - -### Find inactive campaigns - -```bash -# List campaigns with their states -gh aw campaign --json | jq '.campaigns[] | {id, state}' -``` - -### Monitor campaign progress - -```bash -# Watch campaign status (requires watch/jq) -watch -n 300 'gh aw campaign status my-campaign' -``` - -## Exit codes - -| Code | Meaning | -|------|---------| -| 0 | Success | -| 1 | Validation error or command failed | -| 2 | Invalid arguments | - -## Further reading - -- [Campaign specs](/gh-aw/guides/campaigns/specs/) - Configuration format -- [Getting started](/gh-aw/guides/campaigns/getting-started/) - Create your first campaign -- [Campaign lifecycle](/gh-aw/guides/campaigns/lifecycle/) - Execution model diff --git a/docs/src/content/docs/guides/campaigns/creating-campaigns.md b/docs/src/content/docs/guides/campaigns/creating-campaigns.md deleted file mode 100644 index a4bf2048bb..0000000000 --- a/docs/src/content/docs/guides/campaigns/creating-campaigns.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -title: Creating Campaigns -description: How to create agentic campaigns using custom agent, interactive CLI wizard, or manual commands -banner: - content: '⚠️ Deprecated: Campaign creation for .campaign.md files is deprecated. Use the project field in workflow frontmatter instead.' ---- - -:::caution[File format deprecated] -This guide describes creating campaigns with the deprecated `.campaign.md` file format. For current project tracking, use the `project` field in workflow frontmatter. See [Project Tracking](/gh-aw/reference/frontmatter/#project-tracking-project). -::: - -There are three ways to create a campaign (deprecated format): - -## Recommended: CLI Interactive wizard - -Use the interactive wizard for guided campaign creation: - -```bash -gh aw campaign new --interactive -``` - -The wizard prompts you for: -- Campaign objective and description -- Repository scope (current repo, multiple repos, or org-wide) -- Workflow discovery and selection -- Owners and stakeholders -- Risk level assessment -- Project board creation - -This is the easiest way to create a well-configured campaign with all required fields. - -## Alternative: Custom agent (via Copilot Chat) - -You can also use the custom agent in GitHub Copilot Chat: - -1. **Open Copilot Chat** in your repository -2. **Type `/agent` with your campaign goal**, for example: - ``` - /agent create campaign: Burn down all open code security alerts, - prioritizing file-write alerts first and batching up to 3 related - alerts/PR with a brief fix rationale comment. - ``` -3. **Wait for the agent** to generate: - - GitHub Project board with required fields and views - - Campaign spec file (`.campaign.md`) - - Pull request with the campaign configuration -4. **Review and merge** the PR to activate the campaign - -The custom agent analyzes your goal description, discovers relevant workflows, and generates a complete campaign configuration ready for review. - -See [Getting started](/gh-aw/guides/campaigns/getting-started/) for a detailed walkthrough. - -## CLI-based creation - -### Manual mode - -For advanced users who prefer direct control: - -```bash -# Create campaign spec and GitHub Project -gh aw campaign new my-campaign-id --project --owner @me - -# Or for organizations -gh aw campaign new my-campaign-id --project --owner myorg -``` - -This scaffolds the campaign spec and creates a Project board, but you'll need to manually configure all fields, add worker workflows, and test thoroughly. - -See [CLI commands](/gh-aw/guides/campaigns/cli-commands/) for complete command reference. - -## Example: Security Alert Campaign - -Here's what a campaign spec looks like after creation: - -**Issue description** you provide: - -> Burn down all open code security alerts, prioritizing file-write alerts first -> and batching up to 3 related alerts/PR with a brief fix rationale comment. - -**Generated [campaign spec](/gh-aw/guides/campaigns/specs/)**: - -```markdown ---- -id: security-alert-burndown -name: "Security Alert Burndown" -description: "Drive the code security alerts backlog to zero" - -# GitHub Project for tracking -project-url: "https://github.com/orgs/ORG/projects/1" - -# Worker workflows to dispatch -workflows: - - security-alert-fix - -# Governance and pacing -governance: - max-project-updates-per-run: 10 - max-comments-per-run: 10 - -owners: - - "@security-team" ---- - -# Security Alert Burndown - -## Objective - -Reduce open code security alerts to zero without breaking CI. - -## Key Performance Indicators (KPIs) - -### Primary KPI: Open alerts -- **Baseline**: (fill in) -- **Target**: 0 -- **Time Window**: (fill in) -- **Direction**: Decrease -``` - -Notes: -- `tracker-label` is optional; when omitted it defaults to `z_campaign_`. -- Campaign narrative (objective, KPIs, timeline) belongs in the markdown body. - -The spec compiles into a campaign orchestrator workflow (`.campaign.lock.yml`) that GitHub Actions executes on schedule. The orchestrator [dispatches workers, tracks outputs, updates the Project board, and reports progress](/gh-aw/guides/campaigns/lifecycle/). - -## Next steps - -- [Getting started](/gh-aw/guides/campaigns/getting-started/) – step-by-step tutorial -- [Campaign specs](/gh-aw/guides/campaigns/specs/) – configuration reference -- [Campaign Lifecycle](/gh-aw/guides/campaigns/lifecycle/) – execution model -- [CLI commands](/gh-aw/guides/campaigns/cli-commands/) – command reference diff --git a/docs/src/content/docs/guides/campaigns/getting-started.md b/docs/src/content/docs/guides/campaigns/getting-started.md index 5f97f83ba0..f0fc127a73 100644 --- a/docs/src/content/docs/guides/campaigns/getting-started.md +++ b/docs/src/content/docs/guides/campaigns/getting-started.md @@ -1,67 +1,105 @@ --- title: Getting started -description: Quick start guide for creating and launching agentic campaigns -banner: - content: '⚠️ Deprecated: The .campaign.md file format is deprecated. Use the project field in workflow frontmatter instead.' +description: Quick start guide for creating campaign workflows --- -:::caution[File format deprecated] -This guide describes the deprecated `.campaign.md` file format. For current project tracking, use the `project` field in workflow frontmatter. See [Project Tracking](/gh-aw/reference/frontmatter/#project-tracking-project). -::: - -Create your first campaign using the custom agent in GitHub Copilot Chat. The agent generates a Project board, campaign spec, and orchestrator workflow based on your goal description. +This guide shows how to create a campaign workflow that coordinates work across repositories. ## Prerequisites - Repository with GitHub Agentic Workflows installed -- GitHub Copilot access -- Write access to create pull requests and merge them - GitHub Actions enabled +- A GitHub Projects board (or create one during setup) + +## Create a campaign workflow + +1. **Create a new workflow file** at `.github/workflows/my-campaign.md`: + +```yaml wrap +--- +name: My Campaign +on: + schedule: daily + workflow_dispatch: + +permissions: + issues: read + pull-requests: read + +imports: + - shared/campaign.md +--- + +# My Campaign + +- Project URL: https://github.com/orgs/myorg/projects/1 +- Campaign ID: my-campaign + +Your campaign instructions here... +``` + +2. **Set up authentication** for project access: + +```bash +gh aw secrets set GH_AW_PROJECT_GITHUB_TOKEN --value "YOUR_PROJECT_TOKEN" +``` + +See [GitHub Projects V2 Tokens](/gh-aw/reference/tokens/#gh_aw_project_github_token-github-projects-v2) for token setup. + +3. **Compile the workflow**: + +```bash +gh aw compile +``` -## Create a campaign +4. **Commit and push**: -1. **Open Copilot Chat** in your repository -2. **Describe your campaign** using `/agent`: - ``` - /agent create campaign: Burn down all open code security alerts, - prioritizing file-write alerts first - ``` -3. **Wait for the agent** - A pull request appears with your campaign configuration -4. **Review the PR** - Verify the generated Project, spec, and orchestrator -5. **Merge the PR** when ready -6. **Run the orchestrator** from the Actions tab to start the campaign +```bash +git add .github/workflows/my-campaign.md +git add .github/workflows/my-campaign.lock.yml +git commit -m "Add my campaign workflow" +git push +``` -## Generated files +## How it works -The pull request creates three components: +The campaign workflow: -**Project board** - GitHub Project for tracking campaign progress with custom fields and views. +1. Imports standard orchestration rules from `shared/campaign.md` +2. Runs on schedule to discover work items +3. Processes items according to your instructions +4. Updates the GitHub Project board with progress +5. Reports status via project status updates -**Campaign spec** - Configuration file at `.github/workflows/.campaign.md` defining campaign configuration (project URL, workflows, scope, governance). The markdown body contains narrative goals and success criteria. +## Campaign orchestration -**Orchestrator workflow** - Compiled workflow at `.github/workflows/.campaign.lock.yml` that executes the campaign logic. +The `imports: [shared/campaign.md]` provides: -## Campaign execution +- **Safe-output defaults**: Pre-configured limits for project operations +- **Execution phases**: Discover → Decide → Write → Report +- **Best practices**: Deterministic execution, pagination budgets, cursor management +- **Project integration**: Standard field mappings and status updates -The orchestrator runs on the configured schedule (daily by default): +## Example: Dependabot Burner -1. Dispatches worker workflows via `workflow_dispatch` (if configured) -2. Discovers issues and pull requests created by workers -3. Updates the Project board with new items -4. Posts a status update summarizing progress +See the [Dependabot Burner](https://github.com/githubnext/gh-aw/blob/main/.github/workflows/dependabot-burner.md) workflow for a complete example: -See [Campaign lifecycle](/gh-aw/guides/campaigns/lifecycle/) for execution details. +- Discovers open Dependabot PRs +- Creates bundle issues for upgrades +- Tracks everything in a GitHub Project +- Runs daily with smart conditional execution ## Best practices -Start with focused scope: -- Define one clear objective -- Include 1-3 worker workflows maximum -- Set conservative governance limits (e.g., 10 project updates per run) +- **Use imports** - Include `shared/campaign.md` for standard orchestration +- **Define campaign ID** - Include a clear Campaign ID in your workflow +- **Specify project URL** - Document the GitHub Projects board URL +- **Test manually** - Use `workflow_dispatch` trigger to test before scheduling +- **Monitor progress** - Check your project board to see tracked items -Configure worker triggers: -- Workers should accept `workflow_dispatch` only -- Remove cron schedules, push, and pull_request triggers -- Let the orchestrator control execution timing +## Next Steps -See [Campaign specs](/gh-aw/guides/campaigns/specs/) for configuration options. +- [Campaign Orchestration](/gh-aw/guides/campaigns/) - Overview and patterns +- [Project Tracking Example](/gh-aw/examples/project-tracking/) - Complete configuration reference +- [Safe Outputs](/gh-aw/reference/safe-outputs/) - Available project operations +- [Trigger Events](/gh-aw/reference/triggers/) - Workflow trigger options diff --git a/docs/src/content/docs/guides/campaigns/index.mdx b/docs/src/content/docs/guides/campaigns/index.mdx index 1be3210ecc..a60bf406c9 100644 --- a/docs/src/content/docs/guides/campaigns/index.mdx +++ b/docs/src/content/docs/guides/campaigns/index.mdx @@ -1,103 +1,100 @@ --- title: About Campaigns -description: Coordinate agentic workflows toward one goal, tracked in GitHub Projects. +description: Coordinate workflows to track progress in GitHub Projects using standard orchestration patterns. sidebar: - label: About Campaigns -banner: - content: 'Do not use. Campaigns are still incomplete and may produce unreliable or unintended results.' + label: Campaigns --- ## What are Agentic Campaigns? -**[Agentic campaigns](/gh-aw/reference/glossary/#agentic-campaign)** are initiatives that coordinate multiple [agentic workflows](/gh-aw/reference/glossary/#agentic-workflow) toward a shared goal. +Agentic Campaigns are workflows that coordinate autonomous agents across one or more repositories to drive progress toward a shared goal and make that progress easy to track. -When you need to roll out changes across many repositories—like upgrading dependencies, fixing security issues, or standardizing configurations—campaigns provide the orchestration layer you need to do that efficiently. +They run on a schedule and use: +- `imports` to include standard campaign orchestration from `shared/campaign.md` +- `project` to optionally track work with automatic [GitHub Projects](/gh-aw/reference/glossary/#github-projects-projects-v2) integration -They run on a schedule to work through repositories systematically, dispatch workers in batches to control load and avoid rate limits, and track all progress centrally in [GitHub Projects](/gh-aw/reference/glossary/#github-projects-projects-v2). +## Quick Start -You get complete visibility into what's done, what's in progress, and what's blocked—all in one place. +Add `project` and `imports` to your workflow frontmatter: -## When to use Campaigns - -Choose a campaign when you need coordination across repos or multiple workflows, with progress tracked in one place. - -- **Coordinate multiple worker workflows** that each do part of the job -- **Work across many repositories** (using campaign scope) -- **Track progress in one place** (a single [GitHub Project](/gh-aw/reference/glossary/#github-projects-projects-v2)) -- **Roll out changes in batches across repos** (repeatable runs with per-run limits and central tracking) - -**Don't need a campaign?** A single scheduled workflow is simpler if you're just automating work in one repository. - -**Not sure?** Start with one repository and one worker workflow, then scale up once you see how it works. - -## Campaign Specs - -To create a campaign, you write a single markdown file: +```yaml wrap +--- +name: Dependabot Burner +on: + schedule: daily + workflow_dispatch: -- `.github/workflows/.campaign.md` +imports: + - shared/campaign.md +--- -That file has two parts: +# Dependabot Burner -- **[Frontmatter](/gh-aw/reference/glossary/#frontmatter)**: the configuration ([GitHub Project URL](/gh-aw/reference/glossary/#github-projects-projects-v2), scope, worker list, governance limits) -- **Markdown body**: the narrative (objective, KPIs, prioritization, constraints) +- Project URL: https://github.com/orgs/myorg/projects/1 +- Campaign ID: dependabot-burner -Here’s a small example: +Find all open Dependabot PRs and add them to the project. +``` -```markdown wrap ---- -id: dependency-upgrades -project-url: "https://github.com/orgs/ORG/projects/1" -scope: - - "org:ORG" -workflows: - - dependency-scanner - - dependency-upgrader ---- +The `imports: [shared/campaign.md]` includes standard campaign orchestration rules and safe-output defaults. -# Dependency Upgrades +## Campaign Orchestration Pattern -Upgrade dependencies across repos. -``` +The `shared/campaign.md` import provides: -See: [Campaign specs](/gh-aw/guides/campaigns/specs/). +- **Safe-output defaults** - Pre-configured limits for project updates, issues, and status updates +- **Orchestration rules** - Best practices for discovery, deterministic execution, and reporting +- **Project integration** - Standard patterns for tracking work in GitHub Projects -## From markdown to GitHub Actions +## When to Use Campaigns -Just like regular workflows, campaigns get compiled: +Use the campaign pattern when you need: -- You run `gh aw compile` -- The compiler produces a **[workflow lock file](/gh-aw/reference/glossary/#workflow-lock-file-lockyml)** (`.lock.yml`) that GitHub Actions runs +- **Coordinated work** - Manage related tasks across multiple repositories +- **Progress visibility** - Track all work items in a central project board +- **Systematic execution** - Discover, process, and update work items on a schedule +- **Team coordination** - Share progress with stakeholders through project updates -This is the same idea as [compilation](/gh-aw/reference/glossary/#compilation) for normal workflows: you edit human-friendly markdown, GitHub Actions runs the compiled YAML. +**Don't need orchestration?** A simple workflow without `imports` is simpler if you just need basic project tracking. -## Orchestrator + workers (who does what) +## Example: Dependabot Burner -Campaigns split responsibilities on purpose: +The [Dependabot Burner workflow](https://github.com/githubnext/gh-aw/blob/main/.github/workflows/dependabot-burner.md) demonstrates the campaign pattern: -- **Orchestrator (generated from the spec)** - - Runs on a schedule - - Coordinates and decides what to do next - - Uses **[safe outputs](/gh-aw/reference/glossary/#safe-outputs)** to dispatch worker workflows -- **Workers (you provide these)** - - Are normal agentic workflows triggered via [`workflow_dispatch`](/gh-aw/reference/glossary/#workflow_dispatch) - - Do the actual GitHub writes (issues/PRs/Projects/comments) using their own safe outputs +```yaml +--- +name: Dependabot Burner +on: + schedule: daily + skip-if-no-match: 'is:pr is:open author:app/dependabot' + workflow_dispatch: + +permissions: + issues: read + pull-requests: read + +imports: + - shared/campaign.md +--- -## What you’ll see in GitHub +# Dependabot Burner -When a campaign is working, you’ll usually see: +- Project URL: https://github.com/orgs/githubnext/projects/144 +- Campaign ID: dependabot-burner -- A compiled workflow named `.campaign.lock.yml` in `.github/workflows/` -- A GitHub Project filling up with issues/PRs created by workers -- Tracking labels on those issues/PRs like `z_campaign_` -- Optional [labels](/gh-aw/reference/glossary/#labels) you can use to group and filter related workflows -- Worker workflow runs that create PRs and move items through statuses +- Find all open Dependabot PRs and add them to the project. +- Create bundle issues for each runtime + manifest file. +- Add bundle issues to the project and assign to Copilot. +``` -If you want the full execution story, see: [Campaign lifecycle](/gh-aw/guides/campaigns/lifecycle/). +Key features: +- **Imports orchestration** - Uses `shared/campaign.md` for standard rules +- **Scheduled execution** - Runs daily to process new Dependabot PRs +- **Smart discovery** - Only runs if matching PRs exist (`skip-if-no-match`) +- **Project tracking** - Adds discovered items to GitHub Projects board -## Next steps +## Next Steps -- [Getting started](/gh-aw/guides/campaigns/getting-started/) – step-by-step tutorial -- [Creating campaigns](/gh-aw/guides/campaigns/creating-campaigns/) – issue-based or CLI creation -- [Campaign specs](/gh-aw/guides/campaigns/specs/) – configuration reference -- [Campaign Lifecycle](/gh-aw/guides/campaigns/lifecycle/) – execution model -- [CLI commands](/gh-aw/guides/campaigns/cli-commands/) – command reference +- [Getting Started](/gh-aw/guides/campaigns/getting-started/) - Create your first campaign +- [Project Tracking Example](/gh-aw/examples/project-tracking/) - Complete guide with examples +- [Safe Outputs Reference](/gh-aw/reference/safe-outputs/) - Project operations documentation diff --git a/docs/src/content/docs/guides/campaigns/lifecycle.md b/docs/src/content/docs/guides/campaigns/lifecycle.md deleted file mode 100644 index 48bfabc439..0000000000 --- a/docs/src/content/docs/guides/campaigns/lifecycle.md +++ /dev/null @@ -1,185 +0,0 @@ ---- -title: Campaign Lifecycle -description: Campaign execution phases, state management, and workflow coordination -banner: - content: '⚠️ Deprecated: This describes the deprecated .campaign.md format. Use the project field in workflow frontmatter instead.' ---- - -:::caution[File format deprecated] -This guide describes the deprecated `.campaign.md` file format. For current project tracking, use the `project` field in workflow frontmatter. See [Project Tracking](/gh-aw/reference/frontmatter/#project-tracking-project). -::: - -Campaign orchestrators execute on a schedule to coordinate worker workflows and discover outputs. Orchestrators are dispatch-only: they can coordinate, but all GitHub writes (Projects, issues/PRs, comments) happen in worker workflows. - -## Execution flow - -```mermaid -graph TD - A[Orchestrator Triggered] --> B[Pre-step: Discovery Precomputation] - B --> C[Agent: Decide & Dispatch Workers] - C --> D[Workers: Apply Side Effects] - D --> E[Next Run: Discover Outputs] -``` - -Each run follows this sequence: - -1. **Pre-step** - A deterministic discovery script runs (via `actions/github-script`) and writes `./.gh-aw/campaign.discovery.json` -2. **Agent** - Reads the discovery manifest and campaign spec, then dispatches worker workflows via `safe-outputs.dispatch-workflow` -3. **Workers** - Create/update issues/PRs, apply labels, and update Project boards using their own safe-outputs - -## Campaign states - -| State | Description | Execution | -|-------|-------------|-----------| -| `planned` | Draft configuration under review | Not running | -| `active` | Production campaign | Runs on schedule | -| `paused` | Temporarily stopped | Not running | -| `completed` | Objectives achieved | Not running | -| `archived` | Historical reference | Not running | - -:::caution -The `state` field is documentation only. To stop execution, disable the workflow in GitHub Actions settings. -::: - -## Worker workflows - -Worker workflows perform campaign tasks (scanning, analysis, remediation). The orchestrator dispatches them via `workflow_dispatch` and discovers their outputs. - -### Requirements - -Worker workflows in the campaign's `workflows` list must: - -- Accept `workflow_dispatch` as the **only** trigger -- Remove all other triggers (`schedule`, `push`, `pull_request`) -- Label created items with the campaign tracker label (defaults to `z_campaign_`) -- Accept standardized inputs: `campaign_id` (string) and `payload` (string JSON) - -```yaml -# Worker workflow configuration -on: - workflow_dispatch: - inputs: - campaign_id: - description: 'Campaign identifier' - required: true - type: string - payload: - description: 'JSON payload with work details' - required: true - type: string -``` - -### Independent workflows - -Workflows not in the `workflows` list can keep their original triggers. The campaign discovers their outputs via tracker labels without controlling execution. - -```yaml -# Campaign spec -workflows: - - vulnerability-scanner # Orchestrator controls this one - # dependency-check runs independently with its cron schedule -``` - -`tracker-label` is optional; when omitted it defaults to `z_campaign_`. - -## Discovery and governance - -Discovery finds items created by workers based on tracker labels. Governance limits control the pace of work. - -```yaml -governance: - max-discovery-items-per-run: 50 - max-project-updates-per-run: 10 -``` - -When limits are reached: -- Discovery cursor saves the current position -- Remaining items are deferred to the next run -- Status update reports deferred count -- Campaign continues on next schedule - -The campaign processes items incrementally across multiple runs until all are handled. - -## Pausing and ending campaigns - -### Pause temporarily - -1. Update spec: `state: paused` -2. Disable workflow in Actions settings - -### Complete permanently - -1. Run orchestrator one final time for completion status -2. Update spec: `state: completed` -3. Disable workflow in Actions settings -4. Optionally delete `.campaign.lock.yml` (keep `.campaign.md` for history) - -### Archive for reference - -```yaml ---- -id: security-q1-2025 -state: archived ---- - -Completed 2025-03-15. Final metrics: -- Tasks: 200/200 -- Duration: 90 days -- Velocity: 7.5 tasks/day -``` - -## Troubleshooting - -**Worker dispatch fails** -- Verify workflow exists and has `workflow_dispatch` trigger -- Check workflow file name matches spec -- Ensure no compilation errors in worker - -**Discovery finds no items** -- Verify tracker label matches campaign ID -- Check workers are creating items with correct labels -- Confirm discovery scope includes correct repos/orgs - -**Project updates hit limit** -- Increase `max-project-updates-per-run` in governance (used as a pacing signal for generated instructions) -- Accept incremental processing across multiple runs -- Verify the worker workflow token has required Projects permissions - -**Items processed multiple times** -- Ensure workers use deterministic keys -- Check for duplicate labels on items -- Verify idempotency logic in worker code - -## Advanced: Pre-existing workflows - -### Converting scheduled workflows - -When adding an existing scheduled workflow to a campaign: - -**Before** (independent): -```yaml -on: - schedule: daily - workflow_dispatch: -``` - -**After** (campaign-controlled): -```yaml -on: - workflow_dispatch: # Only this trigger - # schedule: daily # Removed - campaign controls timing -``` - -### Event-driven workflows - -Workflows triggered by code events (`push`, `pull_request`) should not be campaign-controlled. These respond to specific events, not campaign schedules. - -**Recommended**: Keep them independent and let the campaign discover their outputs. - -**Not recommended**: Adding them to campaign's `workflows` list requires removing event triggers, which defeats their purpose. - -## Further reading - -- [Campaign specs](/gh-aw/guides/campaigns/specs/) - Configuration reference -- [Getting started](/gh-aw/guides/campaigns/getting-started/) - Create your first campaign -- [CLI commands](/gh-aw/guides/campaigns/cli-commands/) - Management commands diff --git a/docs/src/content/docs/guides/campaigns/specs.md b/docs/src/content/docs/guides/campaigns/specs.md deleted file mode 100644 index 6d55f4577c..0000000000 --- a/docs/src/content/docs/guides/campaigns/specs.md +++ /dev/null @@ -1,320 +0,0 @@ ---- -title: Campaign specs -description: Campaign specification format and configuration reference -banner: - content: '⚠️ Deprecated: The .campaign.md file format is deprecated. Use the project field in workflow frontmatter instead. See Project Tracking for the current approach.' ---- - -:::caution[File format deprecated] -The `.campaign.md` standalone file format described in this document is **deprecated** and has been removed from gh-aw. - -**Migration:** Use the `project` field in workflow frontmatter instead: -```yaml ---- -on: - schedule: - - cron: "0 0 * * *" -project: - url: https://github.com/orgs/myorg/projects/1 - workflows: - - worker-workflow-name ---- -``` -See [Project Tracking documentation](/gh-aw/reference/frontmatter/#project-tracking-project) for details. -::: - -Campaign specs were YAML frontmatter configuration files at `.github/workflows/.campaign.md`. The frontmatter defined pure configuration (id, project-url, workflows, governance, etc.), while the markdown body contained narrative context including objectives, KPIs, timelines, and strategy. - -## Spec structure (deprecated) - -:::note[Historical reference] -This section documents the deprecated `.campaign.md` file format for historical reference only. -::: - -A minimal campaign spec was a `.github/workflows/.campaign.md` file with YAML frontmatter plus a markdown body. Most fields had sensible defaults. - -```markdown ---- -id: framework-upgrade -name: "Framework Upgrade" -project-url: "https://github.com/orgs/ORG/projects/1" - -workflows: - - framework-upgrade-scanner - -governance: - max-project-updates-per-run: 10 ---- - -# Framework Upgrade Campaign - -## Objective - -Upgrade all services to Framework vNext with zero downtime. - -## Key Performance Indicators (KPIs) - -### Primary KPI: Services Upgraded -- **Baseline**: 0 services -- **Target**: 50 services -- **Time Window**: 30 days -- **Direction**: Increase - -## Timeline - -- **Phase 1** (Weeks 1-2): Discovery and planning -- **Phase 2** (Weeks 3-6): Incremental upgrades -- **Phase 3** (Week 7+): Validation and monitoring -``` - -## Required fields - -### Identity - -**id** - Stable identifier for file naming and reporting -- Format: lowercase letters, digits, hyphens only -- Example: `security-audit-2025` -- Auto-generates defaults for: tracker-label, memory-paths, metrics-glob, cursor-glob -- If omitted, defaults to the filename basename (e.g. `security-audit.campaign.md` → `security-audit`) - -**name** - Human-friendly display name -- Example: `"Security Audit 2025"` -- Default: Uses `id` if not specified - -**project-url** - GitHub Project board URL for tracking -- Format: `https://github.com/orgs/ORG/projects/N` -- Example: `https://github.com/orgs/mycompany/projects/1` - -**workflows** - Worker workflows that implement the campaign -- Format: List of workflow IDs (file names without .md extension) -- Example: `["security-scanner", "dependency-fixer"]` - -`workflows` is strongly recommended for most campaigns (and `gh aw campaign validate` will flag empty workflows). It can be omitted for campaigns that only do coordination/discovery work. - -## Fields with defaults - -Many fields have automatic defaults based on the campaign ID: - -**state** - Lifecycle stage -- Default: `active` -- Values: `planned`, `active`, `paused`, `completed`, `archived` - -**tracker-label** - Label for discovering worker outputs -- Default: `z_campaign_{id}` (e.g., `z_campaign_security-audit`) -- Can be customized if needed - -**memory-paths** - Where campaign writes repo-memory -- Default: `["memory/campaigns/{id}/**"]` - -**metrics-glob** - Glob for JSON metrics snapshots -- Default: `memory/campaigns/{id}/metrics/*.json` - -**cursor-glob** - Glob for durable cursor/checkpoint file -- Default: `memory/campaigns/{id}/cursor.json` - -**scope** - Repositories and organizations this campaign can operate on -- Default: Current repository (where campaign is defined) - -Campaign scope is defined once and used for both discovery and execution: - -**scope** - Scope selectors -- Repository selector: `owner/repo` -- Organization selector: `org:` -- Example: `["myorg/api", "myorg/web", "org:myorg"]` - -## Optional fields - -**description** - Brief campaign description -- Provides context in listings and dashboards - -**version** - Spec format version -- Default: `v1` -- Usually not needed in specs - -**owners** - Primary human owners -- Format: List of team or user names -- Example: `["@security-team", "alice"]` - -**governance** - Pacing and safety limits -- See [Governance fields](#governance-fields) below - -No other scope fields are needed; use `scope`. - -Campaign orchestrators are **dispatch-only by design**: -- The orchestrator can make decisions and coordinate work. -- The orchestrator may only *act* by dispatching allowlisted worker workflows via `safe-outputs.dispatch-workflow`. -- All side effects (Projects, issues/PRs, comments) happen in worker workflows with their own safe-outputs. - -## Markdown body content - -The markdown body contains narrative context and goals. Include: - -**Objective** - Clear statement of what the campaign aims to achieve -- Example: "Reduce all critical security vulnerabilities to zero" -- Can include multiple paragraphs with context and rationale - -**Key Performance Indicators (KPIs)** - Measurable success metrics -- Define 1 primary KPI + up to 2 supporting KPIs -- Include baseline, target, time window, direction for each -- Example format: - ```markdown - ### Primary KPI: Critical Vulnerabilities - - **Baseline**: 15 issues - - **Target**: 0 issues - - **Time Window**: 90 days - - **Direction**: Decrease - ``` - -**Timeline** - Campaign phases and milestones -**Worker Workflows** - Descriptions of automated workflows -**Success Criteria** - Concrete conditions for completion -**Risk Management** - Mitigation strategies and approvals - -## Governance fields - -Governance controls execution pace and safety: - -```yaml -governance: - max-project-updates-per-run: 10 - max-discovery-items-per-run: 50 - max-discovery-pages-per-run: 5 - max-new-items-per-run: 10 - max-comments-per-run: 10 - do-not-downgrade-done-items: true - opt-out-labels: ["campaign:skip", "no-bot"] -``` - -**max-project-updates-per-run** - Maximum project board updates per execution -- Default: Conservative limit -- Start low (10) and increase with confidence - -**max-discovery-items-per-run** - Maximum items to discover per execution -- Controls API load -- Remaining items discovered on next run - -**max-discovery-pages-per-run** - Maximum API pages to fetch -- Alternative to item limit - -**max-new-items-per-run** - Maximum new items to add to project -- Separate from total updates - -**max-comments-per-run** - Maximum comments to post -- Prevents notification spam - -**do-not-downgrade-done-items** - Prevent moving completed items backward -- Recommended: `true` - -**opt-out-labels** - Labels that exclude items from campaign -- Default: `["no-bot", "no-campaign"]` - -## Discovery configuration -Campaign discovery uses the same `scope` as execution. - -```yaml -scope: - - "myorg/frontend" - - "myorg/backend" - - "myorg/api" - - "org:myorg" # optional org-wide scope -``` - -## Validation - -Validate campaign specs before committing: - -```bash -gh aw campaign validate -``` - -Common validation errors: - -- Missing required fields (`id`, `project-url`, `workflows`) -- Invalid `state` value -- Malformed URLs or identifiers - -## Minimal example - -The simplest possible campaign: - -```markdown ---- -id: security-audit-q1 -name: "Security Audit Q1 2025" -project-url: "https://github.com/orgs/myorg/projects/5" - -workflows: - - security-scanner - - dependency-updater ---- - -# Security Audit Q1 2025 Campaign - -Document your objectives, KPIs, timeline, and strategy here... -``` - -This automatically gets: -- `state: active` -- `tracker-label: z_campaign_security-audit-q1` -- `memory-paths: ["memory/campaigns/security-audit-q1/**"]` -- `metrics-glob: memory/campaigns/security-audit-q1/metrics/*.json` -- `cursor-glob: memory/campaigns/security-audit-q1/cursor.json` -- `scope`: current repository - -## Full example - -With governance and org scope: - -```yaml ---- -id: security-audit-q1 -name: "Security Audit Q1 2025" -description: "Quarterly security review and remediation" -project-url: "https://github.com/orgs/myorg/projects/5" - -scope: - - "org:myorg" - -workflows: - - security-scanner - - dependency-updater - -governance: - max-project-updates-per-run: 20 - max-discovery-items-per-run: 100 - do-not-downgrade-done-items: true - -owners: - - "security-team" ---- - -# Security Audit Q1 2025 Campaign - -## Objective - -Resolve all high and critical security vulnerabilities across the organization. - -## Key Performance Indicators (KPIs) - -### Primary KPI: Critical Vulnerabilities -- **Baseline**: 15 vulnerabilities -- **Target**: 0 vulnerabilities -- **Time Window**: 90 days -- **Direction**: Decrease - -### Supporting KPI: Mean Time to Resolution -- **Baseline**: 14 days -- **Target**: 3 days -- **Time Window**: 30 days -- **Direction**: Decrease - -## Timeline - -This campaign runs weekly to scan for vulnerabilities and track remediation. Workers create issues with severity labels and automated fix PRs where possible. -``` - -## Further reading - -- [Campaign lifecycle](/gh-aw/guides/campaigns/lifecycle/) - Execution model -- [Getting started](/gh-aw/guides/campaigns/getting-started/) - Create your first campaign -- [CLI commands](/gh-aw/guides/campaigns/cli-commands/) - Validation and management diff --git a/docs/src/content/docs/guides/specops.md b/docs/src/content/docs/guides/specops.md index 0b29c9a313..4059a232b2 100644 --- a/docs/src/content/docs/guides/specops.md +++ b/docs/src/content/docs/guides/specops.md @@ -192,7 +192,6 @@ The [MCP Gateway Specification](/gh-aw/reference/mcp-gateway/) demonstrates Spec ## Related Patterns -- **[Campaign Specs](/gh-aw/guides/campaigns/specs/)** - Reviewable workflow contracts - **[MultiRepoOps](/gh-aw/guides/multirepoops/)** - Cross-repository coordination ## References diff --git a/scratchpad/campaigns-files.md b/scratchpad/campaigns-files.md deleted file mode 100644 index 47c4e6bca0..0000000000 --- a/scratchpad/campaigns-files.md +++ /dev/null @@ -1,949 +0,0 @@ -# Campaign Files Architecture - -This document describes how campaigns are discovered, compiled, and executed in GitHub Agentic Workflows. It covers the complete lifecycle from campaign spec files to running workflows. - -## Overview - -Campaigns are a first-class feature in gh-aw that enable coordinated, multi-repository initiatives. The campaign system consists of: - -1. **Campaign Spec Files** (`.campaign.md`) - Declarative YAML frontmatter defining campaign configuration -2. **Discovery Script** (`campaign_discovery.cjs`) - JavaScript that searches GitHub for campaign items -3. **Orchestrator Generator** - Go code that builds agentic workflows from campaign specs -4. **Compiled Workflows** (`.campaign.lock.yml`) - GitHub Actions workflows that run the campaigns - -## File Locations - -``` -.github/workflows/ -├── .campaign.md # Campaign spec (source of truth) -├── .campaign.g.md # Generated orchestrator (debug artifact, not committed) -└── .campaign.lock.yml # Compiled workflow (committed) - -actions/setup/js/ -└── campaign_discovery.cjs # Discovery precomputation script - -pkg/campaign/ -├── spec.go # Campaign spec data structures -├── loader.go # Campaign discovery and loading -├── orchestrator.go # Orchestrator generation -└── validation.go # Campaign spec validation -``` - -## Campaign Discovery Process - -### 1. Local Repository Discovery - -**Implementation**: `pkg/campaign/loader.go:LoadSpecs()` - -The campaign system discovers campaign specs by scanning the local repository: - -```go -// Scan .github/workflows/ for *.campaign.md files -workflowsDir := filepath.Join(rootDir, ".github", "workflows") -entries, err := os.ReadDir(workflowsDir) - -// For each .campaign.md file: -// 1. Read file contents -// 2. Parse YAML frontmatter using parser.ExtractFrontmatterFromContent() -// 3. Unmarshal to CampaignSpec struct -// 4. Set default ID and Name if not provided -// 5. Store relative path in ConfigPath field -``` - -**Key features**: -- Only scans `.campaign.md` files (not `.md` or `.g.md`) -- Returns empty slice if `.github/workflows/` doesn't exist (no error) -- Populates `ConfigPath` with repository-relative path -- Auto-generates ID from filename if not specified in frontmatter - -### 2. Campaign Spec Structure - -**Implementation**: `pkg/campaign/spec.go:CampaignSpec` - -Campaign specs use YAML frontmatter with these key fields: - -```yaml ---- -id: security-q1-2025 -name: Security Q1 2025 -version: v1 -state: active - -# Project integration -project-url: https://github.com/orgs/ORG/projects/1 -tracker-label: z_campaign_security-q1-2025 - -# Associated workflows -workflows: - - vulnerability-scanner - - dependency-updater - -# Repo-memory configuration -memory-paths: - - memory/campaigns/security-q1-2025/** -metrics-glob: memory/campaigns/security-q1-2025/metrics/*.json -cursor-glob: memory/campaigns/security-q1-2025/cursor.json - -# Governance -governance: - max-new-items-per-run: 25 - max-discovery-items-per-run: 200 - max-discovery-pages-per-run: 10 - opt-out-labels: [no-campaign, no-bot] - max-project-updates-per-run: 10 - max-comments-per-run: 10 ---- -``` - -## Campaign Compilation Process - -### 1. Detection During Compile - -**Implementation**: `pkg/cli/compile_workflow_processor.go:processCampaignSpec()` - -During `gh aw compile`, the system: - -1. Scans `.github/workflows/` for both `.md` and `.campaign.md` files -2. Detects `.campaign.md` suffix to trigger campaign processing -3. Loads and validates the campaign spec -4. Generates an orchestrator workflow if the spec has meaningful details - -**Meaningful details check** (`pkg/campaign/orchestrator.go:BuildOrchestrator()`): -- Must have at least one of: workflows, memory paths, metrics glob, cursor glob, project URL, governance, or KPIs -- Returns `nil` if campaign has no actionable configuration -- This prevents empty orchestrators from being generated - -### 2. Orchestrator Generation - -**Implementation**: `pkg/campaign/orchestrator.go:BuildOrchestrator()` - -The orchestrator generator creates a `workflow.WorkflowData` struct containing: - -#### A. Discovery Precomputation Steps - -**Function**: `buildDiscoverySteps()` - -When a campaign has workflows or a tracker label, the generator adds discovery steps: - -```yaml -steps: - - name: Create workspace directory - run: mkdir -p ./.gh-aw - - - name: Run campaign discovery precomputation - id: discovery - uses: actions/github-script@v8.0.0 - env: - GH_AW_CAMPAIGN_ID: security-q1-2025 - GH_AW_WORKFLOWS: "vulnerability-scanner,dependency-updater" - GH_AW_TRACKER_LABEL: z_campaign_security-q1-2025 - GH_AW_PROJECT_URL: https://github.com/orgs/ORG/projects/1 - GH_AW_MAX_DISCOVERY_ITEMS: 200 - GH_AW_MAX_DISCOVERY_PAGES: 10 - GH_AW_CURSOR_PATH: /tmp/gh-aw/repo-memory/campaigns/security-q1-2025/cursor.json - with: - github-token: ${{ secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN || secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN }} - script: | - const { setupGlobals } = require('/opt/gh-aw/actions/setup_globals.cjs'); - setupGlobals(core, github, context, exec, io); - const { main } = require('/opt/gh-aw/actions/campaign_discovery.cjs'); - await main(); -``` - -**Discovery script location**: The script is loaded from `/opt/gh-aw/actions/campaign_discovery.cjs`, which is copied during the `actions/setup` step. - -#### B. Workflow Metadata - -```go -data := &workflow.WorkflowData{ - Name: spec.Name, - Description: spec.Description, - On: "on:\n schedule:\n - cron: \"0 18 * * *\"\n workflow_dispatch:\n", - Concurrency: fmt.Sprintf("concurrency:\n group: \"campaign-%s-orchestrator-${{ github.ref }}\"\n cancel-in-progress: false", spec.ID), - RunsOn: "runs-on: ubuntu-latest", - Roles: []string{"admin", "maintainer", "write"}, -} -``` - -#### C. Tools Configuration - -```go -Tools: map[string]any{ - "repo-memory": []any{ - map[string]any{ - "id": "campaigns", - "branch-name": "memory/campaigns", - "file-glob": extractFileGlobPatterns(spec), - "campaign-id": spec.ID, - }, - }, - "bash": []any{"*"}, - "edit": nil, -} -``` - -Note: orchestrators deliberately omit GitHub tool access. All writes and GitHub API operations should be performed by dispatched worker workflows. - -#### D. Safe Outputs Configuration - -```go -safeOutputs := &workflow.SafeOutputsConfig{} - -// Campaign orchestrators are dispatch-only: they may only dispatch allowlisted -// workflows via the dispatch-workflow safe output. -if len(spec.Workflows) > 0 { - safeOutputs.DispatchWorkflow = &workflow.DispatchWorkflowConfig{ - BaseSafeOutputConfig: workflow.BaseSafeOutputConfig{Max: 3}, - Workflows: spec.Workflows, - } -} -``` - -Workers are responsible for side effects (Projects, issues/PRs, comments) using their own tool configuration and safe-outputs. - -#### E. Prompt Section - -The orchestrator includes detailed instructions for the AI agent: - -```go -markdownBuilder.WriteString("# Campaign Orchestrator\n\n") -// Campaign details: objective, KPIs, workflows, memory paths, etc. - -orchestratorInstructions := RenderOrchestratorInstructions(promptData) -projectInstructions := RenderProjectUpdateInstructions(promptData) -closingInstructions := RenderClosingInstructions() -``` - -### 3. Markdown Generation - -**Implementation**: `pkg/cli/compile_orchestrator.go:renderGeneratedCampaignOrchestratorMarkdown()` - -The orchestrator is rendered as a markdown file: - -``` -.campaign.g.md -``` - -**Important**: This `.campaign.g.md` file is a **debug artifact**: -- Generated locally during compilation -- Helps users understand the orchestrator structure -- **NOT committed to git** (excluded via `.gitignore`) -- Can be reviewed locally to see generated workflow structure - -**Compiled output**: Only the `.campaign.lock.yml` file is committed to version control. - -### 4. Lock File Naming - -**Implementation**: `pkg/stringutil/identifiers.go:CampaignOrchestratorToLockFile()` - -Campaign orchestrators follow a special naming convention: - -``` -example.campaign.g.md → example.campaign.lock.yml -``` - -**Not**: `example.campaign.g.lock.yml` (the `.g` suffix is removed) - -This ensures the lock file name matches the campaign spec name pattern. - -## Discovery Script Architecture - -### Script Location - -**Source**: `actions/setup/js/campaign_discovery.cjs` - -**Runtime location**: `/opt/gh-aw/actions/campaign_discovery.cjs` - -The discovery script is copied to `/opt/gh-aw/actions/` during the `actions/setup` action, which runs before the agent job. - -### Discovery Flow - -**Implementation**: `actions/setup/js/campaign_discovery.cjs:main()` - -1. **Read configuration from environment variables**: - - `GH_AW_CAMPAIGN_ID` - Campaign identifier - - `GH_AW_WORKFLOWS` - Comma-separated list of workflow IDs (tracker-ids) - - `GH_AW_TRACKER_LABEL` - Optional label for discovery - - `GH_AW_MAX_DISCOVERY_ITEMS` - Budget for items to discover (default: 100) - - `GH_AW_MAX_DISCOVERY_PAGES` - Budget for API pages to fetch (default: 10) - - `GH_AW_CURSOR_PATH` - Path to cursor file for pagination - - `GH_AW_PROJECT_URL` - Project URL for reference - -2. **Load cursor from repo-memory** (if configured): - ```javascript - function loadCursor(cursorPath) { - if (fs.existsSync(cursorPath)) { - const content = fs.readFileSync(cursorPath, "utf8"); - return JSON.parse(content); - } - return null; - } - ``` - -3. **Primary discovery: search by campaign-specific label**: - - Derived label: `z_campaign_` - - Query: `label:"z_campaign_"` scoped to repos/orgs - -4. **Secondary discovery: search by generic `agentic-campaign` label**: - - Query: `label:"agentic-campaign"` scoped to repos/orgs - -5. **Fallback discovery: search by tracker-id markers**: - ```javascript - // For each workflow in spec.workflows: - const searchQuery = `"gh-aw-tracker-id: ${trackerId}" type:issue`; - // (scoped with repo: and/or org: terms) - ``` - -6. **Legacy discovery: search by configured tracker label** (if provided): - - Query: `label:"${label}"` - -5. **Normalize discovered items**: - ```javascript - function normalizeItem(item, contentType) { - return { - url: item.html_url || item.url, - content_type: contentType, // "issue" or "pull_request" - number: item.number, - repo: item.repository?.full_name || "", - created_at: item.created_at, - updated_at: item.updated_at, - state: item.state, - title: item.title, - closed_at: item.closed_at, - merged_at: item.merged_at, - }; - } - ``` - -6. **Deduplicate items** (when using both tracker-id and tracker-label): - ```javascript - const existingUrls = new Set(allItems.map(i => i.url)); - for (const item of result.items) { - if (!existingUrls.has(item.url)) { - allItems.push(item); - } - } - ``` - -7. **Sort for stable ordering**: - ```javascript - allItems.sort((a, b) => { - if (a.updated_at !== b.updated_at) { - return a.updated_at.localeCompare(b.updated_at); - } - return a.number - b.number; - }); - ``` - -8. **Calculate summary counts**: - ```javascript - const needsAddCount = allItems.filter(i => i.state === "open").length; - const needsUpdateCount = allItems.filter(i => i.state === "closed" || i.merged_at).length; - ``` - -9. **Write manifest to `./.gh-aw/campaign.discovery.json`**: - ```json - { - "schema_version": "v1", - "campaign_id": "security-q1-2025", - "generated_at": "2025-01-08T12:00:00.000Z", - "project_url": "https://github.com/orgs/ORG/projects/1", - "discovery": { - "total_items": 42, - "items_scanned": 100, - "pages_scanned": 2, - "max_items_budget": 200, - "max_pages_budget": 10, - "cursor": { "page": 3, "trackerId": "vulnerability-scanner" } - }, - "summary": { - "needs_add_count": 25, - "needs_update_count": 17, - "open_count": 25, - "closed_count": 10, - "merged_count": 7 - }, - "items": [ - { - "url": "https://github.com/org/repo/issues/123", - "content_type": "issue", - "number": 123, - "repo": "org/repo", - "created_at": "2025-01-01T00:00:00Z", - "updated_at": "2025-01-07T12:00:00Z", - "state": "open", - "title": "Upgrade dependency X" - } - ] - } - ``` - -10. **Save cursor to repo-memory** (for next run): - ```javascript - function saveCursor(cursorPath, cursor) { - const dir = path.dirname(cursorPath); - if (!fs.existsSync(dir)) { - fs.mkdirSync(dir, { recursive: true }); - } - fs.writeFileSync(cursorPath, JSON.stringify(cursor, null, 2)); - } - ``` - -### Pagination Budgets - -The discovery system enforces strict pagination budgets to prevent unbounded API usage: - -- **Max items per run** (`governance.max-discovery-items-per-run`): Default 100, configurable -- **Max pages per run** (`governance.max-discovery-pages-per-run`): Default 10, configurable - -When budgets are reached: -```javascript -if (itemsScanned >= maxItems || pagesScanned >= maxPages) { - core.warning(`Reached discovery budget limits. Stopping discovery.`); - break; -} -``` - -### Cursor Persistence - -The cursor enables incremental discovery across runs: - -**Cursor format**: -```json -{ - "page": 3, - "trackerId": "vulnerability-scanner" -} -``` - -**Storage location**: Configured via `spec.CursorGlob`, typically: -``` -memory/campaigns//cursor.json -``` - -**How it works**: -1. Discovery loads cursor from repo-memory -2. Continues from saved page number -3. Updates cursor after each workflow/label search -4. Saves updated cursor back to repo-memory -5. Next run picks up where previous run left off - -### Campaign Item Protection - -The current campaign system’s primary tracking label format is: - -``` -z_campaign_ -``` - -Workers should apply this label to all created issues/PRs so discovery can find them reliably. - -Some workflows in this repo also treat `campaign:*` labels as a “do not touch” signal (legacy convention). If you need that compatibility, have workers apply both labels: - -``` -z_campaign_ -campaign: -``` - -Campaign specs can also define `governance.opt-out-labels` (for example: `no-bot`, `no-campaign`) to let humans opt items out of automated handling. - -## Campaign Workers - -Campaign workers are specialized workflows designed to be orchestrated by campaign orchestrators. They follow a first-class worker pattern with explicit contracts and idempotency. - -### Worker Design Principles - -1. **Dispatch-only triggers**: Workers use `workflow_dispatch` as the primary/only trigger - - No schedule, push, or pull_request triggers - - Clear ownership: workers are orchestrated, not autonomous - - Prevents duplicate execution from multiple trigger sources - -2. **Standardized input contract**: All workers accept: - - `campaign_id` (string): The campaign identifier orchestrating this worker - - `payload` (string): JSON-encoded data specific to the work item - -3. **Idempotency**: Workers implement deterministic behavior: - - Compute deterministic work item keys (e.g., `campaign-{id}-{repo}-{alert-id}`) - - Use keys in branch names, PR titles, issue titles - - Check for existing PR/issue with key + tracker label before creating - - Skip or update existing items rather than creating duplicates - -4. **Orchestration agnostic**: Workers don't know about orchestration policy - - Sequential vs parallel execution is orchestrator's concern - - Workers are simple, focused, deterministic units - -### Worker Workflow Template - -```yaml ---- -name: Campaign Worker Example -description: Example worker workflow for campaign orchestration - -on: - workflow_dispatch: - inputs: - campaign_id: - description: 'Campaign identifier' - required: true - type: string - payload: - description: 'JSON payload with work item details' - required: true - type: string - -tracker-id: campaign-worker-example - -tools: - github: - toolsets: [default] - -safe-outputs: - create-pull-request: - max: 1 - add-comment: - max: 2 ---- - -# Campaign Worker Example - -You are a campaign worker that processes work items from a campaign orchestrator. - -## Input Contract - -The `payload` input contains JSON with the following structure: -```json -{ - "repository": "owner/repo", - "work_item_id": "unique-identifier", - "target_ref": "main", - "additional_context": {} -} -``` - -Parse the payload and extract the work item details. - -## Idempotency Requirements - -Before creating any GitHub resources: - -1. **Generate deterministic key**: - - Format: `campaign-${campaign_id}-${repository}-${work_item_id}` - - Use this key in branch names, PR titles, issue titles - -2. **Check for existing work**: - - Search for PRs/issues with the deterministic key in the title - - Filter by tracker label: `campaign:${campaign_id}` - - If found: Skip creation or update existing item - - If not found: Proceed with creation - -3. **Label all created items**: - - Apply tracker label: `campaign:${campaign_id}` - - This enables discovery by the orchestrator - - Prevents interference from other workflows - -## Work to Perform - -[Specific task description for this worker] - -## Expected Output - -Report completion status including: -- Whether work was skipped (already exists) or completed -- Links to created/updated PRs or issues -- Any errors or blockers encountered -``` - -### Idempotency Implementation Patterns - -#### Pattern 1: Deterministic Branch Names - -```yaml -# In worker prompt -Generate a deterministic branch name: -- Format: `campaign-${campaign_id}-${repository.replace('/', '-')}-${work_item_id}` -- Example: `campaign-security-q1-2025-myorg-myrepo-alert-123` - -Before creating a new branch: -1. Check if the branch already exists -2. If exists: checkout and update -3. If not: create new branch -``` - -#### Pattern 2: PR Title Prefixing - -```yaml -# In worker prompt -Use a deterministic PR title prefix: -- Format: `[campaign-${campaign_id}] ${work_item_description}` -- Example: `[campaign-security-q1-2025] Fix SQL injection in user.go` - -Before creating a PR: -1. Search for open PRs with this title prefix in the target repo -2. If found: Add a comment with updates or close as duplicate -3. If not: Create new PR with title -``` - -#### Pattern 3: Issue Title Keying - -```yaml -# In worker prompt -Use a deterministic issue title with key: -- Format: `[${work_item_id}] ${description}` -- Example: `[alert-123] High severity: Path traversal vulnerability` - -Before creating an issue: -1. Search for issues with `[${work_item_id}]` in title -2. Filter by label: `z_campaign_${campaign_id}` -3. If found: Update existing issue with new information -4. If not: Create new issue -``` - -#### Pattern 4: Cursor-based Work Tracking - -```yaml -# In worker prompt -Track processed work items in repo-memory: -- File: `memory/campaigns/${campaign_id}/processed-items.json` -- Structure: `{"processed": ["item-1", "item-2", ...]}` - -Before processing a work item: -1. Load the processed items list from repo-memory -2. Check if current work_item_id is in the list -3. If found: Skip processing -4. If not: Process and add to list -5. Save updated list back to repo-memory -``` - -### Worker Discovery - -Campaign orchestrators discover worker-created items via: - -1. **Tracker Label**: Items labeled with `campaign:${campaign_id}` -2. **Tracker ID**: Items with `tracker-id: worker-name` in their description -3. **Discovery Script**: `campaign_discovery.cjs` searches for both - -Workers should: -- Apply the campaign tracker label to all created items -- Include the worker's tracker-id in issue/PR descriptions (optional) -- This enables orchestrators to find and track worker output - -### Example: Security Fix Worker - -```yaml ---- -name: Security Fix Worker -description: Creates PRs with security fixes for code scanning alerts - -on: - workflow_dispatch: - inputs: - campaign_id: - description: 'Campaign identifier' - required: true - type: string - payload: - description: 'JSON with alert details' - required: true - type: string - -tracker-id: security-fix-worker - -tools: - github: - toolsets: [default, code_security] - bash: ["*"] - edit: true - -safe-outputs: - create-pull-request: - max: 1 ---- - -# Security Fix Worker - -Process a code scanning alert and create a fix PR. - -## Idempotency Implementation - -```javascript -const payload = JSON.parse(process.env.PAYLOAD); -const campaignId = process.env.CAMPAIGN_ID; -const alertId = payload.alert_id; -const repository = payload.repository; - -// Deterministic key -const workKey = `campaign-${campaignId}-alert-${alertId}`; -const branchName = `fix/${workKey}`; -const prTitle = `[${workKey}] Fix: ${payload.alert_title}`; - -// Check for existing PR -const existingPRs = await searchPullRequests({ - query: `repo:${repository} is:pr is:open "${workKey}" in:title` -}); - -if (existingPRs.length > 0) { - console.log(`PR already exists: ${existingPRs[0].url}`); - // Optionally update with new information - return; -} - -// Proceed with fix and PR creation... -``` - -## Expected Behavior - -1. Parse payload to get alert details -2. Check for existing PR with deterministic key -3. If exists: Skip or update -4. If not: Generate fix and create PR -5. Apply labels: `campaign:${campaign_id}`, `security`, `automated` -6. Report completion status -``` - -## For Third-Party Users - -### Using gh-aw Compiler Outside This Repository - -**Yes, it works!** The campaign system is designed to work in any repository with gh-aw installed. - -#### Prerequisites - -```bash -# Install gh-aw CLI -gh extension install githubnext/gh-aw - -# Or use local binary -./gh-aw --help -``` - -#### Creating a Campaign - -1. **Create campaign spec** in your repository: - ```bash - mkdir -p .github/workflows - gh aw campaign new my-campaign - ``` - -2. **Edit the spec** (`.github/workflows/my-campaign.campaign.md`): - ```yaml - --- - id: my-campaign - name: My Campaign - version: v1 - project-url: https://github.com/orgs/ORG/projects/1 - tracker-label: z_campaign_my-campaign - workflows: - - my-worker-workflow - memory-paths: - - memory/campaigns/my-campaign/** - --- - - # Campaign description goes here - ``` - -3. **Compile the campaign**: - ```bash - gh aw compile - ``` - - This generates: - - `.github/workflows/my-campaign.campaign.g.md` (local debug artifact) - - `.github/workflows/my-campaign.campaign.lock.yml` (committed) - -4. **Commit and push**: - ```bash - git add .github/workflows/my-campaign.campaign.md - git add .github/workflows/my-campaign.campaign.lock.yml - git commit -m "Add my-campaign" - git push - ``` - -5. **Run the orchestrator** from GitHub Actions tab - -#### What Gets Executed - -When the orchestrator runs: - -1. **Setup Actions** - Copies JavaScript files to `/opt/gh-aw/actions/`: - - Source: `actions/setup/js/campaign_discovery.cjs` (from gh-aw repository) - - Runtime: `/opt/gh-aw/actions/campaign_discovery.cjs` - -2. **Discovery Step** - Executes discovery precomputation: - - Uses `actions/github-script@v8.0.0` - - Calls `require('/opt/gh-aw/actions/campaign_discovery.cjs')` - - Generates `./.gh-aw/campaign.discovery.json` - -3. **Agent Job** - AI agent processes the manifest: - - Reads `./.gh-aw/campaign.discovery.json` - - Updates GitHub Project board via safe-outputs - - Uses repo-memory for state persistence - -#### Required Files - -**In the gh-aw repository** (automatically included): -- `actions/setup/` - Setup action that copies JavaScript files -- `actions/setup/js/campaign_discovery.cjs` - Discovery script -- `actions/setup/js/setup_globals.cjs` - Global utilities - -**In your repository** (you create): -- `.github/workflows/.campaign.md` - Campaign spec -- `.github/workflows/.campaign.lock.yml` - Compiled workflow (generated) - -#### How the Compiler Finds Scripts - -The discovery script is **not** included in the compiled `.lock.yml` file. Instead: - -1. The compiled workflow includes an `actions/setup` step -2. `actions/setup` copies files from its repository to `/opt/gh-aw/actions/` -3. The discovery step uses `require('/opt/gh-aw/actions/campaign_discovery.cjs')` -4. This works because the path is available at runtime via the setup action - -**Key insight**: The setup action is a composite action that copies JavaScript files to a runtime location. This allows campaigns in any repository to use the discovery script without duplicating it. - -## Cross-References - -### Code References - -**Campaign package** (`pkg/campaign/`): -- `spec.go` - Data structures (CampaignSpec, CampaignKPI, CampaignGovernancePolicy) -- `loader.go` - Discovery and loading (LoadSpecs, FilterSpecs, CreateSpecSkeleton) -- `orchestrator.go` - Orchestrator generation (BuildOrchestrator, buildDiscoverySteps) -- `validation.go` - Spec validation (ValidateSpec) -- `command.go` - CLI commands (campaign, campaign status, campaign new, campaign validate) - -**CLI package** (`pkg/cli/`): -- `compile_workflow_processor.go` - Workflow processing (processCampaignSpec) -- `compile_orchestrator.go` - Orchestrator rendering (renderGeneratedCampaignOrchestratorMarkdown) -- `compile_helpers.go` - Utility functions - -**Actions** (`actions/setup/js/`): -- `campaign_discovery.cjs` - Discovery precomputation script -- `setup_globals.cjs` - Global utilities for GitHub Actions scripts - -### Key Workflows - -**Example campaigns** (in `.github/workflows/`): -- Look for `*.campaign.md` files in the repository root -- Compiled to `*.campaign.lock.yml` files - -## Design Decisions - -### Why Separate Discovery Step? - -**Problem**: AI agents performing GitHub-wide discovery during Phase 1 is: -- Non-deterministic (different results on each run) -- Expensive (many API calls) -- Slow (sequential search) - -**Solution**: Precomputation step that runs before the agent: -- Deterministic output (stable manifest) -- Enforced budgets (max items, max pages) -- Fast (parallel search possible) -- Cacheable (manifest can be reused) - -### Why `.campaign.g.md` is Not Committed - -**Rationale**: -- It's a generated artifact, not source code -- Users edit `.campaign.md`, not `.campaign.g.md` -- The `.lock.yml` file is the authoritative compiled output -- Keeping `.g.md` local aids debugging without cluttering git history - -**Benefits**: -- Cleaner git history -- No merge conflicts on generated files -- Users can regenerate anytime with `gh aw compile` -- `.lock.yml` provides reproducible execution - -### Why Cursor is in Repo-Memory - -**Rationale**: -- Campaigns need durable state across runs -- Git branches provide versioned, auditable history -- Repo-memory integrates with existing GitHub workflows - -**Alternatives considered**: -- Environment variables (lost between runs) -- Workflow artifacts (expire after 90 days) -- External database (requires additional infrastructure) - -### Why Campaign-Specific Lock File Naming - -**Problem**: Standard naming would produce: -``` -example.campaign.g.md → example.campaign.g.lock.yml -``` - -This is verbose and inconsistent with the spec file name. - -**Solution**: Special handling in `stringutil.CampaignOrchestratorToLockFile()`: -``` -example.campaign.g.md → example.campaign.lock.yml -``` - -This keeps lock files aligned with spec files: -``` -example.campaign.md (spec) -example.campaign.lock.yml (compiled) -``` - -## Debugging - -### Enable Debug Logging - -```bash -DEBUG=campaign:*,cli:* gh aw compile -``` - -### Check Generated Orchestrator - -```bash -# Review local debug artifact -cat .github/workflows/.campaign.g.md - -# Review compiled workflow -cat .github/workflows/.campaign.lock.yml -``` - -### Inspect Discovery Manifest - -After running the orchestrator: - -```bash -# Download workflow artifacts -gh run download - -# Check discovery manifest -cat .gh-aw/campaign.discovery.json -``` - -### Validate Campaign Spec - -```bash -gh aw campaign validate -gh aw campaign validate my-campaign -gh aw campaign validate --json -``` - -## Future Enhancements - -### Planned Improvements - -1. **Multi-repository discovery**: Search across organization repositories -2. **Advanced filtering**: Filter items by milestone, assignee, or custom fields -3. **Discovery caching**: Cache discovery results to reduce API calls -4. **Incremental updates**: Only update changed items in project board -5. **Workflow templates**: Pre-built campaign templates for common scenarios - -### Extension Points - -1. **Custom discovery scripts**: Allow campaigns to provide custom discovery logic -2. **Discovery plugins**: Plugin system for discovery sources (Jira, Linear, etc.) -3. **Campaign hierarchies**: Parent/child campaigns with rollup metrics -4. **Cross-campaign dependencies**: Express dependencies between campaigns - ---- - -**Last Updated**: 2025-01-08 - -**Related Issues**: #1234 (Campaign Architecture), #5678 (Discovery Optimization)