Add safe output items manifest for audit trail by Copilot · Pull Request #16842 · github/gh-aw

Copilot · 2026-02-19T14:28:01Z

Original prompt

Update the safe output JavaScript handlers such that every create item in GitHub is logged in a JSONL manifest file and uploaded as an artifact (/tmp/safe-output-items.jsonl) . Uploads always.

The goal of this artifact is to be able to audit the trail of AW and generate item reliably.

Also record the temporary id if any.

One entry per generated item.

Create a JavaScript helper function that can be used by the safe output handlers. Make it easily testable.

Update audit command to download this artifact and display a table with URLs to the user.

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Changeset

Type: patch
Description: Log safe output creations to the new JSONL manifest artifact and surface the created-items table during audit report generation

Generated by Changeset Generator

✨ PR Review Safe Output Test - Run 22187312231

💥 [THE END] — Illustrated by Smoke Claude

Changeset

Type: patch
Description: Log safe output creations to the new manifest artifact and surface the created-items table during audit reports

Generated by Changeset Generator

✨ PR Review Safe Output Test - Run 22190175158

💥 [THE END] — Illustrated by Smoke Claude

Changeset

Type: patch
Description: Log created safe output items to the manifest artifact and surface the created-items table in gh aw audit output.

Generated by Changeset Generator

✨ PR Review Safe Output Test - Run 22190930520

💥 [THE END] — Illustrated by Smoke Claude

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan

@copilot handle staged mode

.github/workflows/ai-moderator.lock.yml

pelikhan · 2026-02-19T15:02:37Z

@copilot logs command should show aggregate stats in result like number of GitHub items created by safe outputs (safe items)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-02-19T15:05:31Z

@copilot handle staged mode

Done in 79656df. In staged mode (GH_AW_SAFE_OUTPUTS_STAGED === "true"): the manifest logger is not created, ensureManifestExists() is not called in any code path, and extractCreatedItemFromResult explicitly rejects results with staged: true.

github-actions · 2026-02-19T15:07:14Z

📰 BREAKING: Smoke macOS ARM64 is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-02-19T15:07:15Z

📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-02-19T15:07:18Z

🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

github-actions · 2026-02-19T15:07:18Z

🧪 Smoke Project is now testing project operations...

github-actions · 2026-02-19T15:07:19Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-02-19T15:07:19Z

🧪 Smoke Temporary ID is now testing temporary ID functionality...

github-actions · 2026-02-19T15:07:50Z

📰 DEVELOPING STORY: Smoke macOS ARM64 reports failed. Our correspondents are investigating the incident...

Copilot

Pull request overview

Adds an auditable trail of GitHub items created by safe output handlers by logging each created item into a JSONL manifest, uploading it as an artifact, and surfacing it in gh aw audit as a “Created Items” table (and in --json output).

Changes:

Introduces a JS manifest helper (safe_output_manifest.cjs) plus unit tests to append created-item entries to /tmp/safe-output-items.jsonl.
Extends the unified safe output handler manager to invoke an onItemCreated callback and ensure the manifest file exists for upload.
Updates the safe outputs job compiler and workflow lock files to always upload the manifest artifact, and updates the audit command to parse/render created items.

Reviewed changes

Copilot reviewed 152 out of 152 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`actions/setup/js/safe_output_manifest.cjs`	New helper to write created-item JSONL entries and normalize handler results into manifest entries.
`actions/setup/js/safe_output_manifest.test.cjs`	Adds Vitest coverage for manifest logger, ensure-file behavior, and result extraction.
`actions/setup/js/safe_output_unified_handler_manager.cjs`	Adds optional `onItemCreated` callback plumbing and ensures the manifest exists on early-return paths.
`pkg/workflow/compiler_safe_outputs_job.go`	Appends an `if: always()` upload-artifact step for the manifest in the consolidated safe outputs job.
`pkg/cli/audit_report.go`	Adds `CreatedItems` to `AuditData` and parses `safe-output-items.jsonl` from the run’s downloaded artifacts directory.
`pkg/cli/audit_report_render.go`	Renders a new “Created Items” section/table when created items are present.
`.github/workflows/audit-workflows.lock.yml`	Adds the manifest upload step to the compiled workflow.
`.github/workflows/artifacts-summary.lock.yml`	Adds the manifest upload step to the compiled workflow.
`.github/workflows/archie.lock.yml`	Adds the manifest upload step to the compiled workflow.
`.github/workflows/ai-moderator.lock.yml`	Adds the manifest upload step to the compiled workflow.
`.github/workflows/agent-persona-explorer.lock.yml`	Adds the manifest upload step to the compiled workflow.
`.github/workflows/agent-performance-analyzer.lock.yml`	Adds the manifest upload step to the compiled workflow.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pkg/cli/audit_report.go

actions/setup/js/safe_output_manifest.test.cjs

pkg/cli/audit_report.go

Copilot · 2026-02-19T15:10:43Z

actions/setup/js/safe_output_unified_handler_manager.cjs

      core.info(`Exported ${issuesToAssignCopilot.length} issue(s) for copilot assignment: ${issuesToAssignStr}`);
    } else {
      core.setOutput("issues_to_assign_copilot", "");
    }

+    // Ensure the manifest file always exists for artifact upload (even if no items were created).
+    // Skip in staged mode — no real items were created so no manifest should be emitted.


If main() throws before the normal completion path, the manifest may never be created, so the subsequent actions/upload-artifact step won’t have /tmp/safe-output-items.jsonl to upload (despite the goal of “uploads always”). Consider calling ensureManifestExists() in a finally block (or in the catch block, wrapped so it can’t mask the original failure) to guarantee the file exists even on handler-manager errors.

github-actions · 2026-02-19T16:41:50Z

🧪 Smoke Temporary ID is now testing temporary ID functionality...

github-actions · 2026-02-19T16:41:50Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-02-19T16:41:51Z

📰 BREAKING: Smoke macOS ARM64 is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-02-19T16:41:51Z

📰 BREAKING: Smoke Copilot is now investigating this pull request. Sources say the story is developing...

github-actions · 2026-02-19T16:41:53Z

🎬 THE END — Smoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

github-actions · 2026-02-19T16:41:53Z

🧪 Smoke Project is now testing project operations...

github-actions · 2026-02-19T16:42:25Z

📰 DEVELOPING STORY: Smoke macOS ARM64 reports failed. Our correspondents are investigating the incident...

github-actions · 2026-02-19T16:45:23Z

✅ Smoke Project completed successfully. All project operations validated.

github-actions · 2026-02-19T16:46:20Z

Smoke test summary
Merged PR titles: [slides] Update slide deck - remove experimental labels, fix MCP toolsets; docs: add run failure issues and no-op report sections to monitoring guide
GitHub MCP: ✅
Serena MCP: ✅
Playwright: ✅
File write + bash cat: ✅
Build gh-aw: ✅
Overall: PASS

🔮 The oracle has spoken through Smoke Codex for issue #16842

github-actions · 2026-02-19T16:47:31Z

Smoke test results for @pelikhan:

Test	Status
GitHub MCP	✅
Safe Inputs GH CLI	✅
Playwright	✅
File Writing + Bash	✅
Discussion Interaction	✅
Build gh-aw	✅
Discussion Creation	✅
Workflow Dispatch	✅
PR Review	✅

Overall: PASS ✅

📰 BREAKING: Report filed by Smoke Copilot for issue #16842

github-actions

Good PR — the safe output manifest adds useful audit trail capability. Two minor observations about logging consistency in the manifest integration. Overall looks solid.

📰 BREAKING: Report filed by Smoke Copilot for issue #16842

github-actions · 2026-02-19T16:47:33Z

actions/setup/js/safe_output_handler_manager.cjs

 * @returns {Promise<{success: boolean, results: Array<any>, temporaryIdMap: Object, outputsWithUnresolvedIds: Array<any>, missings: Object}>}
 */
-async function processMessages(messageHandlers, messages) {
+async function processMessages(messageHandlers, messages, onItemCreated = null) {


The onItemCreated callback follows a clean optional pattern with = null default. Consider documenting in the JSDoc that callers can use createManifestLogger() from safe_output_manifest.cjs as the standard implementation for this parameter.

github-actions · 2026-02-19T16:47:33Z

actions/setup/js/safe_output_handler_manager.cjs

+          const createdItem = extractCreatedItemFromResult(messageType, result);
+          if (createdItem) {
+            core.info(`📝 Manifest: logged ${createdItem.type} → ${createdItem.url}`);
+            onItemCreated(createdItem);


The array branch (lines 377-384) logs core.info for each item, but the else branch (lines 385-390) doesn't log. This inconsistency might make debugging harder — consider adding the core.info log in the single-item path too for consistency with the array path.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2026-02-19T16:48:09Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

Copilot · 2026-02-19T16:48:11Z

@copilot do not generate teh actions/upload-artifact step in staged mode

Done in 72721a8. The buildSafeOutputItemsManifestUploadStep() call in compiler_safe_outputs_job.go is now guarded by !isStaged (where isStaged = c.trialMode || data.SafeOutputs.Staged). Recompiled all 152 lock files — staged workflows like poem-bot and daily-choice-test no longer contain the upload-artifact step.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/graphql
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/cli/access_log.go /home/REDACTED/work/gh-aw/gh-aw/pkg/cli/actionlint.go (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/11bd71901bbe5b1630ceea73d27597364c9af683
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/11bd71901bbe5b1630ceea73d27597364c9af683 --jq .object.sha (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v7
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v7 --jq .object.sha (http block)
https://api.github.com/repos/actions/setup-go/git/ref/tags/4dc6199c7b1a012772edbd06daecab0f50c9053c
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/4dc6199c7b1a012772edbd06daecab0f50c9053c --jq .object.sha (http block)
https://api.github.com/repos/actions/setup-go/git/ref/tags/v5
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v5 --jq .object.sha (http block)
https://api.github.com/repos/actions/setup-go/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v6 --jq .object.sha (http block)
https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq .object.sha (http block)
https://api.github.com/repos/anchore/sbom-action/git/ref/tags/v0
- Triggering command: /usr/bin/gh gh api /repos/anchore/sbom-action/git/ref/tags/v0 --jq .object.sha (http block)
https://api.github.com/repos/docker/build-push-action/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/docker/build-push-action/git/ref/tags/v6 --jq .object.sha (http block)
https://api.github.com/repos/docker/login-action/git/ref/tags/v3
- Triggering command: /usr/bin/gh gh api /repos/docker/login-action/git/ref/tags/v3 --jq .object.sha (http block)
https://api.github.com/repos/docker/metadata-action/git/ref/tags/v5
- Triggering command: /usr/bin/gh gh api /repos/docker/metadata-action/git/ref/tags/v5 --jq .object.sha (http block)
https://api.github.com/repos/docker/setup-buildx-action/git/ref/tags/v3
- Triggering command: /usr/bin/gh gh api /repos/docker/setup-buildx-action/git/ref/tags/v3 --jq .object.sha (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/a70c5eada06553e3510ac27f2c3bda9d3705bccb
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/a70c5eada06553e3510ac27f2c3bda9d3705bccb --jq .object.sha (http block)
https://api.github.com/repos/githubnext/agentics/git/ref/tags/-
- Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/- --jq .object.sha (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

github-actions · 2026-02-19T16:51:15Z

Smoke Test Results – Run §22190930520

Core Tests #1–10: ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
PR Review Tests #11–17: ✅ ✅ ✅ ✅ ✅ ✅ ⚠️

Overall: PARTIAL (test #17 skipped – no safe test PR to close)

💥 [THE END] — Illustrated by Smoke Claude for issue #16842

github-actions

💥 Automated smoke test review - all systems nominal!

💥 [THE END] — Illustrated by Smoke Claude for issue #16842

Initial plan

2be425e

Copilot AI assigned Copilot and pelikhan Feb 19, 2026

Copilot started work on behalf of pelikhan February 19, 2026 14:28 View session

Add safe output items manifest for audit trail

e30ade4

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Update safe output handlers to log created items in JSONL~~ Add safe output items manifest for audit trail Feb 19, 2026

Copilot finished work on behalf of pelikhan February 19, 2026 14:55

pelikhan reviewed Feb 19, 2026

View reviewed changes

.github/workflows/ai-moderator.lock.yml Show resolved Hide resolved

Copilot started work on behalf of pelikhan February 19, 2026 14:57 View session

pelikhan marked this pull request as ready for review February 19, 2026 15:03

Copilot AI review requested due to automatic review settings February 19, 2026 15:03

Copilot started reviewing on behalf of pelikhan February 19, 2026 15:04 View session

Skip manifest emission in staged mode

79656df

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan February 19, 2026 15:06

Copilot started work on behalf of pelikhan February 19, 2026 15:06 View session

pelikhan added the smoke label Feb 19, 2026

This was referenced Feb 19, 2026

[smoke-temporary-id] Test Parent: Temporary ID Validation #16852

Closed

[smoke-temporary-id] Sub-Issue 1: Test Temporary ID References #16853

Closed

[smoke-temporary-id] Sub-Issue 2: Test Different ID Length #16854

Closed

Add changeset [skip-ci]

47d0e71

Copilot AI reviewed Feb 19, 2026

View reviewed changes

Copilot started work on behalf of pelikhan February 19, 2026 16:40 View session

pelikhan added the smoke label Feb 19, 2026

Add changeset [skip-ci]

6c32f8d

This was referenced Feb 19, 2026

[smoke-temporary-id] Test Parent: Temporary ID Validation #16874

Closed

[smoke-temporary-id] Sub-Issue 1: Test Temporary ID References #16875

Closed

[smoke-temporary-id] Sub-Issue 2: Test Different ID Length #16876

Closed

github-actions bot removed the smoke label Feb 19, 2026

github-actions bot mentioned this pull request Feb 19, 2026

Smoke Test: Copilot - 22190930490 #16878

Closed

github-actions bot reviewed Feb 19, 2026

View reviewed changes

Skip upload-artifact step in staged mode (no items are created)

72721a8

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Merge branch 'main' into copilot/update-safe-output-logging

6093bee

Copilot finished work on behalf of pelikhan February 19, 2026 16:48

test: Add smoke test file for run 22190930520

f2961d2

github-actions bot mentioned this pull request Feb 19, 2026

Smoke Test: Claude - 22190930520 #16879

Closed

github-actions bot reviewed Feb 19, 2026

View reviewed changes

pelikhan merged commit 5a79466 into main Feb 19, 2026

pelikhan deleted the copilot/update-safe-output-logging branch February 19, 2026 16:53

Conversation

Copilot AI commented Feb 19, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changeset

Changeset

Changeset

Uh oh!

pelikhan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pelikhan commented Feb 19, 2026

Uh oh!

Copilot AI commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 19, 2026

Uh oh!

Copilot AI commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

github-actions bot commented Feb 19, 2026

Copilot AI commented Feb 19, 2026 •

edited by github-actions bot

Loading

github-actions bot commented Feb 19, 2026 •

edited

Loading

github-actions bot commented Feb 19, 2026 •

edited

Loading

github-actions bot commented Feb 19, 2026 •

edited

Loading

github-actions bot commented Feb 19, 2026 •

edited

Loading

Copilot AI commented Feb 19, 2026 •

edited

Loading