Skip to content

feat: port smoke-codex workflow from gh-aw#557

Merged
Mossaka merged 4 commits intomainfrom
copilot/port-smoke-codex
Feb 6, 2026
Merged

feat: port smoke-codex workflow from gh-aw#557
Mossaka merged 4 commits intomainfrom
copilot/port-smoke-codex

Conversation

Copy link
Contributor

Copilot AI commented Feb 6, 2026

Ports the smoke-codex.md workflow from gh-aw to validate Codex engine functionality with AWF firewall capabilities.

Changes

New Workflows

  • smoke-codex.md/lock.yml - Validates Codex engine through 8 test scenarios: GitHub MCP, GH CLI safe-inputs, Playwright, Tavily web search, file operations, bash tools, discussion interactions, and AWF build verification
  • Runs every 12h on schedule, manual dispatch, and PR label smoke

Shared Dependencies

Created 4 shared workflow files required by smoke-codex:

  • shared/gh.md - Authenticated GitHub CLI safe-input tool
  • shared/mcp/tavily.md - Tavily MCP server config
  • shared/reporting.md - Report formatting guidelines
  • shared/github-queries-safe-input.md - GitHub queries (issues/PRs/discussions) with jq filtering

Adaptations for gh-aw-firewall

  • Removed Serena MCP tests (Go language server not applicable to this Node.js project)
  • Changed build test from GOCACHE=/tmp/go-cache make build to npm ci && npm run build
  • Updated Tavily search query to "GitHub Agentic Workflows Firewall"

Maintenance

  • Updated postprocess-smoke-workflows.ts to include smoke-codex.lock.yml - ensures CI tests local builds instead of published binaries

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 2 commits February 6, 2026 22:36
Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Co-authored-by: Mossaka <5447827+Mossaka@users.noreply.github.com>
Copilot AI changed the title [WIP] Port smoke codex workflow to new repository feat: port smoke-codex workflow from gh-aw Feb 6, 2026
Copilot AI requested a review from Mossaka February 6, 2026 22:38
@Mossaka Mossaka marked this pull request as ready for review February 6, 2026 23:03
Copilot AI review requested due to automatic review settings February 6, 2026 23:03
@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

💫 TO BE CONTINUED... Smoke Claude failed! Our hero faces unexpected challenges...

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

📰 DEVELOPING STORY: Smoke Copilot reports failed. Our correspondents are investigating the incident...

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 82.05% 82.05% ➡️ +0.00%
Statements 82.09% 82.09% ➡️ +0.00%
Functions 81.95% 81.95% ➡️ +0.00%
Branches 75.54% 75.54% ➡️ +0.00%

Coverage comparison generated by scripts/ci/compare-coverage.ts

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ports the smoke-codex gh-aw workflow into this repo to validate the Codex engine under AWF firewall constraints, and updates the repo’s generated gh-aw workflow lockfiles/scripts to include it.

Changes:

  • Adds a new Smoke Codex workflow plus shared workflow imports needed for Codex smoke scenarios (GitHub CLI safe-input, Tavily MCP config, reporting guidance, GitHub query safe-inputs).
  • Updates scripts/ci/postprocess-smoke-workflows.ts to include smoke-codex.lock.yml in postprocessing.
  • Regenerates multiple *.lock.yml workflows (gh-aw v0.42.10, updated AWF/agent versions, MCP gateway payload dir support, sparse checkout adjustments, etc.).

Reviewed changes

Copilot reviewed 31 out of 31 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
scripts/ci/postprocess-smoke-workflows.ts Adds smoke-codex.lock.yml to the set of workflows postprocessed in CI.
.github/workflows/smoke-codex.md New Codex smoke test definition covering MCP/GH CLI/Playwright/Tavily/files/bash/discussion/build checks.
.github/workflows/shared/gh.md New shared safe-input wrapper for authenticated gh CLI usage.
.github/workflows/shared/mcp/tavily.md New shared Tavily MCP server configuration.
.github/workflows/shared/reporting.md New shared guidelines for consistent report formatting.
.github/workflows/update-release-notes.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/test-coverage-improver.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/smoke-claude.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/smoke-chroot.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/security-review.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/security-guard.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/pelis-agent-factory-advisor.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/issue-monster.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/issue-duplication-detector.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/doc-maintainer.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/dependency-security-monitor.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/cli-flag-consistency-checker.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/ci-doctor.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/ci-cd-gaps-assessment.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/build-test-rust.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/build-test-node.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/build-test-go.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/build-test-cpp.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/build-test-bun.lock.yml Regenerated gh-aw lock workflow with updated versions and MCP gateway payload dir support.
.github/workflows/agentics-maintenance.yml Updates the generated maintenance workflow version metadata and schedule frequency.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +13 to +16
run: |
echo "gh $INPUT_ARGS"
echo " token: ${GH_AW_GH_TOKEN:0:6}..."
GH_TOKEN="$GH_AW_GH_TOKEN" gh $INPUT_ARGS
Copy link

Copilot AI Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The safe-input implementation is vulnerable to shell injection because $INPUT_ARGS is expanded unquoted in the gh invocation. An agent could pass characters like ;, &&, or command substitutions to execute arbitrary commands outside the intended gh CLI call. Consider changing the input to a JSON array (or otherwise safely parsing into an argv array) and invoking gh with an argument array (gh "${ARGS[@]}") so the input cannot be interpreted by the shell.

Copilot uses AI. Check for mistakes.
@Mossaka Mossaka added the smoke label Feb 6, 2026
@Mossaka Mossaka force-pushed the copilot/port-smoke-codex branch from d1b8565 to 03e744b Compare February 6, 2026 23:27
@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

📰 DEVELOPING STORY: Smoke Copilot reports failed. Our correspondents are investigating the incident...

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

💫 TO BE CONTINUED... Smoke Claude failed! Our hero faces unexpected challenges...

The lock files were manually edited by Copilot instead of being
generated through the proper gh-aw compile + postprocess pipeline.
This caused all agent jobs to fail with TS18003 because sparse-checkout
only fetched .github/.agents folders, missing src/ and package.json.

Recompiled with gh-aw v0.42.11 and ran postprocess-smoke-workflows.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Mossaka Mossaka force-pushed the copilot/port-smoke-codex branch from 03e744b to b5bc8f1 Compare February 6, 2026 23:30
@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

🎬 THE ENDSmoke Claude MISSION: ACCOMPLISHED! The hero saves the day! ✨

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

C++ Build Test Results

Project CMake Build Status
fmt PASS
json PASS

Overall: PASS

All C++ projects built successfully.

AI generated by Build Test C++

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Smoke Test Results 🧪

Last 2 merged PRs:

✅ GitHub MCP: Retrieved last 2 merged PRs
✅ Playwright: Navigated to GitHub, title verified
✅ File Writing: Created test file successfully
✅ Bash Tool: File read back verified

Overall: PASS

cc @Mossaka @Copilot

AI generated by Smoke Copilot

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Build Test: Rust - FAILED ❌

Error: Rust toolchain not available in the environment.

Project Build Tests Status
fd - FAILED
zoxide - FAILED

Overall: FAIL

Error Details:

  • cargo: command not found
  • The GitHub Actions runner does not have Rust (cargo/rustc) installed
  • The workflow needs to install Rust before running these tests

Resolution: Add Rust installation to the workflow:

- name: Install Rust
  run: |
    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
    echo "$HOME/.cargo/bin" >> $GITHUB_PATH

AI generated by Build Test Rust

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Node.js Build Test Results

Project Install Tests Status
clsx PASS PASS
execa PASS PASS
p-limit PASS PASS

Overall: PASS

All Node.js projects built and tested successfully.

AI generated by Build Test Node.js

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

✅ Deno Build Test Results

Project Tests Status
oak 1/1 ✅ PASS
std 1/1 ✅ PASS

Overall: ✅ PASS

All Deno tests completed successfully.

AI generated by Build Test Deno

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

❌ Java Build Test Failed

Status: FAILED - Maven Unavailable

Error

Maven installation on the GitHub Actions runner is corrupted:

/usr/share/apache-maven-3.9.12/boot/plexus-classworlds-2.9.0.jar: cannot execute binary file: Exec format error

Test Results

Project Compile Tests Status
gson FAILED - Maven unavailable
caffeine FAILED - Maven unavailable

Overall: FAIL

Resolution Required

  • Fix Maven installation on GitHub Actions runner, or
  • Add Maven wrapper (mvnw) to test projects in Mossaka/gh-aw-firewall-test-java

AI generated by Build Test Java

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Go Build Test Results

Project Download Tests Status
color 1/1 PASS
env 1/1 PASS
uuid 1/1 PASS

Overall: PASS

All Go projects successfully downloaded dependencies and passed tests.

AI generated by Build Test Go

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Build Test: Bun - FAILED ❌

Test Results

Project Install Tests Status
elysia FAIL
hono FAIL

Overall: FAIL

Error Details

Both projects encountered critical Bun errors:

Install Error:

error: An internal error occurred (NotDir)
```

**Test Error:**
```
bun test v1.3.8 (b64edcb4)
Aborted (core dumped)

Environment

  • Bun version: 1.3.8
  • Test repository: Mossaka/gh-aw-firewall-test-bun

Analysis

Bun is experiencing internal errors that prevent both dependency installation and test execution. This appears to be an environment compatibility issue with the current GitHub Actions runner or Bun version 1.3.8.

AI generated by Build Test Bun

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2026

Smoke Test Results - Claude Engine

Last 2 Merged PRs:

Test Results:

  • ✅ GitHub MCP (fetched PR data)
  • ✅ Playwright (navigated to github.com, verified title)
  • ✅ File Writing (created test file)
  • ✅ Bash Tool (verified file contents)

Overall: PASS

AI generated by Smoke Claude

@Mossaka Mossaka merged commit 61432e3 into main Feb 6, 2026
80 checks passed
@Mossaka Mossaka deleted the copilot/port-smoke-codex branch February 6, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants