[CI Failure Doctor] Test job failing due to TestCollectDockerImages expecting no api-proxy

[github-actions]

🏥 CI Failure Investigation - Run #35708

Summary

• The test job in the CI workflow fails because TestCollectDockerImages_APIProxyForEnginesWithLLMGateway panics during initialization for the Claude, Copilot, and Codex subtests.

Failure Details

• Run: 35708
• Commit: bb8c9da
• Trigger: push

Root Cause Analysis

• collectDockerImages now interrogates engine.SupportsLLMGateway() and adds the AWF api-proxy image whenever the returned port is greater than 0.
• ClaudeEngine, CodexEngine, and the new CopilotSDKEngine all override SupportsLLMGateway() to return actual port numbers (10000, 10001, and 10002, respectively), so a firewall-enabled workflow with any of those engines should pull in the proxy image.
• The regression test still asserts the legacy expectation that the proxy image is absent for Claude, Copilot, and Codex. As soon as the new logic registers these engines with positive LLM gateway ports, the test setup diverges from reality and Go reports a package-level failure with “Test initialization panics.”

Failed Jobs and Errors

• test
  • Fails in pkg/workflow/docker_api_proxy_test.go:TestCollectDockerImages_APIProxyForEnginesWithLLMGateway with the Go runner indicating “Test initialization panics” after the package reports a panic during setup (no individual test failure is emitted because the harness terminates early).

Investigation Findings

• collectDockerImages adds the proxy image via constants.DefaultFirewallRegistry + "/api-proxy:" + awfImageTag when SupportsLLMGateway() > 0.
• All three engines under test now return a positive port, so a firewall-enabled workflow pulls in the proxy image as intended.
• The test still looks for the absence of the proxy image and thus never reaches its Squid/Agent assertions; the go test harness collapses the package with an initialization panic.

Recommended Actions

• Update TestCollectDockerImages_APIProxyForEnginesWithLLMGateway (and any other tests that assumed boolean LLM gateway support) to derive the expected presence of the proxy image from SupportsLLMGateway() rather than hard-coding engine names.
  • For engines with SupportsLLMGateway() > 0, assert that the proxy image is returned.
  • Add a subtest that ensures engines returning -1 continue to skip the proxy image.
• Once the test reflects the new semantics, rerun go test ./pkg/workflow -run TestCollectDockerImages_APIProxyForEnginesWithLLMGateway locally (note: running tests currently requires downloading Go 1.25, which is blocked in this environment).

Prevention Strategies

• When changing the contract of capability helpers like SupportsLLMGateway(), immediately update any regression tests that embed the previous behavior.
• Consider adding golden data derived from GetGlobalEngineRegistry() so tests automatically keep pace with engine metadata.

AI Team Self-Improvement

• "After changing an Engine capability API, regenerate or revalidate the tests that assert capability-specific successes/failures, instead of assuming the previous behavior still applies."

Historical Context

• This failure surfaced in commit bb8c9da4da014da05cecdea2e148eb596b50b7b2 while adding the Copilot SDK engine and port-based LLM gateway handling. No existing [CI Failure Doctor] issue covers the same test failure (unable to search for duplicates because the workflow environment cannot access the GitHub API).

🩺 Diagnosis provided by CI Failure Doctor

To install this workflow, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. View source at https://github.com/githubnext/agentics/tree/ea350161ad5dcc9624cf510f134c6a9e39a6f94d/workflows/ci-doctor.md.

• expires on Feb 15, 2026, 5:27 PM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #15732

View newer issue

---

This action was performed automatically by the CI Failure Doctor workflow.