Skip to content

Fix heredoc tests and production code to use GenerateHeredocDelimiter#14993

Merged
pelikhan merged 4 commits intomainfrom
copilot/fix-heredoc-prompt-eof
Feb 11, 2026
Merged

Fix heredoc tests and production code to use GenerateHeredocDelimiter#14993
pelikhan merged 4 commits intomainfrom
copilot/fix-heredoc-prompt-eof

Conversation

Copy link
Contributor

Copilot AI commented Feb 11, 2026

Fixes CI test failures caused by hardcoded PROMPT_EOF string literals in tests and production code after the template injection validator introduced prefixed heredoc delimiters (GH_AW_PROMPT_EOF).

Changes Made

  • Test Files Updated: Modified sh_test.go (5 test functions), unified_prompt_creation_test.go, xml_comments_test.go, secure_markdown_rendering_test.go, and template_expression_integration_test.go to use GenerateHeredocDelimiter("PROMPT") instead of hardcoded "PROMPT_EOF" string literals
  • Production Code Updated: Fixed unified_prompt_step.go to use GenerateHeredocDelimiter("PROMPT") in both generateUnifiedPromptStep and generateUnifiedPromptCreationStep functions (13 locations total)
  • Workflow Lock Files: All 151 workflow .lock.yml files automatically recompiled with the new GH_AW_PROMPT_EOF delimiter format

Root Cause

After commit 10699a3 introduced prefixed heredoc delimiters via GenerateHeredocDelimiter, tests continued asserting on the old unprefixed PROMPT_EOF format. This caused TestWritePromptTextToYAML_EmptyText and other tests to fail because they couldn't find the expected delimiter strings.

Testing

  • ✅ All heredoc-related unit tests passing
  • TestWritePromptTextToYAML* tests (5 functions) - all passing
  • TestUnifiedPromptCreation* tests - all passing
  • TestXMLComments, TestSecureMarkdownRendering, TestTemplateExpression tests - all passing
  • ✅ Memory stored to prevent future hardcoded delimiter issues

Prevention

Tests now dynamically derive delimiter strings from GenerateHeredocDelimiter("PROMPT"), ensuring they automatically stay synchronized with any future delimiter format changes.

Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] Heredoc tests expect PROMPT_EOF instead of prefixed delimiter</issue_title>
<issue_description># 🏥 CI Failure Investigation - Run github/gh-aw#34894

Summary

The test job now fails because the heredoc helpers transitioned to GenerateHeredocDelimiter("PROMPT") (which returns GH_AW_PROMPT_EOF), but the unit tests still asserted on the literal PROMPT_EOF markers.

Failure Details

Root Cause Analysis

WritePromptTextToYAML builds heredocs using GenerateHeredocDelimiter, so the generated shell steps look for GH_AW_PROMPT_EOF. The empty-text test (TestWritePromptTextToYAML_EmptyText) still searches for cat << 'PROMPT_EOF' and PROMPT_EOF, so it never finds any heredoc blocks and fails during initialization, making the entire github.com/github/gh-aw/pkg/workflow package fail.

Failed Jobs and Errors

  • test: TestWritePromptTextToYAML_EmptyText reports "Expected at least 1 heredoc block even for empty text" because the strings it looks for no longer exist.

Investigation Findings

  • Log output shows the package-level failure and the helper panic message. There are no logical errors in the helper; the mismatch is purely in the string literal being asserted in the test.
  • The failure appeared right after the template injection validator change introduced prefixed delimiters, so the old literal no longer matches the runtime behavior.

Recommended Actions

  • Update the heredoc expectations in pkg/workflow/sh_test.go to build the expected command/marker strings from GenerateHeredocDelimiter("PROMPT") so the tests stay in sync with any future delimiter changes.
  • Re-run go test ./pkg/workflow (under go 1.25) to validate that no other assertions rely on the old literal.

Prevention Strategies

Always derive heredoc markers from GenerateHeredocDelimiter (or a thin helper) when asserting on shell steps, and avoid hard-coding PROMPT_EOF so tests continue working whenever the delimiter format evolves.

AI Team Self-Improvement

Add a reminder to instructions.md such as: "When checking generated heredocs, use GenerateHeredocDelimiter instead of hard-coded PROMPT_EOF so tests follow any prefix changes."

Historical Context

No prior CI Failure Doctor investigation has captured this exact issue; the failure started with the latest prefixed delimiter support.

AI generated by CI Failure Doctor

To add this workflow in your repository, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. See usage guide.

  • expires on Feb 12, 2026, 5:34 PM UTC

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits February 11, 2026 17:41
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

@copilot Keep working

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 11, 2026

@copilot Keep working

Done! All test files have been updated to use GenerateHeredocDelimiter("PROMPT") instead of hardcoded PROMPT_EOF. Changes committed in c76ac05.

Fixed files:

  • sh_test.go - All 5 test functions updated
  • unified_prompt_creation_test.go - 2 locations updated
  • unified_prompt_step.go - Production code updated (13 locations)
  • xml_comments_test.go - Chunking test updated
  • secure_markdown_rendering_test.go - 3 locations updated
  • template_expression_integration_test.go - 2 locations updated

All heredoc-related tests now pass. The TestSafeInputsHTTPMode failure is pre-existing (confirmed in base commit 10699a3) and unrelated to these changes.

Copilot AI changed the title [WIP] Fix heredoc tests to expect GH_AW_PROMPT_EOF Fix heredoc tests and production code to use GenerateHeredocDelimiter Feb 11, 2026
@pelikhan pelikhan marked this pull request as ready for review February 11, 2026 18:31
Copilot AI review requested due to automatic review settings February 11, 2026 18:32
@pelikhan pelikhan merged commit d87eb66 into main Feb 11, 2026
132 of 133 checks passed
@pelikhan pelikhan deleted the copilot/fix-heredoc-prompt-eof branch February 11, 2026 18:32
@github-actions
Copy link
Contributor

🔍 PR Triage Results

Category: bug | Risk: medium | Priority: 72/100

Scores Breakdown

  • Impact: 35/50 - CI test failures blocking development, affects 151 workflow lock files and core delimiter generation logic
  • Urgency: 25/30 - Fixes CI failures blocking other work (+5), affects multiple workflows, recent creation (6 hours old) (+5), important for maintaining CI health (+15)
  • Quality: 12/20 - Good PR description with clear root cause analysis (+5), includes testing section (+3), but CI still pending (+4)

📋 Recommended Action: fast_track

Critical CI fix affecting 30 files including test infrastructure and all workflow lock files. Well-documented with clear root cause. Needs fast-track review to unblock CI.


Triaged by PR Triage Agent on 2026-02-11T18:30:00Z - Run #21917749368

AI generated by PR Triage Agent

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates heredoc delimiter usage across tests and compiled workflow lockfiles to align with the newer GenerateHeredocDelimiter("PROMPT") output (e.g., prefixed GH_AW_PROMPT_EOF), fixing CI failures caused by stale hardcoded PROMPT_EOF literals.

Changes:

  • Replaced hardcoded heredoc delimiter literals in multiple Go tests with GenerateHeredocDelimiter("PROMPT").
  • Updated compiled workflow .lock.yml files to use the prefixed heredoc delimiter in generated shell steps.

Reviewed changes

Copilot reviewed 154 out of 154 changed files in this pull request and generated no comments.

Show a summary per file
File Description
pkg/workflow/xml_comments_test.go Derives the heredoc delimiter dynamically when counting heredoc blocks.
pkg/workflow/unified_prompt_creation_test.go Uses generated delimiter for marker counting and heredoc detection assertions.
pkg/workflow/template_expression_integration_test.go Searches for heredoc start/end markers using generated delimiter.
pkg/workflow/secure_markdown_rendering_test.go Uses generated delimiter when locating heredoc boundaries in compiled YAML.
.github/workflows/*.lock.yml Recompiled lockfiles to use the prefixed heredoc delimiter in emitted cat << '…' blocks.
Comments suppressed due to low confidence (2)

pkg/workflow/unified_prompt_creation_test.go:1

  • The comment hardcodes the current delimiter format (GH_AW_PROMPT_EOF), but the test intentionally derives it via GenerateHeredocDelimiter. To avoid the comment becoming stale if the delimiter format changes again, update the wording to something format-agnostic (e.g., 'Count heredoc delimiter markers').
    pkg/workflow/unified_prompt_creation_test.go:1
  • The comment hardcodes the current delimiter format (GH_AW_PROMPT_EOF), but the test intentionally derives it via GenerateHeredocDelimiter. To avoid the comment becoming stale if the delimiter format changes again, update the wording to something format-agnostic (e.g., 'Count heredoc delimiter markers').

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] Heredoc tests expect PROMPT_EOF instead of prefixed delimiter

2 participants

Comments