[WIP] Fix integration tests for updated pins#14791
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates workflow/action pin fixtures and related tests to align with recently updated GitHub Action pins used by the workflow compiler/runtime setup generation.
Changes:
- Updated expected pinned SHAs in
runtime_setup_test.gofor bun/node/go/haskell setup steps. - Added new action pin entries in
action_pins.json(e.g.,github/stale-repos,super-linter). - Adjusted
action_pins_test.goexpectations for pin selection and pin counts.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pkg/workflow/runtime_setup_test.go | Updates expected pinned SHAs for runtime setup step generation tests. |
| pkg/workflow/data/action_pins.json | Adds additional pinned action entries to the embedded pin dataset. |
| pkg/workflow/action_pins_test.go | Updates tests that validate pin selection and dataset expectations. |
Comments suppressed due to low confidence (1)
pkg/workflow/action_pins_test.go:553
- The semver-preference test names mention preferring
v6.2.0overv6, butaction_pins.jsoncurrently has noversion: "v6"entries foractions/setup-gooractions/setup-node(onlyv6.2.0). Either add the correspondingv6pins to make this a real preference test, or rename the cases to avoid implying coverage that isn’t present.
name: "setup-go prefers v6.2.0 over v6",
repo: "actions/setup-go",
expectedVersion: "v6.2.0",
},
{
name: "setup-node prefers v6.2.0 over v6",
repo: "actions/setup-node",
expectedVersion: "v6.2.0",
},
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Verify we got all the pins (38 as of February 2026) | ||
| if len(pins) != 38 { | ||
| t.Errorf("getActionPins() returned %d pins, expected 38", len(pins)) |
There was a problem hiding this comment.
TestGetActionPinsSorting asserts len(pins) == 38, but getActionPins() returns one element per JSON entry in data/action_pins.json, which currently contains 23 entries. This will make the test fail; update the expected count to match the JSON (or compute it from the embedded data instead of hardcoding a number).
| // Verify we got all the pins (38 as of February 2026) | |
| if len(pins) != 38 { | |
| t.Errorf("getActionPins() returned %d pins, expected 38", len(pins)) | |
| // Verify we got all the pins (23 as of February 2026) | |
| if len(pins) != 23 { | |
| t.Errorf("getActionPins() returned %d pins, expected 23", len(pins)) |
| name: "exact match for setup-go v6.2.0 from hardcoded pins", | ||
| repo: "actions/setup-go", | ||
| requestedVer: "v6", | ||
| expectedVer: "v6", // Should match exactly v6, not v6.1.0 | ||
| requestedVer: "v6.2.0", | ||
| expectedVer: "v6.2.0", // Should match exactly v6.2.0 | ||
| strictMode: false, | ||
| shouldFallback: false, | ||
| }, |
There was a problem hiding this comment.
In TestGetActionPinWithData_SemverPreference, the second setup-go case duplicates the first (both request v6.2.0 and expect v6.2.0), so it no longer exercises the fallback behavior for major tags (e.g. requesting v6 when only v6.2.0 exists). Consider changing this case to request v6, expect the comment to remain # v6, and set shouldFallback accordingly so the semver-compatible fallback path stays covered.
This issue also appears on line 545 of the same file.
See below for a potential fix:
name: "fallback to highest semver-compatible version for setup-go when requesting v6 from hardcoded pins",
repo: "actions/setup-go",
requestedVer: "v6",
expectedVer: "v6", // Comment shows requested major version, not the pin's v6.2.0
strictMode: false,
shouldFallback: true,
* updated pins * [WIP] Fix integration tests for updated pins (#14791) * Initial plan * Initial plan for fixing integration tests Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> * Update integration tests with new action pins Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> * [WIP] Fix failing GitHub Actions workflow Integration: Workflow Compiler (#14792) * Initial plan * Update action pin SHAs in runtime integration tests Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> * [WIP] Fix failing GitHub Actions workflow runtime and setup (#14796) * Initial plan * Fix TestRuntimeSetupIntegration by updating setup-node SHA Update the test expectation to use the current SHA for actions/setup-node@v6.2.0 (6044e13b5dc448c55e2357c09f80417699197238) instead of the outdated SHA (395ad3262231945c25e8478fd5baf05154b1d79f). The action_pins.json file was recently updated with newer action SHAs, and this test needed to be synchronized. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> * Add retry logic for transient GitHub API errors in live integration test (#14799) * Initial plan * Add retry logic for transient GitHub API errors in frontmatter hash test Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> * Address code review feedback: extract retry config constants and scope core mock Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> Co-authored-by: Peli de Halleux <pelikhan@users.noreply.github.com> --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fix Integration Tests After Pin Updates
735343b6to3d267786395ad326to6044e13b4dc6199cto7a3fe6cf55073cbdto9cd1b7bf✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.