-
Notifications
You must be signed in to change notification settings - Fork 46
perf: parallelize git ls-remote test for 70% faster execution #2693
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: parallelize git ls-remote test for 70% faster execution #2693
Conversation
Optimize TestActionPinSHAsMatchVersionTags by running git ls-remote operations in parallel instead of sequentially. Performance Impact: - Before: 3.83s (slowest test in suite) - After: ~1.1s average (no longer in top 10 slowest) - Improvement: 70% faster (3.5x speedup) Implementation: - Added t.Parallel() to subtests - Captured loop variables for safe parallel execution - Zero functional changes, all tests pass The test verifies that action pin SHAs match their version tags by querying GitHub repositories. Running these network operations in parallel significantly reduces execution time while maintaining correctness.
Document the successful optimization of TestActionPinSHAsMatchVersionTags as a concrete example of how to use t.Parallel() for network-bound tests. This provides future developers with: - Before/after code comparison - Measured performance impact (70% improvement) - Key insight about I/O-bound vs CPU-bound tests - Best practice for variable capture in parallel tests
|
I have a pr with a persistent cache for pins |
|
Agentic Changeset Generator triggered by this pull request. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR optimizes test execution time by parallelizing network-bound test subtests. The TestActionPinSHAsMatchVersionTags test, which validates GitHub Action SHAs by making sequential git ls-remote network calls, is converted to run subtests in parallel.
- Adds
t.Parallel()to enable concurrent execution of network-bound subtests - Properly captures loop variables (
keyandpin) for safe parallel execution - Documents the optimization approach in build-performance.md with a success story showing 70% improvement
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| pkg/workflow/action_pins_test.go | Adds parallel execution to TestActionPinSHAsMatchVersionTags with proper loop variable capture |
| .github/copilot/instructions/build-performance.md | Documents the parallelization pattern as a success story with before/after example and performance metrics |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Parallel Git Ls-Remote Test Optimization
Goal and Rationale
Performance Target: Reduce test suite execution time by optimizing the slowest test.
The
TestActionPinSHAsMatchVersionTagstest was the slowest in the test suite at 3.83 seconds, accounting for a significant portion of test execution time. This test validates that action pin SHAs match their version tags by making sequentialgit ls-remotenetwork calls to GitHub repositories.Why This Matters:
make test-unitApproach
Implemented parallel test execution using Go's built-in
t.Parallel()functionality:t.Parallel()call to enable concurrent subtest executionkey,pin) for safe parallel accessStrategy: Network-bound operations benefit significantly from parallelization since they spend most time waiting for I/O rather than consuming CPU.
Performance Impact
Before Optimization
After Optimization
Measured Results
make test-perf)Implementation Details
Changes: Modified only
pkg/workflow/action_pins_test.goCode Quality:
gofmtTrade-offs
Complexity: Minimal (+5 lines, standard pattern)
Resource Usage: Negligible
Maintainability: Improved
Validation
Test Correctness
All tests continue to pass with identical behavior:
Performance Consistency
Multiple test runs confirm stable performance improvement:
$ go test -count=1 -run=TestActionPinSHAsMatchVersionTags ./pkg/workflow ok github.com/githubnext/gh-aw/pkg/workflow 1.076s ok github.com/githubnext/gh-aw/pkg/workflow 1.221s ok github.com/githubnext/gh-aw/pkg/workflow 1.122sReproducibility
To Reproduce Performance Measurement
Before: Check out main branch
After: Check out this PR branch
Individual Test Timing:
Next Steps
This optimization contributes to the broader test performance goal from the research plan:
Future Opportunities:
make test -p Nfor parallel package executionRelated to: Daily Perf Improver research plan (Discussion #2191)
Performance Area: Test Suite Performance
Alignment: Directly addresses "Unit tests: < 20s" target