fix: prevent non-E2E labels from triggering workflow runs#23459
fix: prevent non-E2E labels from triggering workflow runs#23459keithwillcode merged 3 commits intomainfrom
Conversation
- Add conditional logic to all jobs to skip when non-ready-for-e2e labels are added - Only allow ready-for-e2e label additions to trigger workflows - Preserve all other trigger types (opened, synchronize, reopened) - Fixes issue where labels like 'Improvements' cause unnecessary CI runs Resolves the annoyance where adding labels like 'Improvements' to PRs triggers full workflow runs including all checks, when only the ready-for-e2e label should trigger E2E tests and associated builds. Co-Authored-By: keith@cal.com <keithwillcode@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
WalkthroughThe PR updates .github/workflows/pr.yml to introduce an env variable SKIP_WORKFLOW, evaluated true when the event is a label action and the label is not ready-for-e2e. This variable is used to gate most jobs via if conditions, including changes, check-label, deps, type-check, lint, unit-test, build variants, integration and e2e jobs, analyze, report merge/publish/cleanup, and the required fail step. The check-label job’s run-e2e decision is reworked to depend on SKIP_WORKFLOW. When SKIP_WORKFLOW is true, these jobs are skipped. Possibly related PRs
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
Status, Documentation and Community
|
…iable - Replace repeated conditional logic with single SKIP_WORKFLOW env variable - Simplify all job conditions from complex expression to env.SKIP_WORKFLOW != 'true' - Maintain exact same functional behavior while improving maintainability - Reduces code duplication across 20+ jobs in the workflow This addresses feedback to consolidate the repeated conditional logic: github.event.action != 'labeled' || github.event.label.name == 'ready-for-e2e' The refactored approach is cleaner and easier to maintain. Co-Authored-By: keith@cal.com <keithwillcode@gmail.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
- Remove 'skipped' from failure conditions in required job - Add SKIP_WORKFLOW condition to required job logic - Prevents false failures when E2E jobs legitimately skip - Maintains proper failure detection for actual job failures This fixes the CI failure where the required job was treating legitimately skipped E2E jobs as failures when no ready-for-e2e label is present. Co-Authored-By: keith@cal.com <keithwillcode@gmail.com>
E2E results are ready! |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
.github/workflows/pr.yml (1)
95-97: Minor hardening: guard against odd label payloadsUnlikely, but to be defensive you can normalize labels before mapping.
- const labelFound = labels.map(l => l.name).includes('ready-for-e2e'); + const names = Array.isArray(labels) ? labels.map(l => l?.name).filter(Boolean) : []; + const labelFound = names.includes('ready-for-e2e');
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
.github/workflows/pr.yml(5 hunks)
🧰 Additional context used
🪛 actionlint (1.7.7)
.github/workflows/pr.yml
25-25: label "buildjet-2vcpu-ubuntu-2204" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2025", "windows-2022", "windows-2019", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-24.04-arm", "ubuntu-22.04", "ubuntu-22.04-arm", "ubuntu-20.04", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
26-26: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
50-50: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
102-102: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
108-108: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
115-115: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
122-122: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
129-129: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
136-136: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
143-143: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
150-150: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
157-157: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
164-164: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
171-171: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
178-178: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
185-185: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
192-192: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
199-199: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
206-206: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
213-213: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
219-219: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
226-226: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
237-237: context "env" is not allowed here. available contexts are "github", "inputs", "needs", "vars". see https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability for more details
(expression)
🔇 Additional comments (1)
.github/workflows/pr.yml (1)
25-26: Verify runner labels exist in the orgbuildjet-2vcpu-ubuntu-2204 is unknown to actionlint. If these are self-hosted, you’re fine; otherwise provide a fallback or update actionlint config.
You can validate runner labels and add an actionlint config entry if needed.
Also applies to: 48-49
| env: | ||
| SKIP_WORKFLOW: ${{ github.event.action == 'labeled' && github.event.label.name != 'ready-for-e2e' }} | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Job-level if cannot use env context; gating will not work as intended
actionlint is correct: env is unavailable in job.if. This will either error or be ignored, defeating the skip intent across all jobs. Inline the label gate expression (or derive it via a needs output) and drop the workflow env.
Apply this diff to remove the global env and rely on direct expressions:
- env:
- SKIP_WORKFLOW: ${{ github.event.action == 'labeled' && github.event.label.name != 'ready-for-e2e' }}Then update job conditions as shown in the next comment. If you prefer a single source of truth, add a tiny “gate” job that outputs skip=true/false and reference it via needs.gate.outputs.skip in job.if (needs context is allowed at job level).
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| env: | |
| SKIP_WORKFLOW: ${{ github.event.action == 'labeled' && github.event.label.name != 'ready-for-e2e' }} |
🤖 Prompt for AI Agents
.github/workflows/pr.yml around lines 19-21: the workflow sets a global env
SKIP_WORKFLOW using an expression that references github.event and expects it to
be usable inside job.if, but job-level if cannot access env values so the gate
will be ignored or error; remove the global env entry and instead put the
label-check expression directly into each job's if (or create a tiny “gate” job
that computes skip=true/false and exposes it as an output which other jobs
reference via needs.gate.outputs.skip), updating all job.if conditions to use
the inline expression or needs.gate.outputs.skip as appropriate.
| run-e2e: ${{ steps.check-if-pr-has-label.outputs.run-e2e == 'true' && env.SKIP_WORKFLOW != 'true' }} | ||
| steps: |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Do not couple run-e2e output to SKIP gating
Since the job itself is gated, keep run-e2e purely about label presence.
- run-e2e: ${{ steps.check-if-pr-has-label.outputs.run-e2e == 'true' && env.SKIP_WORKFLOW != 'true' }}
+ run-e2e: ${{ steps.check-if-pr-has-label.outputs.run-e2e == 'true' }}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| run-e2e: ${{ steps.check-if-pr-has-label.outputs.run-e2e == 'true' && env.SKIP_WORKFLOW != 'true' }} | |
| steps: | |
| run-e2e: ${{ steps.check-if-pr-has-label.outputs.run-e2e == 'true' }} | |
| steps: |
🤖 Prompt for AI Agents
.github/workflows/pr.yml around lines 54-55: the run-e2e job condition currently
combines the label check with an environment SKIP_WORKFLOW gate; remove the
SKIP_WORKFLOW portion so run-e2e only depends on the label output. Update the if
expression to check only that steps.check-if-pr-has-label.outputs.run-e2e ==
'true' and leave any SKIP_WORKFLOW gating to the job-level if that already
exists.
keithwillcode
deleted the
devin/1756510618-fix-workflow-label-triggers
branch
|
This was reverted out of main due to an issue not triggering all jobs properly. |
2 participants
What does this PR do?
Fixes an annoyance with the
ready-for-e2elabel where adding any label to a PR (like "Improvements") triggers unnecessary workflow runs that consume CI resources.Problem: Currently, the PR workflow triggers on all
labeledevents, but only controls E2E test execution - not whether the entire workflow should run. This means labels like "Improvements" trigger full workflow runs including type checking, linting, unit tests, and builds, even though only theready-for-e2elabel should trigger these comprehensive checks.Solution: Added conditional logic
github.event.action != 'labeled' || github.event.label.name == 'ready-for-e2e'to all jobs in the PR workflow. This ensures:ready-for-e2elabel triggers workflow runs when labels are addedHow should this be tested?
Critical testing steps:
Test non-E2E label behavior:
Test ready-for-e2e label behavior:
ready-for-e2elabel to this PRTest other trigger types:
synchronizetriggers still workopened) trigger workflows normallyreopenedPRs trigger workflowsEdge case testing:
ready-for-e2elabelMandatory Tasks (DO NOT REMOVE)
High Priority - Manual Testing Required:
github.event.action != 'labeled' || github.event.label.name == 'ready-for-e2e'works as intended across all 22+ modified jobsMedium Priority:
Potential Risks:
Session Details: