Skip to content

Optimize CI with change-driven gating #1902

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 5, 2025
Merged

Optimize CI with change-driven gating #1902

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a41f78c
Optimize CI workflows with change-driven gating
justin808 Nov 5, 2025
e569c15
Optimize CI for faster branch builds and fix critical issues
justin808 Nov 5, 2025
b0b49cd
Fix Pro workflows: use correct working-directory for detect-changes
justin808 Nov 5, 2025
deb7bb9
Fix workflow syntax: remove matrix context from job-level if conditions
justin808 Nov 5, 2025
fbda2dc
Update CONTRIBUTING.md files with CI optimization documentation
justin808 Nov 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .claude/commands/run-ci.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Run CI Command

Analyze the current branch changes and run appropriate CI checks locally.

## Instructions

1. First, run `script/ci-changes-detector origin/master` to analyze what changed
2. Show the user what the detector recommends
3. Ask the user if they want to:
- Run the recommended CI jobs (`bin/ci-local`)
- Run all CI jobs (`bin/ci-local --all`)
- Run a fast subset (`bin/ci-local --fast`)
- Run specific jobs manually
4. Execute the chosen option and report results
5. If any jobs fail, offer to help fix the issues

## Options

- `bin/ci-local` - Run CI based on detected changes
- `bin/ci-local --all` - Run all CI checks (same as CI on master)
- `bin/ci-local --fast` - Run only fast checks, skip slow integration tests
- `bin/ci-local [base-ref]` - Compare against a specific ref instead of origin/master
57 changes: 57 additions & 0 deletions .github/workflows/detect-changes.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Detect Changes

on:
workflow_call:
outputs:
docs_only:
description: 'Only documentation files changed'
value: ${{ jobs.detect.outputs.docs_only }}
run_lint:
description: 'Should run linting'
value: ${{ jobs.detect.outputs.run_lint }}
run_ruby_tests:
description: 'Should run Ruby tests'
value: ${{ jobs.detect.outputs.run_ruby_tests }}
run_js_tests:
description: 'Should run JS tests'
value: ${{ jobs.detect.outputs.run_js_tests }}
run_dummy_tests:
description: 'Should run dummy app tests'
value: ${{ jobs.detect.outputs.run_dummy_tests }}
run_generators:
description: 'Should run generator tests'
value: ${{ jobs.detect.outputs.run_generators }}

jobs:
detect:
runs-on: ubuntu-22.04
outputs:
docs_only: ${{ steps.changes.outputs.docs_only }}
run_lint: ${{ steps.changes.outputs.run_lint }}
run_ruby_tests: ${{ steps.changes.outputs.run_ruby_tests }}
run_js_tests: ${{ steps.changes.outputs.run_js_tests }}
run_dummy_tests: ${{ steps.changes.outputs.run_dummy_tests }}
run_generators: ${{ steps.changes.outputs.run_generators }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false

- name: Detect changes
id: changes
run: |
# For master branch, always run everything
if [ "${{ github.ref }}" = "refs/heads/master" ]; then
echo "docs_only=false" >> "$GITHUB_OUTPUT"
echo "run_lint=true" >> "$GITHUB_OUTPUT"
echo "run_ruby_tests=true" >> "$GITHUB_OUTPUT"
echo "run_js_tests=true" >> "$GITHUB_OUTPUT"
echo "run_dummy_tests=true" >> "$GITHUB_OUTPUT"
echo "run_generators=true" >> "$GITHUB_OUTPUT"
exit 0
fi

# For PRs, analyze changes
BASE_SHA="${{ github.event.pull_request.base.sha || github.event.before }}"
script/ci-changes-detector "$BASE_SHA"
Comment on lines +1 to +57
Copy link
Contributor

@coderabbitai coderabbitai bot Nov 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify if this reusable workflow is actually called via workflow_call.

The detect-changes.yml is defined as a reusable workflow using workflow_call, but the other workflows (main.yml, lint-js-and-ruby.yml, etc.) don't appear to call it. Instead, they duplicate the detect-changes job inline. This breaks the DRY principle and creates maintenance burden if the detection logic changes.

Option A (recommended): Refactor workflows to call this reusable workflow instead of duplicating it. Example:

jobs:
  detect-changes:
    uses: ./.github/workflows/detect-changes.yml

Option B: If inline duplication is intentional (e.g., to avoid job dependency overhead), remove the standalone detect-changes.yml file to avoid confusion.

Please clarify which approach is intended.

🏁 Script executed:

#!/bin/bash
# Count how many times detect-changes job is defined across workflows
echo "=== detect-changes job definitions ==="
find .github/workflows -name "*.yml" -type f -exec grep -l "jobs:" {} \; | while read file; do
  if grep -A5 "^  detect-changes:" "$file" > /dev/null 2>&1; then
    echo "Inline in: $file"
  fi
done

echo ""
echo "=== Uses of reusable detect-changes.yml ==="
grep -r "uses:.*detect-changes" .github/workflows || echo "No reusable workflow calls found"

Length of output: 497


---

🏁 Script executed:

```shell
#!/bin/bash
# Check if script/ci-changes-detector exists
if [ -f "script/ci-changes-detector" ]; then
  echo "✓ script/ci-changes-detector exists"
  head -20 "script/ci-changes-detector"
else
  echo "✗ script/ci-changes-detector NOT found"
  find . -name "*ci-changes*" -o -name "*changes-detector*" 2>/dev/null | head -20
fi

echo ""
echo "=== Checking inline detect-changes job from main.yml ==="
grep -A 30 "^  detect-changes:" .github/workflows/main.yml | head -35

Length of output: 1893


---

🏁 Script executed:

```shell
#!/bin/bash
# Compare BASE_SHA/BASE_REF logic across workflows

echo "=== Checking BASE_SHA/BASE_REF fallback in reusable detect-changes.yml ==="
grep -A 2 "BASE_SHA=" .github/workflows/detect-changes.yml

echo ""
echo "=== Checking BASE_REF/BASE_SHA fallback in inline detect-changes jobs ==="
for file in .github/workflows/{main,lint-js-and-ruby,examples}.yml; do
  if [ -f "$file" ]; then
    echo "--- $file ---"
    grep -A 2 "BASE_REF\|BASE_SHA" "$file" | head -5
  fi
done

Length of output: 930

Fix missing fallback in detect-changes.yml and consolidate reusable workflow usage.

The reusable workflow is correctly defined but not used—8 workflows duplicate the detect-changes job inline instead. More critically, detect-changes.yml has a bug: it lacks the 'origin/master' fallback for BASE_SHA that all inline implementations include.

Line 56 in detect-changes.yml:

BASE_SHA="${{ github.event.pull_request.base.sha || github.event.before }}"

Should match the robust fallback used in main.yml, lint-js-and-ruby.yml, and others:

BASE_SHA="${{ github.event.pull_request.base.sha || github.event.before || 'origin/master' }}"

Once fixed, refactor workflows to call the reusable workflow via uses: ./.github/workflows/detect-changes.yml to eliminate duplication.

🤖 Prompt for AI Agents 
.github/workflows/detect-changes.yml lines 1-57: the Detect changes step sets
BASE_SHA using only github.event.pull_request.base.sha or github.event.before,
missing the 'origin/master' fallback used elsewhere; update the BASE_SHA
assignment to include the third fallback 'origin/master' (so it tries
pull_request.base.sha, then github.event.before, then 'origin/master') and then
replace duplicated inline detect-changes jobs across workflows with calls to
this reusable workflow via uses: ./.github/workflows/detect-changes.yml to
consolidate and eliminate duplication.

57 changes: 32 additions & 25 deletions .github/workflows/examples.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,49 @@ on:
push:
branches:
- 'master'
paths-ignore:
- '**.md'
- 'docs/**'
pull_request:
paths-ignore:
- '**.md'
- 'docs/**'

jobs:
detect-changes:
runs-on: ubuntu-22.04
outputs:
docs_only: ${{ steps.detect.outputs.docs_only }}
run_lint: ${{ steps.detect.outputs.run_lint }}
run_js_tests: ${{ steps.detect.outputs.run_js_tests }}
run_ruby_tests: ${{ steps.detect.outputs.run_ruby_tests }}
run_dummy_tests: ${{ steps.detect.outputs.run_dummy_tests }}
run_generators: ${{ steps.detect.outputs.run_generators }}
steps:
- uses: actions/checkout@v4
with:
# Fetch enough history for change detection (50 commits is usually sufficient for PRs)
fetch-depth: 50
persist-credentials: false
- name: Detect relevant changes
id: detect
run: |
BASE_REF="${{ github.event.pull_request.base.sha || github.event.before || 'origin/master' }}"
script/ci-changes-detector "$BASE_REF"
shell: bash

examples:
needs: detect-changes
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_generators == 'true'
strategy:
fail-fast: false
matrix:
ruby-version: ['3.2', '3.4']
dependency-level: ['minimum', 'latest']
include:
- ruby-version: '3.2'
dependency-level: 'minimum'
# Always run: Latest versions (fast feedback on PRs)
- ruby-version: '3.4'
dependency-level: 'latest'
exclude:
# Master only: Minimum supported versions (full coverage)
- ruby-version: '3.2'
dependency-level: 'latest'
- ruby-version: '3.4'
dependency-level: 'minimum'
env:
SKIP_YARN_COREPACK_CHECK: 0
Expand All @@ -31,23 +56,6 @@ jobs:
- uses: actions/checkout@v4
with:
persist-credentials: false
- name: Get changed files
id: changed-files
run: |
BASE_SHA=${{ github.event.pull_request.base.sha || github.event.before }}
git fetch origin $BASE_SHA
CHANGED_FILES=$(git diff --name-only $BASE_SHA ${{ github.sha }} -- \
lib/generators/ \
rakelib/example_type.rb \
rakelib/example_config.yml \
rakelib/examples.rake \
rakelib/run_rspec.rake)
if [ -n "$CHANGED_FILES" ]; then
ANY_CHANGED=true
else
ANY_CHANGED=false
fi
echo "any_changed=$ANY_CHANGED" >> "$GITHUB_OUTPUT"
- name: Setup Ruby
uses: ruby/setup-ruby@v1
with:
Expand Down Expand Up @@ -108,7 +116,6 @@ jobs:
run: |
echo "CI_DEPENDENCY_LEVEL=${{ matrix.dependency-level }}" >> $GITHUB_ENV
- name: Main CI
if: steps.changed-files.outputs.any_changed == 'true'
run: bundle exec rake run_rspec:shakapacker_examples
- name: Store test results
uses: actions/upload-artifact@v4
Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/lint-js-and-ruby.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,47 @@ on:
push:
branches:
- 'master'
paths-ignore:
- '**.md'
- 'docs/**'
pull_request:
paths-ignore:
- '**.md'
- 'docs/**'

jobs:
detect-changes:
runs-on: ubuntu-22.04
outputs:
docs_only: ${{ steps.detect.outputs.docs_only }}
run_lint: ${{ steps.detect.outputs.run_lint }}
run_js_tests: ${{ steps.detect.outputs.run_js_tests }}
run_ruby_tests: ${{ steps.detect.outputs.run_ruby_tests }}
run_dummy_tests: ${{ steps.detect.outputs.run_dummy_tests }}
run_generators: ${{ steps.detect.outputs.run_generators }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
persist-credentials: false
- name: Detect relevant changes
id: detect
run: |
BASE_REF="${{ github.event.pull_request.base.sha || github.event.before || 'origin/master' }}"
script/ci-changes-detector "$BASE_REF"
shell: bash

build:
needs: detect-changes
if: github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_lint == 'true'
env:
BUNDLE_FROZEN: true
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
with:
# No need for history in lint job
fetch-depth: 1
persist-credentials: false
- name: Setup Ruby
uses: ruby/setup-ruby@v1
Expand Down
59 changes: 42 additions & 17 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,53 @@ on:
push:
branches:
- 'master'
paths-ignore:
- '**.md'
- 'docs/**'
pull_request:
paths-ignore:
- '**.md'
- 'docs/**'

jobs:
detect-changes:
runs-on: ubuntu-22.04
outputs:
docs_only: ${{ steps.detect.outputs.docs_only }}
run_lint: ${{ steps.detect.outputs.run_lint }}
run_js_tests: ${{ steps.detect.outputs.run_js_tests }}
run_ruby_tests: ${{ steps.detect.outputs.run_ruby_tests }}
run_dummy_tests: ${{ steps.detect.outputs.run_dummy_tests }}
run_generators: ${{ steps.detect.outputs.run_generators }}
steps:
- uses: actions/checkout@v4
with:
# Fetch enough history for change detection (50 commits is usually sufficient for PRs)
fetch-depth: 50
persist-credentials: false
- name: Detect relevant changes
id: detect
run: |
BASE_REF="${{ github.event.pull_request.base.sha || github.event.before || 'origin/master' }}"
script/ci-changes-detector "$BASE_REF"
shell: bash

build-dummy-app-webpack-test-bundles:
needs: detect-changes
# Run on master OR when tests needed on PR (but skip minimum deps on PR)
if: |
(github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_dummy_tests == 'true')
strategy:
matrix:
ruby-version: ['3.2', '3.4']
node-version: ['20', '22']
include:
- ruby-version: '3.2'
node-version: '20'
dependency-level: 'minimum'
# Always run: Latest versions (fast feedback on PRs)
- ruby-version: '3.4'
node-version: '22'
dependency-level: 'latest'
exclude:
# Master only: Minimum supported versions (full coverage)
- ruby-version: '3.2'
node-version: '22'
- ruby-version: '3.4'
node-version: '20'
dependency-level: 'minimum'
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -91,24 +118,22 @@ jobs:
key: dummy-app-webpack-bundle-${{ steps.get-sha.outputs.sha }}-ruby${{ matrix.ruby-version }}-${{ matrix.dependency-level }}

dummy-app-integration-tests:
needs: build-dummy-app-webpack-test-bundles
needs: [detect-changes, build-dummy-app-webpack-test-bundles]
# Run on master OR when tests needed on PR (but skip minimum deps on PR)
if: |
(github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_dummy_tests == 'true')
strategy:
fail-fast: false
matrix:
ruby-version: ['3.2', '3.4']
node-version: ['20', '22']
include:
- ruby-version: '3.2'
node-version: '20'
dependency-level: 'minimum'
# Always run: Latest versions (fast feedback on PRs)
- ruby-version: '3.4'
node-version: '22'
dependency-level: 'latest'
exclude:
# Master only: Minimum supported versions (full coverage)
- ruby-version: '3.2'
node-version: '22'
- ruby-version: '3.4'
node-version: '20'
dependency-level: 'minimum'
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
Expand Down
42 changes: 41 additions & 1 deletion .github/workflows/package-js-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,53 @@ on:
push:
branches:
- 'master'
paths-ignore:
- '**.md'
- 'docs/**'
- 'lib/**'
- 'spec/react_on_rails/**'
pull_request:
paths-ignore:
- '**.md'
- 'docs/**'
- 'lib/**'
- 'spec/react_on_rails/**'

jobs:
detect-changes:
runs-on: ubuntu-22.04
outputs:
docs_only: ${{ steps.detect.outputs.docs_only }}
run_lint: ${{ steps.detect.outputs.run_lint }}
run_js_tests: ${{ steps.detect.outputs.run_js_tests }}
run_ruby_tests: ${{ steps.detect.outputs.run_ruby_tests }}
run_dummy_tests: ${{ steps.detect.outputs.run_dummy_tests }}
run_generators: ${{ steps.detect.outputs.run_generators }}
steps:
- uses: actions/checkout@v4
with:
# Fetch enough history for change detection (50 commits is usually sufficient for PRs)
fetch-depth: 50
persist-credentials: false
- name: Detect relevant changes
id: detect
run: |
BASE_REF="${{ github.event.pull_request.base.sha || github.event.before || 'origin/master' }}"
script/ci-changes-detector "$BASE_REF"
shell: bash

build:
needs: detect-changes
# Run on master OR when JS tests needed on PR (but skip Node 20 on PR)
if: |
(github.ref == 'refs/heads/master' || needs.detect-changes.outputs.run_js_tests == 'true')
strategy:
matrix:
node-version: ['20', '22']
include:
# Always run: Latest Node version (fast feedback on PRs)
- node-version: '22'
# Master only: Minimum supported Node version (full coverage)
- node-version: '20'
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
Expand Down
Loading
Loading