Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate backend checks #19126

Closed
wants to merge 83 commits into from
Closed
Show file tree
Hide file tree
Changes from 81 commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
f2c63a8
rename files for clarity/add notes
ryan-mcneil Oct 22, 2024
1f8dff7
replace rspec run with single spec for testing
ryan-mcneil Oct 28, 2024
f93499c
add backend-pr-labeler
ryan-mcneil Oct 29, 2024
d32d8b4
disable old checks
ryan-mcneil Oct 29, 2024
c623e10
add mobile comment
ryan-mcneil Oct 29, 2024
df89103
fix code checks and export var
ryan-mcneil Oct 29, 2024
cf4dd36
run full specs + add 'labeled' option
ryan-mcneil Oct 29, 2024
b6b801d
test exit 1 and echo pr info
ryan-mcneil Oct 29, 2024
ec532ee
Add require-be label
ryan-mcneil Oct 29, 2024
4fa36ff
add curly braces
ryan-mcneil Oct 30, 2024
0a3b415
fix draft check
ryan-mcneil Oct 30, 2024
69f42a4
remove backend label
ryan-mcneil Oct 30, 2024
a4cfbfa
add ready-for-backend-review from old job with small mods
ryan-mcneil Oct 31, 2024
42935d4
correctly use outputs
ryan-mcneil Oct 31, 2024
bc6ba71
add better logging
ryan-mcneil Oct 31, 2024
70e411c
Merge branch 'master' into rm-consolidate-backend-checks
ryan-mcneil Oct 31, 2024
56c2bdf
fix if statement format
ryan-mcneil Oct 31, 2024
197d6bf
adjust failure check logic
ryan-mcneil Oct 31, 2024
818fede
adjust logging
ryan-mcneil Oct 31, 2024
3cac44d
fix id
ryan-mcneil Oct 31, 2024
70d09b8
more output
ryan-mcneil Nov 1, 2024
8b46a7b
correctly use 'booleans'
ryan-mcneil Nov 1, 2024
19b1bd4
remove unnecessary if statements
ryan-mcneil Nov 1, 2024
bbd1e93
debug label logic
ryan-mcneil Nov 1, 2024
620de09
Merge branch 'master' into rm-consolidate-backend-checks
ryan-mcneil Nov 4, 2024
63dd1b7
fix codechecks and remove unecessary labels
ryan-mcneil Nov 5, 2024
9a187aa
run labeler after code_checks
ryan-mcneil Nov 5, 2024
295ae5b
fix error in codeowners check
ryan-mcneil Nov 5, 2024
b658265
Merge branch 'master' into rm-consolidate-backend-checks
ryan-mcneil Nov 5, 2024
cb1252b
fix if syntax
ryan-mcneil Nov 5, 2024
8bea003
add synchronize to test
ryan-mcneil Nov 5, 2024
235bb5c
debugging
ryan-mcneil Nov 5, 2024
356b177
again
ryan-mcneil Nov 5, 2024
f4d1860
remove sync
ryan-mcneil Nov 5, 2024
b0f2f92
fix ==
ryan-mcneil Nov 5, 2024
ee0b631
debugging
ryan-mcneil Nov 5, 2024
2dc75fd
indentation :(
ryan-mcneil Nov 6, 2024
ff37ff2
Try running without specs
ryan-mcneil Nov 6, 2024
887945a
remove steps and completed type
ryan-mcneil Nov 6, 2024
3db8c7d
codechecks logging
ryan-mcneil Nov 6, 2024
2081c15
merge master
ryan-mcneil Nov 6, 2024
56b5e7f
Trigger build
ryan-mcneil Nov 6, 2024
cb8c3c8
add pull_request_review type, parse pull request, move test labels
ryan-mcneil Nov 7, 2024
008a821
add be pr approval
ryan-mcneil Nov 7, 2024
51f7f80
fix pr grab
ryan-mcneil Nov 7, 2024
d9c8e6b
convert to JSON to store object
ryan-mcneil Nov 7, 2024
29020ff
correctly set pull_reqeust
ryan-mcneil Nov 7, 2024
dceb9e6
fix pull_request usage
ryan-mcneil Nov 7, 2024
a671abe
fix/refactor env vars
ryan-mcneil Nov 7, 2024
a44777e
fix labels env var
ryan-mcneil Nov 7, 2024
e1825a3
Used AI to do a bit of ENV refactoring
ryan-mcneil Nov 7, 2024
d52c120
back before AI
ryan-mcneil Nov 8, 2024
4726d20
retrieve pr values once
ryan-mcneil Nov 8, 2024
b264d8c
fix array JSON
ryan-mcneil Nov 8, 2024
53d0886
use different quotes
ryan-mcneil Nov 8, 2024
b581b4b
Try Claude rec
ryan-mcneil Nov 8, 2024
5e2939f
remove broken debugging lines
ryan-mcneil Nov 8, 2024
3df4335
fix failure label comparison
ryan-mcneil Nov 8, 2024
f2ab5de
log PR data
ryan-mcneil Nov 8, 2024
a54514c
remove spaces after ||
ryan-mcneil Nov 8, 2024
411d240
try without square brackets
ryan-mcneil Nov 8, 2024
347b157
fix draft var
ryan-mcneil Nov 8, 2024
370f621
Update labeling notes/requirements
ryan-mcneil Nov 8, 2024
658055d
merge master
ryan-mcneil Nov 8, 2024
4c20dde
retrieve entire pr object
ryan-mcneil Nov 8, 2024
163d600
Merge branch 'master' into rm-consolidate-backend-checks
ryan-mcneil Nov 8, 2024
c652466
update for Code Checks running on push to master
ryan-mcneil Nov 8, 2024
5385db0
Use pull_request structure for pull_request_review
ryan-mcneil Nov 8, 2024
92bf1fe
update logging
ryan-mcneil Nov 8, 2024
0493802
merge master
ryan-mcneil Nov 8, 2024
43cfab4
merge master
ryan-mcneil Nov 8, 2024
d92ce59
Add extra job to filter on:push events
ryan-mcneil Nov 11, 2024
5f656f2
merge master
ryan-mcneil Nov 11, 2024
8e3a7fc
Comment out unused test-status code
ryan-mcneil Nov 11, 2024
56df7bb
fix get-pr-data var
ryan-mcneil Nov 11, 2024
9a2e3de
Merge branch 'master' into rm-consolidate-backend-checks
ryan-mcneil Nov 11, 2024
825ab22
Clean up labeler
ryan-mcneil Nov 11, 2024
599ba0e
Fix label logic
ryan-mcneil Nov 12, 2024
d9f9b7a
Clean up code checks
ryan-mcneil Nov 12, 2024
f60e1ac
remove comment
ryan-mcneil Nov 12, 2024
072aeca
remove old files
ryan-mcneil Nov 12, 2024
12451e9
few more things from code_checks
ryan-mcneil Nov 12, 2024
a290b27
update workflow name
ryan-mcneil Nov 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 24 additions & 7 deletions .github/workflows/backend-pr-approver.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,30 @@
name: Backend PR Approver
on:
workflow_run:
workflows:
- "Backend PR Labeler"
types: [completed]
pull_request:
types: [labeled, unlabeled]
jobs:
say-hi:
check-approval-labels:
if: github.event.label.name == 'require-backend-approval' || github.event.label.name == 'ready-for-backend-review'
runs-on: ubuntu-latest
steps:
- name: Say Hi
- name: Check for exemption
id: check_exemption
run: |
echo "Hi"
if [ ! ${{ contains(toJSON(github.event.pull_request.requested_teams.*.name), 'backend-review-group') }} ]; then
echo "exempt=true" >> $GITHUB_OUTPUT
echo "PR is exempt from backend approval."
else
echo "exempt=false" >> $GITHUB_OUTPUT
echo "PR requires backend approval."
fi
Comment on lines +14 to +20
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this true? In CODEOWNERS, files either have backend-review-group listed, OR they have a lighthouse team, octo-identity, or mobile? 🤔 no... I just found this line: spec/lib/token_validation @department-of-veterans-affairs/lighthouse-dash @department-of-veterans-affairs/lighthouse-banana-peels @department-of-veterans-affairs/va-api-engineers @department-of-veterans-affairs/backend-review-group
With the logic here, a PR that changes files in this dir would hit the else block.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe the solution is to remove backend-review-group from those lines in codeowners 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would be an example of a CODEOWNERS issue. If they own it, our team doesn't need to be on it. If we should be involved in the review, then it'll work as expected. I think these cases will iron themselves out. I can't think of a good reason for why both would be needed yet we shouldn't be a required approver.


- name: Check for backend approval
id: check_approval
if: ${{ steps.check_exemption.outputs.exempt == 'false' }}
run: |
if [ ${{ contains(toJSON(github.event.pull_request.labels.*.name), 'require-backend-approval') && contains(toJSON(github.event.pull_request.labels.*.name), 'ready-for-backend-review') }} ]; then
echo "Pull Request missing approval."
exit 1
else
echo "Pull Request approved."
fi
73 changes: 66 additions & 7 deletions .github/workflows/backend-pr-labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ jobs:
pr_labels: ${{ needs.get-pr-data.outputs.pr_labels }}
pr_requested_teams: ${{ needs.get-pr-data.outputs.pr_requested_teams }}
outputs:
test_status: ${{ steps.get_code_checks_conclusion.outputs.test_status }}
test_result: ${{ steps.get_code_checks_conclusion.outputs.test_result }}
exempt: ${{ steps.check_exemption.outputs.exempt }}
failures_detected: ${{ steps.audit_pr_labels.outputs.failures_detected }}
steps:
Expand All @@ -86,8 +86,8 @@ jobs:
- name: Get Code Checks result
if: github.event_name == 'workflow_run'
run: |
echo "test_status=${{ github.event.workflow_run.conclusion }}" >> $GITHUB_OUTPUT
echo "test_status: ${{ github.event.workflow_run.conclusion }}"
echo "test_result=${{ github.event.workflow_run.conclusion }}" >> $GITHUB_OUTPUT
echo "test_result: ${{ github.event.workflow_run.conclusion }}"

- name: Check for exemption
id: check_exemption
Expand All @@ -107,8 +107,7 @@ jobs:
${{ contains(env.pr_labels, 'code-health-failure') }} || \
${{ contains(env.pr_labels, 'codeowners-addition-failure') }} || \
${{ contains(env.pr_labels, 'codeowners-delete-failure') }} || \
${{ contains(env.pr_labels, 'lint-failure') }} || \
${{ contains(env.pr_labels, 'test-failure') }} ; then
${{ contains(env.pr_labels, 'lint-failure') }} ; then
echo "failures_detected=true" >> $GITHUB_OUTPUT
echo "Failure labels detected."
else
Expand Down Expand Up @@ -190,7 +189,7 @@ jobs:
pr_number: ${{ needs.get-pr-data.outputs.pr_number }}
pr_draft: ${{ needs.get-pr-data.outputs.pr_draft }}
pr_labels: ${{ needs.get-pr-data.outputs.pr_labels }}
test_status: ${{ needs.check-pr-status.outputs.test_status }}
test_result: ${{ needs.check-pr-status.outputs.test_result }}
failures_detected: ${{ needs.check-pr-status.outputs.failures_detected }}
approval_status: ${{ needs.check-approvals.outputs.approval_status }}
steps:
Expand All @@ -201,6 +200,66 @@ jobs:
echo ${{ env.pr_number }}
echo ${{ env.pr_draft }}
echo ${{ env.pr_labels }}
echo ${{ env.test_status }}
echo ${{ env.test_result }}
echo ${{ env.failures_detected }}
echo ${{ env.approval_status }}

# Remove labels if required
- name: Remove require-backend-approval label
if: ${{ env.exempt == 'true' && contains(env.pr_labels, 'require-backend-approval') }}
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ env.pr_number }}
labels: require-backend-approval

- name: Remove ready-for-backend-review label
if: |
(env.pr_draft || env.test_result == 'failure' || env.failures_detected == 'true' || env.approval_status == 'confirmed') &&
contains(env.pr_labels, 'ready-for-backend-review')
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ env.pr_number }}
labels: ready-for-backend-review

- name: Remove test-passing label
if: ${{ env.test_result == 'failure' && contains(fromJSON(env.pr_labels), 'test-passing') }}
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ env.pr_number }}
labels: test-passing

- name: Remove test-failure label
if: ${{ env.test_result == 'success' && contains(fromJSON(env.pr_labels), 'test-failure') }}
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ env.pr_number }}
labels: test-failure

# Add labels if required
- name: Add require-backend-approval label
if: ${{ env.exempt == 'false'}}
uses: actions-ecosystem/action-add-labels@v1
with:
number: ${{ env.pr_number }}
labels: require-backend-approval

- name: Add ready-for-backend-review label
if: ${{ env.test_result == 'success' && env.failures_detected == 'false' && env.approval_status == 'required' }}
uses: actions-ecosystem/action-add-labels@v1
with:
number: ${{ env.pr_number }}
labels: ready-for-backend-review

- name: Add test-failure label
if: ${{ env.test_result == 'failure' }}
uses: actions-ecosystem/action-add-labels@v1
with:
number: ${{ env.pr_number }}
labels: test-failure

- name: Add test-passing label
if: ${{ env.test_result == 'success' }}
uses: actions-ecosystem/action-add-labels@v1
with:
number: ${{ env.pr_number }}
labels: test-passing
91 changes: 0 additions & 91 deletions .github/workflows/be_review_prs.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/check_codeowners.yml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:

- name: Remove Failure label
uses: actions-ecosystem/action-remove-labels@v1
if: ${{ success() }}
if: ${{ success() && contains(github.event.pull_request.labels.*.name, 'codeowners-addition-failure') }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

notice a quiet failure caused by the workflow attempting to remove a label that wasn't there.

with:
number: ${{ github.event.pull_request.number }}
labels: |
Expand Down
92 changes: 3 additions & 89 deletions .github/workflows/code_checks.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Code Checks
name: "Code Checks"
on:
push:
branches: [master]
Expand All @@ -21,14 +21,6 @@ jobs:
with:
bundler-cache: true

- name: Remove Review label
if: github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'ready-for-backend-review')
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ github.event.pull_request.number }}
labels: |
ready-for-backend-review

- name: Run bundle-audit (checks gems for CVE issues)
run: bundle exec bundle-audit check --update --ignore CVE-2024-27456

Expand Down Expand Up @@ -64,6 +56,8 @@ jobs:
COMPOSE_DOCKER_CLI_BUILD: 1
permissions: write-all
runs-on: ubuntu-32-cores-latest
outputs:
test_status: ${{ steps.test-status.outputs.status }}
steps:
- uses: actions/checkout@v4

Expand Down Expand Up @@ -92,22 +86,6 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Remove Review label
if: github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'ready-for-backend-review')
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ github.event.pull_request.number }}
labels: |
ready-for-backend-review

- name: Remove Test Passing label
if: github.event_name == 'pull_request' && contains(github.event.pull_request.labels.*.name, 'test-passing')
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ github.event.pull_request.number }}
labels: |
test-passing

- name: Build Docker Image
uses: docker/build-push-action@v6
env:
Expand Down Expand Up @@ -155,70 +133,6 @@ jobs:
path: log/*.xml
if-no-files-found: ignore

- name: Add Test Failure label
if: failure() && github.event_name == 'pull_request'
uses: actions-ecosystem/action-add-labels@v1
with:
number: ${{ github.event.pull_request.number }}
labels: |
test-failure

- name: Remove Test Failure label
if: success() && github.event_name == 'pull_request'
uses: actions-ecosystem/action-remove-labels@v1
with:
number: ${{ github.event.pull_request.number }}
labels: |
test-failure

- name: Add Test Passing label
if: success() && github.event_name == 'pull_request'
uses: actions-ecosystem/action-add-labels@v1
with:
number: ${{ github.event.pull_request.number }}
labels: |
test-passing

- name: Add omit-backend-approval label
if: |
github.event_name == 'pull_request' && github.event.pull_request.draft == false &&
(
contains(toJSON(github.event.pull_request.requested_teams.*.name), 'mobile') ||
contains(toJSON(github.event.pull_request.requested_teams.*.name), 'lighthouse') ||
contains(toJSON(github.event.pull_request.requested_teams.*.name), 'identity')
)
uses: actions-ecosystem/action-add-labels@v1
with:
number: ${{ github.event.pull_request.number }}
labels: |
omit-backend-approval

- name: Add require-backend-approval label
uses: actions-ecosystem/action-add-labels@v1
if: |
github.event_name == 'pull_request' && github.event.pull_request.draft == false &&
!contains(toJSON(github.event.pull_request.requested_teams.*.name), 'mobile') &&
!contains(toJSON(github.event.pull_request.requested_teams.*.name), 'lighthouse') &&
!contains(toJSON(github.event.pull_request.requested_teams.*.name), 'identity')
with:
number: ${{ github.event.pull_request.number }}
labels: |
require-backend-approval

- name: Add Review label
uses: actions-ecosystem/action-add-labels@v1
if: |
github.event_name == 'pull_request' && success() && github.event.pull_request.draft == false &&
!contains(github.event.pull_request.labels.*.name, 'code-health-failure') &&
!contains(github.event.pull_request.labels.*.name, 'codeowners-addition-failure') &&
!contains(github.event.pull_request.labels.*.name, 'codeowners-delete-failure') &&
!contains(github.event.pull_request.labels.*.name, 'lint-failure') &&
!contains(github.event.pull_request.labels.*.name, 'test-failure')
with:
number: ${{ github.event.pull_request.number }}
labels: |
ready-for-backend-review

publish_results:
name: Publish Test Results and Coverage
if: always()
Expand Down
Loading
Loading