-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI: add diff-shades integration #2725
Changes from 6 commits
cb3ec99
6b87f86
b797ca4
4f45d2f
5e8b18b
a963292
aab8f28
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,134 @@ | ||
name: diff-shades | ||
|
||
on: | ||
push: | ||
branches: [main] | ||
paths-ignore: ["docs/**", "tests/**", "*.md"] | ||
|
||
pull_request: | ||
path-ignore: ["docs/**", "tests/**", "*.md"] | ||
|
||
workflow_dispatch: | ||
inputs: | ||
baseline: | ||
description: > | ||
The baseline revision. Pro-tip, use `.pypi` to use the latest version | ||
on PyPI or `.XXX` to use a PR. | ||
required: true | ||
default: main | ||
baseline-args: | ||
description: "Custom Black arguments (eg. -l 79)" | ||
required: false | ||
target: | ||
description: > | ||
The target revision to compare against the baseline. Same tip applies here. | ||
required: true | ||
target-args: | ||
description: "Custom Black arguments (eg. -S)" | ||
required: false | ||
|
||
jobs: | ||
analysis: | ||
name: analysis / linux | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
- name: Checkout this repository (full clone) | ||
uses: actions/checkout@v2 | ||
with: | ||
fetch-depth: 0 | ||
|
||
- uses: actions/setup-python@v2 | ||
|
||
- name: Install diff-shades and support dependencies | ||
run: | | ||
python -m pip install pip --upgrade | ||
python -m pip install https://github.com/ichard26/diff-shades/archive/stable.zip | ||
python -m pip install click packaging urllib3 | ||
# After checking out old revisions, this might not exist so we'll use a copy. | ||
cat scripts/diff_shades_gha_helper.py > helper.py | ||
git config user.name "diff-shades-gha" | ||
git config user.email "diff-shades-gha@example.com" | ||
|
||
- name: Calculate run configuration & metadata | ||
id: config | ||
env: | ||
GITHUB_TOKEN: ${{ github.token }} | ||
run: > | ||
python helper.py config ${{ github.event_name }} | ||
${{ github.event.inputs.baseline }} ${{ github.event.inputs.target }} | ||
--baseline-args "${{ github.event.inputs.baseline-args }}" | ||
|
||
- name: Attempt to use cached baseline analysis | ||
id: baseline-cache | ||
uses: actions/cache@v2.1.7 | ||
with: | ||
path: ${{ steps.config.outputs.baseline-analysis }} | ||
key: ${{ steps.config.outputs.baseline-cache-key }} | ||
|
||
- name: Install baseline revision | ||
if: steps.baseline-cache.outputs.cache-hit != 'true' | ||
env: | ||
GITHUB_TOKEN: ${{ github.token }} | ||
run: ${{ steps.config.outputs.baseline-setup-cmd }} && python -m pip install . | ||
|
||
- name: Analyze baseline revision | ||
if: steps.baseline-cache.outputs.cache-hit != 'true' | ||
run: > | ||
diff-shades analyze -v --work-dir projects-cache/ | ||
${{ steps.config.outputs.baseline-analysis }} -- ${{ github.event.inputs.baseline-args }} | ||
|
||
- name: Install target revision | ||
env: | ||
GITHUB_TOKEN: ${{ github.token }} | ||
run: ${{ steps.config.outputs.target-setup-cmd }} && python -m pip install . | ||
|
||
- name: Analyze target revision | ||
run: > | ||
diff-shades analyze -v --work-dir projects-cache/ | ||
${{ steps.config.outputs.target-analysis }} --repeat-projects-from | ||
${{ steps.config.outputs.baseline-analysis }} -- ${{ github.event.inputs.target-args }} | ||
|
||
- name: Generate HTML diff report | ||
run: > | ||
diff-shades --dump-html diff.html compare --diff --quiet | ||
${{ steps.config.outputs.baseline-analysis }} ${{ steps.config.outputs.target-analysis }} | ||
|
||
- name: Upload diff report | ||
uses: actions/upload-artifact@v2 | ||
with: | ||
name: diff.html | ||
path: diff.html | ||
|
||
- name: Upload baseline analysis | ||
uses: actions/upload-artifact@v2 | ||
with: | ||
name: ${{ steps.config.outputs.baseline-analysis }} | ||
path: ${{ steps.config.outputs.baseline-analysis }} | ||
|
||
- name: Upload target analysis | ||
uses: actions/upload-artifact@v2 | ||
with: | ||
name: ${{ steps.config.outputs.target-analysis }} | ||
path: ${{ steps.config.outputs.target-analysis }} | ||
|
||
- name: Generate summary file (PR only) | ||
if: github.event_name == 'pull_request' | ||
run: > | ||
python helper.py comment-body | ||
${{ steps.config.outputs.baseline-analysis }} ${{ steps.config.outputs.target-analysis }} | ||
${{ steps.config.outputs.baseline-sha }} ${{ steps.config.outputs.target-sha }} | ||
|
||
- name: Upload summary file (PR only) | ||
if: github.event_name == 'pull_request' | ||
uses: actions/upload-artifact@v2 | ||
with: | ||
name: .pr-comment-body.md | ||
path: .pr-comment-body.md | ||
|
||
# This is last so the diff-shades-comment workflow can still work even if we | ||
# end up detecting failed files and failing the run. | ||
Comment on lines
+129
to
+130
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ? I don't understand this, why is it last? Will it cause the other workflow to fail? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Even if while running black over files a file does fail, we still want the diff-shades-comment workflow to fire and work successfully, that can't happen if this step fails skipping the PR body upload step. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
- name: Check for failed files in both analyses | ||
run: > | ||
diff-shades show-failed --check --show-log ${{ steps.config.outputs.baseline-analysis }}; | ||
diff-shades show-failed --check --show-log ${{ steps.config.outputs.target-analysis }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
name: diff-shades-comment | ||
|
||
on: | ||
workflow_run: | ||
workflows: [diff-shades] | ||
types: [completed] | ||
|
||
permissions: | ||
pull-requests: write | ||
|
||
jobs: | ||
comment: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v2 | ||
- uses: actions/setup-python@v2 | ||
|
||
- name: Install support dependencies | ||
run: | | ||
python -m pip install pip --upgrade | ||
python -m pip install click packaging urllib3 | ||
|
||
- name: Get details from initial workflow run | ||
id: metadata | ||
env: | ||
GITHUB_TOKEN: ${{ github.token }} | ||
run: > | ||
python scripts/diff_shades_gha_helper.py comment-details | ||
${{github.event.workflow_run.id }} | ||
|
||
- name: Try to find pre-existing PR comment | ||
if: steps.metadata.outputs.needs-comment == 'true' | ||
id: find-comment | ||
uses: peter-evans/find-comment@v1 | ||
with: | ||
issue-number: ${{ steps.metadata.outputs.pr-number }} | ||
comment-author: "github-actions[bot]" | ||
body-includes: "diff-shades" | ||
|
||
- name: Create or update PR comment | ||
if: steps.metadata.outputs.needs-comment == 'true' | ||
uses: peter-evans/create-or-update-comment@v1 | ||
with: | ||
comment-id: ${{ steps.find-comment.outputs.comment-id }} | ||
issue-number: ${{ steps.metadata.outputs.pr-number }} | ||
body: ${{ steps.metadata.outputs.comment-body }} | ||
edit-mode: replace |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -57,7 +57,7 @@ repos: | |
rev: v2.5.1 | ||
hooks: | ||
- id: prettier | ||
exclude: ^Pipfile\.lock | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this should still exclude pip file lock, no? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We deleted it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doesn't exist after #2717. |
||
exclude: \.github/workflows/diff_shades\.yml | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ^ and $ should be here too There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sure, PR it |
||
|
||
- repo: https://github.com/pre-commit/pre-commit-hooks | ||
rev: v4.0.1 | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is it a full clone?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The workflow checkouts older revisions of black and it's not known until later which revisions are needed