Skip to content

Commit 0ef324b

Browse files
authored
[ty] ecosystem-analyzer workflow (#18719)
## Summary Adds a new ecosystem-analyzer workflow with a similar purpose to the mypy-primer workflow. It creates a richer ecosystem diff report using [ecosystem-analyzer](https://github.com/astral-sh/ecosystem-analyzer/) ([example report](https://shark.fish/diff-attr-subscript-narrowing.html)). This is still experimental and also quite a bit slower than mypy_primer, so I chose to make this opt-in for now via a `ecosystem-analyzer` label. This would give us a way to play with this while still evaluating if we should further invest in this or not. Advantages over the mypy_primer diff output: - Interactive filtering of diagnostics - Statistics overview which breaks down added/removed/changed diagnostics across lint rules - Has the concept of "changed" diagnostics, which makes it easier to review changes where diagnostic messages have changed (along with other changes). - Compute diff based on old and new project-lists (`good.txt`). This allows us to diff changes to the project list itself. This has caused confusion in the past where we tried to add new projects to `good.txt`, but then ran the `main`-branch version of ty on that new list (where the bug was not yet fixed) Disadvantages: - The report currently needs to be downloaded from the workflow run, as I don't know if we have a way of deploying HTML files like this temporarily to some hosted infrastructure.
1 parent 7982eda commit 0ef324b

File tree

1 file changed

+94
-0
lines changed

1 file changed

+94
-0
lines changed
Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
name: ty ecosystem-analyzer
2+
3+
permissions: {}
4+
5+
on:
6+
pull_request:
7+
types: [labeled]
8+
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
11+
cancel-in-progress: true
12+
13+
env:
14+
CARGO_INCREMENTAL: 0
15+
CARGO_NET_RETRY: 10
16+
CARGO_TERM_COLOR: always
17+
RUSTUP_MAX_RETRIES: 10
18+
RUST_BACKTRACE: 1
19+
REF_NAME: ${{ github.ref_name }}
20+
21+
jobs:
22+
ty-ecosystem-analyzer:
23+
name: Compute diagnostic diff
24+
runs-on: depot-ubuntu-22.04-32
25+
timeout-minutes: 20
26+
if: contains(github.event.label.name, 'ecosystem-analyzer')
27+
steps:
28+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
29+
with:
30+
path: ruff
31+
fetch-depth: 0
32+
persist-credentials: false
33+
34+
- name: Install the latest version of uv
35+
uses: astral-sh/setup-uv@f0ec1fc3b38f5e7cd731bb6ce540c5af426746bb # v6.1.0
36+
37+
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
38+
with:
39+
workspaces: "ruff"
40+
41+
- name: Install Rust toolchain
42+
run: rustup show
43+
44+
- name: Compute diagnostic diff
45+
shell: bash
46+
run: |
47+
cd ruff
48+
49+
echo "Enabling configuration overloads (see .github/mypy-primer-ty.toml)"
50+
mkdir -p ~/.config/ty
51+
cp .github/mypy-primer-ty.toml ~/.config/ty/ty.toml
52+
53+
echo "new commit"
54+
git checkout -b new_commit "$GITHUB_SHA"
55+
git rev-list --format=%s --max-count=1 new_commit
56+
cp crates/ty_python_semantic/resources/primer/good.txt projects_new.txt
57+
58+
echo "old commit (merge base)"
59+
MERGE_BASE="$(git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF")"
60+
git checkout -b old_commit "$MERGE_BASE"
61+
git rev-list --format=%s --max-count=1 old_commit
62+
cp crates/ty_python_semantic/resources/primer/good.txt projects_old.txt
63+
64+
cd ..
65+
66+
uv tool install "git+https://github.com/astral-sh/ecosystem-analyzer@9c34dc514ee9aef6735db1dfebb80f63acbc3440"
67+
68+
ecosystem-analyzer \
69+
--repository ruff \
70+
analyze \
71+
--projects ruff/projects_old.txt \
72+
--commit old_commit \
73+
--output diagnostics_old.json
74+
75+
ecosystem-analyzer \
76+
--repository ruff \
77+
analyze \
78+
--projects ruff/projects_new.txt \
79+
--commit new_commit \
80+
--output diagnostics_new.json
81+
82+
ecosystem-analyzer \
83+
generate-diff \
84+
diagnostics_old.json \
85+
diagnostics_new.json \
86+
--old-name "main (merge base)" \
87+
--new-name "$REF_NAME" \
88+
--output-html diff.html
89+
90+
- name: Upload HTML diff report
91+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
92+
with:
93+
name: diff.html
94+
path: diff.html

0 commit comments

Comments
 (0)