-
Notifications
You must be signed in to change notification settings - Fork 5.4k
ci: add a soft gate for our page load benchmarks #31912
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
6437a73 to
42a4219
Compare
42a4219 to
e4dd141
Compare
Builds ready [e4dd141]
UI Startup Metrics (1206 ± 71 ms)
cc: @HowardBraham |
e4dd141 to
25523c1
Compare
Builds ready [25523c1]
UI Startup Metrics (1212 ± 57 ms)
|
Builds ready [2441535]
UI Startup Metrics (1189 ± 62 ms)
cc: @HowardBraham Benchmark value 1481 exceeds gate value 1480 for firefox browserify home p95 load Benchmark value 1481 exceeds gate value 1480 for firefox browserify home p95 domContentLoaded Benchmark value 1467 exceeds gate value 1465 for firefox browserify home p95 loadScripts |
Builds ready [d115317]
UI Startup Metrics (1209 ± 54 ms)
cc: @HowardBraham |
Builds ready [b726c69]
UI Startup Metrics (1218 ± 63 ms)
cc: @HowardBraham |
|
❌ API Spec Test Failed. View the report here. |
Builds ready [02b69ba]
UI Startup Metrics (1203 ± 62 ms)
cc: @HowardBraham |
Builds ready [1631f57]
UI Startup Metrics (1182 ± 55 ms)
cc: @HowardBraham |
1631f57 to
e4bb26b
Compare
Builds ready [e4bb26b]
UI Startup Metrics (1206 ± 58 ms)
cc: @HowardBraham Bundle size diffs
|
Builds ready [c9d4d65]
UI Startup Metrics (1220 ± 61 ms)
cc: @HowardBraham Bundle size diffs
|
c9d4d65 to
f8c3e75
Compare
Builds ready [f8c3e75]
UI Startup Metrics (1224 ± 48 ms)
cc: @HowardBraham Bundle size diffs
|
f8c3e75 to
7bac480
Compare
Builds ready [7bac480]
UI Startup Metrics (1233 ± 64 ms)
cc: @HowardBraham Bundle size diffs
|
Builds ready [e1d2884]
UI Startup Metrics (1253 ± 77 ms)
cc: @HowardBraham Bundle size diffs
|
e2084b3 to
293b692
Compare
Builds ready [293b692]
UI Startup Metrics (1230 ± 67 ms)
cc: @HowardBraham Bundle size diffs
|
293b692 to
1d389da
Compare
Builds ready [a0c9078]
UI Startup Metrics (1208 ± 57 ms)
Benchmark value 190 exceeds gate value 175 for chrome webpack home mean firstPaint Benchmark value 319 exceeds gate value 310 for chrome webpack home p95 firstPaint Benchmark value 1691 exceeds gate value 1660 for firefox browserify home p95 uiStartup Benchmark value 1506 exceeds gate value 1495 for firefox browserify home p95 load Benchmark value 1506 exceeds gate value 1495 for firefox browserify home p95 domContentLoaded Benchmark value 1493 exceeds gate value 1475 for firefox browserify home p95 loadScripts Sum of mean exceeds: 15ms | Sum of p95 exceeds: 80ms Sum of all benchmark exceeds: 95ms |
cf40d04 to
1d99c59
Compare
Builds ready [1d99c59]
|
1d99c59 to
548f42c
Compare
Builds ready [548f42c]
|
Builds ready [699ac51]
UI Startup Metrics (1311 ± 70 ms)
Benchmark value 1311 exceeds gate value 1234 for chrome browserify home mean uiStartup Benchmark value 1120 exceeds gate value 1070 for chrome browserify home mean load Benchmark value 1109 exceeds gate value 1061 for chrome browserify home mean domContentLoaded Benchmark value 22 exceeds gate value 20 for chrome browserify home mean domInteractive Benchmark value 12 exceeds gate value 10 for chrome browserify home mean backgroundConnect Benchmark value 25 exceeds gate value 23 for chrome browserify home mean firstReactRender Benchmark value 849 exceeds gate value 830 for chrome browserify home mean loadScripts Benchmark value 1434 exceeds gate value 1365 for chrome browserify home p95 uiStartup Benchmark value 1232 exceeds gate value 1190 for chrome browserify home p95 load Benchmark value 1225 exceeds gate value 1180 for chrome browserify home p95 domContentLoaded Benchmark value 71 exceeds gate value 35 for chrome browserify home p95 domInteractive Benchmark value 1203 exceeds gate value 1180 for chrome browserify home p95 firstPaint Benchmark value 23 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 52 exceeds gate value 45 for chrome browserify home p95 firstReactRender Benchmark value 32 exceeds gate value 31 for chrome browserify home p95 getState Benchmark value 965 exceeds gate value 940 for chrome browserify home p95 loadScripts Benchmark value 18 exceeds gate value 17 for chrome browserify home p95 setupStore Benchmark value 184 exceeds gate value 175 for chrome webpack home mean firstPaint Benchmark value 27 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 54 exceeds gate value 45 for firefox webpack home p95 backgroundConnect Sum of mean exceeds: 210ms | Sum of p95 exceeds: 263ms Sum of all benchmark exceeds: 473ms Bundle size diffs
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Description
Adds a gate of do-not-exceed values on every single one of our pageload benchmarks across all configurations (chrome, firefox, browserify, webpack) for mean and p95, across all metrics. (That's 84 gates in total)
As suggested by Mark Stacey, it's a soft gate for now, that does not block the PR, but just pings HowardBraham. Once we merge to main and do more testing on this, we should make it a hard gate.