test: Update test splitting to support new test workflows #35905
+18
−24
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The script responsible for determining how to split up tests in a given workflow was failing when I tried to add a new workflow. It didn't seem to handle the case where a workflow had zero past runs. It has been updated to use a naive fallback splitting algorithm in this case.
📊 Page Load Benchmark ResultsCurrent Commit: 📄 Localhost MetaMask Test DappSamples: 100 Summary
📈 Detailed Results
Results generated automatically by MetaMask CI |
Builds ready [26e1b08]
UI Startup Metrics (1232 ± 70 ms)
Benchmark value 245 exceeds gate value 10 for chrome browserify home mean backgroundConnect Benchmark value 25 exceeds gate value 23 for chrome browserify home mean firstReactRender Benchmark value 5 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 1202 exceeds gate value 1190 for chrome browserify home p95 load Benchmark value 1194 exceeds gate value 1180 for chrome browserify home p95 domContentLoaded Benchmark value 256 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 12 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 946 exceeds gate value 940 for chrome browserify home p95 loadScripts Benchmark value 231 exceeds gate value 195 for chrome webpack home p95 getState Benchmark value 19 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 297 exceeds gate value 65 for chrome webpack home p95 setupStore Benchmark value 1425 exceeds gate value 1405 for firefox browserify home mean uiStartup Benchmark value 36 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 29 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 6 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 10 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 216 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 82 exceeds gate value 70 for firefox browserify home p95 backgroundConnect Benchmark value 11 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 35 exceeds gate value 27 for firefox browserify home p95 setupStore Benchmark value 103 exceeds gate value 100 for firefox webpack home mean domInteractive Benchmark value 32 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 44 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 7 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 259 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 51 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 6 exceeds gate value 2 for firefox webpack home p95 initialActions Sum of mean exceeds: 304ms | Sum of p95 exceeds: 718.8ms Sum of all benchmark exceeds: 1022.8ms Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
7 tasks
7 tasks
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Co-authored-by: Howard Braham <howrad@gmail.com>
📊 Page Load Benchmark ResultsCurrent Commit: 📄 Localhost MetaMask Test DappSamples: 100 Summary
📈 Detailed Results
Results generated automatically by MetaMask CI |
Builds ready [42e588f]
UI Startup Metrics (1220 ± 76 ms)
Benchmark value 24 exceeds gate value 23 for chrome browserify home mean firstReactRender Benchmark value 5 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 1196 exceeds gate value 1190 for chrome browserify home p95 load Benchmark value 1190 exceeds gate value 1180 for chrome browserify home p95 domContentLoaded Benchmark value 42 exceeds gate value 41 for chrome browserify home p95 domInteractive Benchmark value 264 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 11 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 941 exceeds gate value 940 for chrome browserify home p95 loadScripts Benchmark value 19 exceeds gate value 17 for chrome browserify home p95 setupStore Benchmark value 2485 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 221 exceeds gate value 195 for chrome webpack home p95 getState Benchmark value 12 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 1422 exceeds gate value 1405 for firefox browserify home mean uiStartup Benchmark value 35 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 29 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 5 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 10 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 222 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 86 exceeds gate value 70 for firefox browserify home p95 backgroundConnect Benchmark value 14 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 32 exceeds gate value 27 for firefox browserify home p95 setupStore Benchmark value 1649 exceeds gate value 1615 for firefox webpack home mean uiStartup Benchmark value 1408 exceeds gate value 1380 for firefox webpack home mean load Benchmark value 1408 exceeds gate value 1380 for firefox webpack home mean domContentLoaded Benchmark value 107 exceeds gate value 100 for firefox webpack home mean domInteractive Benchmark value 34 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 47 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 6 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 1383 exceeds gate value 1360 for firefox webpack home mean loadScripts Benchmark value 1666 exceeds gate value 1660 for firefox webpack home p95 load Benchmark value 1665 exceeds gate value 1660 for firefox webpack home p95 domContentLoaded Benchmark value 276 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 53 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 11 exceeds gate value 2 for firefox webpack home p95 initialActions Benchmark value 1639 exceeds gate value 1630 for firefox webpack home p95 loadScripts Sum of mean exceeds: 421ms | Sum of p95 exceeds: 549.8ms Sum of all benchmark exceeds: 970.8ms Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
HowardBraham
approved these changes
Sep 16, 2025
itsyoboieltr
approved these changes
Sep 20, 2025
github-project-automation
bot
moved this from Needs dev review
to Review finalised - Ready to be merged
in PR review queue
itsyoboieltr
deleted the
update-tests-splitting-to-support-new-test-workflows
branch
github-project-automation
bot
moved this from Review finalised - Ready to be merged
to Merged, Closed or Archived
in PR review queue
metamaskbot
added
the
release-13.5.0
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The script responsible for determining how to split up tests in a given workflow was failing when I tried to add a new workflow. It didn't seem to handle the case where a workflow had zero past runs.
It has been updated to use a naive fallback splitting algorithm in this case.
Changelog
CHANGELOG entry: null
Related issues
This fixes a CI bug encountered on #31435
Manual testing steps
See this PR for an example of the failures: #35906
See here for a rebased version of that PR on this branch, which works: #35913
Screenshots/Recordings
N/A
Pre-merge author checklist
Pre-merge reviewer checklist