test: Add E2E test for first install #31435
Draft
+101
−13
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
2 times, most recently
from
897848a to
5efe7d9
Compare
Builds ready [5efe7d9]
UI Startup Metrics (1193 ± 56 ms)
Bundle size diffs
|
7 tasks
Comment on lines
+289
to
+362
| /** | ||
| * Reload the extension. | ||
| * | ||
| * This is used for the `first-install` E2E test, which uses a production-like build. This | ||
| * function must be present even if `process.env.IN_TEST` is false. | ||
| */ | ||
| window.stateHooks.reloadExtension = () => { | ||
| browser.runtime.reload(); | ||
| }; |
There was a problem hiding this comment.
Just a quick drive by comment: Since we use browser.runtime.reload() in a few places in our code, it might make sense to update those to use this new stateHooks.reloadExtension line... that way we can say we aren't shipping test functions in production 😆
There was a problem hiding this comment.
Hmm, interesting idea, but it would probably be better to minimize the reliance on functions we make accessible outside the root compartment. Just to minimize the complexity of any audits into what could go wrong if they're tampered with somehow.
davidmurdoch
previously approved these changes
Mar 31, 2025
seaona
reviewed
Apr 1, 2025
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
5efe7d9 to
837e3b0
Compare
Builds ready [837e3b0]
UI Startup Metrics (1195 ± 66 ms)
|
|
Some changes needed here due to #31363, which migrated the vault decryptor test to GHA |
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
81c10ba to
cbc2979
Compare
Builds ready [cbc2979]
UI Startup Metrics (1204 ± 63 ms)
|
|
This PR has been automatically marked as stale because it has not had recent activity in the last 90 days. It will be closed in 7 days. Thank you for your contributions. |
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
8b7cc66 to
2f84708
Compare
Builds ready [2f84708]
UI Startup Metrics (1270 ± 59 ms)
Benchmark value 1091 exceeds gate value 1070 for chrome browserify home mean load Benchmark value 1083 exceeds gate value 1061 for chrome browserify home mean domContentLoaded Benchmark value 212 exceeds gate value 10 for chrome browserify home mean backgroundConnect Benchmark value 3 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 879 exceeds gate value 830 for chrome browserify home mean loadScripts Benchmark value 1378 exceeds gate value 1365 for chrome browserify home p95 uiStartup Benchmark value 1191 exceeds gate value 1190 for chrome browserify home p95 load Benchmark value 1184 exceeds gate value 1180 for chrome browserify home p95 domContentLoaded Benchmark value 229 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 12 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 978 exceeds gate value 940 for chrome browserify home p95 loadScripts Benchmark value 2393 exceeds gate value 2192 for chrome webpack home mean uiStartup Benchmark value 1888 exceeds gate value 1711 for chrome webpack home mean load Benchmark value 1874 exceeds gate value 1704 for chrome webpack home mean domContentLoaded Benchmark value 22 exceeds gate value 21 for chrome webpack home mean domInteractive Benchmark value 57 exceeds gate value 40 for chrome webpack home mean backgroundConnect Benchmark value 36 exceeds gate value 29 for chrome webpack home mean getState Benchmark value 10 exceeds gate value 7 for chrome webpack home mean initialActions Benchmark value 1867 exceeds gate value 1699 for chrome webpack home mean loadScripts Benchmark value 2766 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 2190 exceeds gate value 2030 for chrome webpack home p95 load Benchmark value 2166 exceeds gate value 2005 for chrome webpack home p95 domContentLoaded Benchmark value 63 exceeds gate value 57 for chrome webpack home p95 domInteractive Benchmark value 284 exceeds gate value 90 for chrome webpack home p95 backgroundConnect Benchmark value 250 exceeds gate value 195 for chrome webpack home p95 getState Benchmark value 17 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 2161 exceeds gate value 1970 for chrome webpack home p95 loadScripts Benchmark value 191 exceeds gate value 65 for chrome webpack home p95 setupStore Benchmark value 1545 exceeds gate value 1405 for firefox browserify home mean uiStartup Benchmark value 1340 exceeds gate value 1245 for firefox browserify home mean load Benchmark value 1340 exceeds gate value 1239 for firefox browserify home mean domContentLoaded Benchmark value 123 exceeds gate value 110 for firefox browserify home mean domInteractive Benchmark value 27 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 29 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 13 exceeds gate value 11 for firefox browserify home mean getState Benchmark value 4 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 1318 exceeds gate value 1230 for firefox browserify home mean loadScripts Benchmark value 12 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 1784 exceeds gate value 1660 for firefox browserify home p95 uiStartup Benchmark value 1498 exceeds gate value 1495 for firefox browserify home p95 load Benchmark value 1498 exceeds gate value 1495 for firefox browserify home p95 domContentLoaded Benchmark value 338 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 58 exceeds gate value 55 for firefox browserify home p95 firstReactRender Benchmark value 43 exceeds gate value 24 for firefox browserify home p95 getState Benchmark value 14 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 1478 exceeds gate value 1475 for firefox browserify home p95 loadScripts Benchmark value 48 exceeds gate value 27 for firefox browserify home p95 setupStore Benchmark value 1774 exceeds gate value 1615 for firefox webpack home mean uiStartup Benchmark value 1490 exceeds gate value 1380 for firefox webpack home mean load Benchmark value 1490 exceeds gate value 1380 for firefox webpack home mean domContentLoaded Benchmark value 27 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 52 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 10 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 1467 exceeds gate value 1360 for firefox webpack home mean loadScripts Benchmark value 23 exceeds gate value 13 for firefox webpack home mean setupStore Benchmark value 2105 exceeds gate value 1935 for firefox webpack home p95 uiStartup Benchmark value 1751 exceeds gate value 1660 for firefox webpack home p95 load Benchmark value 1750 exceeds gate value 1660 for firefox webpack home p95 domContentLoaded Benchmark value 59 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 39 exceeds gate value 2 for firefox webpack home p95 initialActions Benchmark value 1726 exceeds gate value 1630 for firefox webpack home p95 loadScripts Benchmark value 195 exceeds gate value 28 for firefox webpack home p95 setupStore Sum of mean exceeds: 2048ms | Sum of p95 exceeds: 2483.8ms Sum of all benchmark exceeds: 4531.8ms Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
5dd8e9c to
2dc0bfb
Compare
Builds ready [2dc0bfb]
UI Startup Metrics (1273 ± 60 ms)
Benchmark value 1093 exceeds gate value 1070 for chrome browserify home mean load Benchmark value 1086 exceeds gate value 1061 for chrome browserify home mean domContentLoaded Benchmark value 211 exceeds gate value 10 for chrome browserify home mean backgroundConnect Benchmark value 3 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 882 exceeds gate value 830 for chrome browserify home mean loadScripts Benchmark value 1387 exceeds gate value 1365 for chrome browserify home p95 uiStartup Benchmark value 1181 exceeds gate value 1180 for chrome browserify home p95 domContentLoaded Benchmark value 226 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 12 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 971 exceeds gate value 940 for chrome browserify home p95 loadScripts Benchmark value 2419 exceeds gate value 2192 for chrome webpack home mean uiStartup Benchmark value 1912 exceeds gate value 1711 for chrome webpack home mean load Benchmark value 1897 exceeds gate value 1704 for chrome webpack home mean domContentLoaded Benchmark value 26 exceeds gate value 21 for chrome webpack home mean domInteractive Benchmark value 57 exceeds gate value 40 for chrome webpack home mean backgroundConnect Benchmark value 10 exceeds gate value 7 for chrome webpack home mean initialActions Benchmark value 1893 exceeds gate value 1699 for chrome webpack home mean loadScripts Benchmark value 2873 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 2357 exceeds gate value 2030 for chrome webpack home p95 load Benchmark value 2274 exceeds gate value 2005 for chrome webpack home p95 domContentLoaded Benchmark value 117 exceeds gate value 57 for chrome webpack home p95 domInteractive Benchmark value 391 exceeds gate value 334 for chrome webpack home p95 firstPaint Benchmark value 281 exceeds gate value 90 for chrome webpack home p95 backgroundConnect Benchmark value 18 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 2265 exceeds gate value 1970 for chrome webpack home p95 loadScripts Benchmark value 1539 exceeds gate value 1405 for firefox browserify home mean uiStartup Benchmark value 1330 exceeds gate value 1245 for firefox browserify home mean load Benchmark value 1330 exceeds gate value 1239 for firefox browserify home mean domContentLoaded Benchmark value 131 exceeds gate value 110 for firefox browserify home mean domInteractive Benchmark value 31 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 32 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 4 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 1307 exceeds gate value 1230 for firefox browserify home mean loadScripts Benchmark value 12 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 1897 exceeds gate value 1660 for firefox browserify home p95 uiStartup Benchmark value 1587 exceeds gate value 1495 for firefox browserify home p95 load Benchmark value 1587 exceeds gate value 1495 for firefox browserify home p95 domContentLoaded Benchmark value 460 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 65 exceeds gate value 55 for firefox browserify home p95 firstReactRender Benchmark value 39 exceeds gate value 24 for firefox browserify home p95 getState Benchmark value 14 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 1495 exceeds gate value 1475 for firefox browserify home p95 loadScripts Benchmark value 29 exceeds gate value 27 for firefox browserify home p95 setupStore Benchmark value 1772 exceeds gate value 1615 for firefox webpack home mean uiStartup Benchmark value 1489 exceeds gate value 1380 for firefox webpack home mean load Benchmark value 1489 exceeds gate value 1380 for firefox webpack home mean domContentLoaded Benchmark value 31 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 54 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 16 exceeds gate value 15 for firefox webpack home mean getState Benchmark value 10 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 1465 exceeds gate value 1360 for firefox webpack home mean loadScripts Benchmark value 18 exceeds gate value 13 for firefox webpack home mean setupStore Benchmark value 2146 exceeds gate value 1935 for firefox webpack home p95 uiStartup Benchmark value 1793 exceeds gate value 1660 for firefox webpack home p95 load Benchmark value 1793 exceeds gate value 1660 for firefox webpack home p95 domContentLoaded Benchmark value 186 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 50 exceeds gate value 49 for firefox webpack home p95 backgroundConnect Benchmark value 67 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 38 exceeds gate value 32 for firefox webpack home p95 getState Benchmark value 23 exceeds gate value 2 for firefox webpack home p95 initialActions Benchmark value 1766 exceeds gate value 1630 for firefox webpack home p95 loadScripts Benchmark value 66 exceeds gate value 28 for firefox webpack home p95 setupStore Sum of mean exceeds: 2125ms | Sum of p95 exceeds: 3372.8ms Sum of all benchmark exceeds: 5497.8ms Bundle size diffs [🚀 Bundle size reduced!]
|
Builds ready [d38cb3d]
UI Startup Metrics (1315 ± 65 ms)
Benchmark value 1136 exceeds gate value 1070 for chrome browserify home mean load Benchmark value 1128 exceeds gate value 1061 for chrome browserify home mean domContentLoaded Benchmark value 810 exceeds gate value 800 for chrome browserify home mean firstPaint Benchmark value 257 exceeds gate value 10 for chrome browserify home mean backgroundConnect Benchmark value 4 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 879 exceeds gate value 830 for chrome browserify home mean loadScripts Benchmark value 1476 exceeds gate value 1365 for chrome browserify home p95 uiStartup Benchmark value 1274 exceeds gate value 1190 for chrome browserify home p95 load Benchmark value 1256 exceeds gate value 1180 for chrome browserify home p95 domContentLoaded Benchmark value 42 exceeds gate value 41 for chrome browserify home p95 domInteractive Benchmark value 1224 exceeds gate value 1180 for chrome browserify home p95 firstPaint Benchmark value 275 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 15 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 988 exceeds gate value 940 for chrome browserify home p95 loadScripts Benchmark value 2451 exceeds gate value 2192 for chrome webpack home mean uiStartup Benchmark value 1939 exceeds gate value 1711 for chrome webpack home mean load Benchmark value 1928 exceeds gate value 1704 for chrome webpack home mean domContentLoaded Benchmark value 25 exceeds gate value 21 for chrome webpack home mean domInteractive Benchmark value 56 exceeds gate value 40 for chrome webpack home mean backgroundConnect Benchmark value 37 exceeds gate value 29 for chrome webpack home mean getState Benchmark value 10 exceeds gate value 7 for chrome webpack home mean initialActions Benchmark value 1919 exceeds gate value 1699 for chrome webpack home mean loadScripts Benchmark value 2818 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 2237 exceeds gate value 2030 for chrome webpack home p95 load Benchmark value 2223 exceeds gate value 2005 for chrome webpack home p95 domContentLoaded Benchmark value 69 exceeds gate value 57 for chrome webpack home p95 domInteractive Benchmark value 358 exceeds gate value 334 for chrome webpack home p95 firstPaint Benchmark value 279 exceeds gate value 90 for chrome webpack home p95 backgroundConnect Benchmark value 244 exceeds gate value 195 for chrome webpack home p95 getState Benchmark value 15 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 2209 exceeds gate value 1970 for chrome webpack home p95 loadScripts Benchmark value 1482 exceeds gate value 1405 for firefox browserify home mean uiStartup Benchmark value 1279 exceeds gate value 1245 for firefox browserify home mean load Benchmark value 1278 exceeds gate value 1239 for firefox browserify home mean domContentLoaded Benchmark value 29 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 13 exceeds gate value 11 for firefox browserify home mean getState Benchmark value 9 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 1257 exceeds gate value 1230 for firefox browserify home mean loadScripts Benchmark value 11 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 1747 exceeds gate value 1660 for firefox browserify home p95 uiStartup Benchmark value 256 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 60 exceeds gate value 55 for firefox browserify home p95 firstReactRender Benchmark value 41 exceeds gate value 24 for firefox browserify home p95 getState Benchmark value 37 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 34 exceeds gate value 27 for firefox browserify home p95 setupStore Benchmark value 1736 exceeds gate value 1615 for firefox webpack home mean uiStartup Benchmark value 1457 exceeds gate value 1380 for firefox webpack home mean load Benchmark value 1457 exceeds gate value 1380 for firefox webpack home mean domContentLoaded Benchmark value 29 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 52 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 8 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 1434 exceeds gate value 1360 for firefox webpack home mean loadScripts Benchmark value 21 exceeds gate value 13 for firefox webpack home mean setupStore Benchmark value 2103 exceeds gate value 1935 for firefox webpack home p95 uiStartup Benchmark value 1778 exceeds gate value 1660 for firefox webpack home p95 load Benchmark value 1778 exceeds gate value 1660 for firefox webpack home p95 domContentLoaded Benchmark value 304 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 64 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 22 exceeds gate value 2 for firefox webpack home p95 initialActions Benchmark value 1748 exceeds gate value 1630 for firefox webpack home p95 loadScripts Benchmark value 114 exceeds gate value 28 for firefox webpack home p95 setupStore Sum of mean exceeds: 2060ms | Sum of p95 exceeds: 2946.8ms Sum of all benchmark exceeds: 5006.8ms Bundle size diffs [🚀 Bundle size reduced!]
|
Gudahtt
added a commit
that referenced
this pull request
Jul 31, 2025
Our E2E driver class now allows the methods `switchWindow` and `switchToWindowWithTitle` methods to be used in tests that have the Mocha background server disabled (such as those that use a production build, or those that involve the extension resetting). Previously the `switchToWindowWithTitleWithoutSocket` driver method served this purpose, but the test author had to know to use this alternative method. Plus it didn't allow switching to a window handle, or any of the other switch methods that internally use `switchWindow`. There are a couple of switch methods still not supported, but in these cases an error will be thrown so that the failure reason will be clear. This unblocks #31435
7 tasks
Gudahtt
added a commit
that referenced
this pull request
Jul 31, 2025
Our E2E driver class now allows the methods `switchWindow` and `switchToWindowWithTitle` methods to be used in tests that have the Mocha background server disabled (such as those that use a production build, or those that involve the extension resetting). Previously the `switchToWindowWithTitleWithoutSocket` driver method served this purpose, but the test author had to know to use this alternative method. Plus it didn't allow switching to a window handle, or any of the other switch methods that internally use `switchWindow`. There are a couple of switch methods still not supported, but in these cases an error will be thrown so that the failure reason will be clear. This unblocks #31435
Gudahtt
added a commit
that referenced
this pull request
Aug 1, 2025
Our E2E driver class now allows the methods `switchWindow` and `switchToWindowWithTitle` methods to be used in tests that have the Mocha background server disabled (such as those that use a production build, or those that involve the extension resetting). Previously the `switchToWindowWithTitleWithoutSocket` driver method served this purpose, but the test author had to know to use this alternative method. Plus it didn't allow switching to a window handle, or any of the other switch methods that internally use `switchWindow`. There are a couple of switch methods still not supported, but in these cases an error will be thrown so that the failure reason will be clear. This unblocks #31435
Gudahtt
added a commit
that referenced
this pull request
Aug 1, 2025
Our E2E driver class now allows the methods `switchWindow` and `switchToWindowWithTitle` methods to be used in tests that have the Mocha background server disabled (such as those that use a production build, or those that involve the extension resetting). Previously the `switchToWindowWithTitleWithoutSocket` driver method served this purpose, but the test author had to know to use this alternative method. Plus it didn't allow switching to a window handle, or any of the other switch methods that internally use `switchWindow`. There are a couple of switch methods still not supported, but in these cases an error will be thrown so that the failure reason will be clear. This unblocks #31435
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 6, 2025
#34772) ## **Description** Our E2E driver class now allows the methods `switchWindow` and `switchToWindowWithTitle` methods to be used in tests that have the WebSocket connection to the background process disabled (such as those that use a production build, or those that involve the extension resetting). Previously the `switchToWindowWithTitleWithoutSocket` driver method served this purpose, but the test author had to know to use this alternative method. Plus it didn't allow switching to a window handle, or any of the other switch methods that internally use `switchWindow`. There are a couple of switch methods still not supported, but in these cases an error will be thrown so that the failure reason will be clear. [](https://codespaces.new/MetaMask/metamask-extension/pull/34772?quickstart=1) ## **Changelog** CHANGELOG entry: null ## **Related issues** This unblocks #31435 ## **Manual testing steps** N/A ## **Screenshots/Recordings** N/A ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Extension Coding Standards](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I’ve included tests if applicable - [x] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.
mcmire
pushed a commit
that referenced
this pull request
Aug 11, 2025
#34772) ## **Description** Our E2E driver class now allows the methods `switchWindow` and `switchToWindowWithTitle` methods to be used in tests that have the WebSocket connection to the background process disabled (such as those that use a production build, or those that involve the extension resetting). Previously the `switchToWindowWithTitleWithoutSocket` driver method served this purpose, but the test author had to know to use this alternative method. Plus it didn't allow switching to a window handle, or any of the other switch methods that internally use `switchWindow`. There are a couple of switch methods still not supported, but in these cases an error will be thrown so that the failure reason will be clear. [](https://codespaces.new/MetaMask/metamask-extension/pull/34772?quickstart=1) ## **Changelog** CHANGELOG entry: null ## **Related issues** This unblocks #31435 ## **Manual testing steps** N/A ## **Screenshots/Recordings** N/A ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Extension Coding Standards](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I’ve included tests if applicable - [x] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
d38cb3d to
02d8745
Compare
Builds ready [02d8745]
UI Startup Metrics (1240 ± 79 ms)
Benchmark value 1089 exceeds gate value 1070 for chrome browserify home mean load Benchmark value 1083 exceeds gate value 1061 for chrome browserify home mean domContentLoaded Benchmark value 240 exceeds gate value 10 for chrome browserify home mean backgroundConnect Benchmark value 24 exceeds gate value 23 for chrome browserify home mean firstReactRender Benchmark value 4 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 850 exceeds gate value 830 for chrome browserify home mean loadScripts Benchmark value 1378 exceeds gate value 1365 for chrome browserify home p95 uiStartup Benchmark value 1234 exceeds gate value 1190 for chrome browserify home p95 load Benchmark value 1226 exceeds gate value 1180 for chrome browserify home p95 domContentLoaded Benchmark value 1197 exceeds gate value 1180 for chrome browserify home p95 firstPaint Benchmark value 251 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 14 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 995 exceeds gate value 940 for chrome browserify home p95 loadScripts Benchmark value 2294 exceeds gate value 2192 for chrome webpack home mean uiStartup Benchmark value 1788 exceeds gate value 1711 for chrome webpack home mean load Benchmark value 1779 exceeds gate value 1704 for chrome webpack home mean domContentLoaded Benchmark value 1775 exceeds gate value 1699 for chrome webpack home mean loadScripts Benchmark value 2757 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 2041 exceeds gate value 2030 for chrome webpack home p95 load Benchmark value 2035 exceeds gate value 2005 for chrome webpack home p95 domContentLoaded Benchmark value 63 exceeds gate value 57 for chrome webpack home p95 domInteractive Benchmark value 358 exceeds gate value 334 for chrome webpack home p95 firstPaint Benchmark value 91 exceeds gate value 90 for chrome webpack home p95 backgroundConnect Benchmark value 394 exceeds gate value 370 for chrome webpack home p95 firstReactRender Benchmark value 9 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 2022 exceeds gate value 1970 for chrome webpack home p95 loadScripts Benchmark value 29 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 28 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 4 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 10 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 237 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 10 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 1685 exceeds gate value 1615 for firefox webpack home mean uiStartup Benchmark value 1431 exceeds gate value 1380 for firefox webpack home mean load Benchmark value 1430 exceeds gate value 1380 for firefox webpack home mean domContentLoaded Benchmark value 115 exceeds gate value 100 for firefox webpack home mean domInteractive Benchmark value 32 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 62 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 6 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 1407 exceeds gate value 1360 for firefox webpack home mean loadScripts Benchmark value 2045 exceeds gate value 1935 for firefox webpack home p95 uiStartup Benchmark value 1705 exceeds gate value 1660 for firefox webpack home p95 load Benchmark value 1704 exceeds gate value 1660 for firefox webpack home p95 domContentLoaded Benchmark value 289 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 74 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 16 exceeds gate value 2 for firefox webpack home p95 initialActions Benchmark value 1680 exceeds gate value 1630 for firefox webpack home p95 loadScripts Sum of mean exceeds: 910ms | Sum of p95 exceeds: 1343.8ms Sum of all benchmark exceeds: 2253.8ms Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
📊 Page Load Benchmark ResultsCurrent Commit: 📄 https://metamask.github.io/test-dapp/Samples: 100 Summary
📈 Detailed Results
Results generated automatically by MetaMask CI |
Builds ready [5d5a50d]
UI Startup Metrics (1200 ± 66 ms)
Benchmark value 3 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 250 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 11 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 2474 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 389 exceeds gate value 370 for chrome webpack home p95 firstReactRender Benchmark value 265 exceeds gate value 195 for chrome webpack home p95 getState Benchmark value 8 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 32 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 28 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 3 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 11 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 222 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 71 exceeds gate value 70 for firefox browserify home p95 backgroundConnect Benchmark value 8 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 29 exceeds gate value 27 for firefox browserify home p95 setupStore Benchmark value 106 exceeds gate value 100 for firefox webpack home mean domInteractive Benchmark value 31 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 58 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 5 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 248 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 68 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 16 exceeds gate value 2 for firefox webpack home p95 initialActions Sum of mean exceeds: 280ms | Sum of p95 exceeds: 511.8ms Sum of all benchmark exceeds: 791.8ms Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
5d5a50d to
4820010
Compare
📊 Page Load Benchmark ResultsCurrent Commit: 📄 https://metamask.github.io/test-dapp/Samples: 100 Summary
📈 Detailed Results
Results generated automatically by MetaMask CI |
Builds ready [4820010]
UI Startup Metrics (1194 ± 66 ms)
Benchmark value 28 exceeds gate value 23 for chrome browserify home mean firstReactRender Benchmark value 4 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 257 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 48 exceeds gate value 45 for chrome browserify home p95 firstReactRender Benchmark value 13 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 19 exceeds gate value 17 for chrome browserify home p95 setupStore Benchmark value 31 exceeds gate value 29 for chrome webpack home mean getState Benchmark value 2505 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 385 exceeds gate value 370 for chrome webpack home p95 firstReactRender Benchmark value 268 exceeds gate value 195 for chrome webpack home p95 getState Benchmark value 8 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 1406 exceeds gate value 1405 for firefox browserify home mean uiStartup Benchmark value 115 exceeds gate value 110 for firefox browserify home mean domInteractive Benchmark value 35 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 27 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 4 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 10 exceeds gate value 9 for firefox browserify home mean setupStore Benchmark value 1718 exceeds gate value 1660 for firefox browserify home p95 uiStartup Benchmark value 263 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 91 exceeds gate value 70 for firefox browserify home p95 backgroundConnect Benchmark value 32 exceeds gate value 24 for firefox browserify home p95 getState Benchmark value 10 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 1688 exceeds gate value 1615 for firefox webpack home mean uiStartup Benchmark value 1443 exceeds gate value 1380 for firefox webpack home mean load Benchmark value 1442 exceeds gate value 1380 for firefox webpack home mean domContentLoaded Benchmark value 126 exceeds gate value 100 for firefox webpack home mean domInteractive Benchmark value 32 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 63 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 4 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 1419 exceeds gate value 1360 for firefox webpack home mean loadScripts Benchmark value 2004 exceeds gate value 1935 for firefox webpack home p95 uiStartup Benchmark value 1711 exceeds gate value 1660 for firefox webpack home p95 load Benchmark value 1711 exceeds gate value 1660 for firefox webpack home p95 domContentLoaded Benchmark value 296 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 79 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 14 exceeds gate value 2 for firefox webpack home p95 initialActions Benchmark value 1686 exceeds gate value 1630 for firefox webpack home p95 loadScripts Sum of mean exceeds: 578ms | Sum of p95 exceeds: 966.8ms Sum of all benchmark exceeds: 1544.8ms Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
2 times, most recently
from
7e3b9fc to
a6e0aae
Compare
📊 Page Load Benchmark ResultsCurrent Commit: 📄 https://metamask.github.io/test-dapp/Samples: 100 Summary
📈 Detailed Results
Results generated automatically by MetaMask CI |
Builds ready [a6e0aae]
UI Startup Metrics (1207 ± 64 ms)
Benchmark value 3 exceeds gate value 1 for chrome browserify home mean initialActions Benchmark value 252 exceeds gate value 18 for chrome browserify home p95 backgroundConnect Benchmark value 10 exceeds gate value 1.2 for chrome browserify home p95 initialActions Benchmark value 2200 exceeds gate value 2192 for chrome webpack home mean uiStartup Benchmark value 33 exceeds gate value 29 for chrome webpack home mean getState Benchmark value 2597 exceeds gate value 2454 for chrome webpack home p95 uiStartup Benchmark value 393 exceeds gate value 370 for chrome webpack home p95 firstReactRender Benchmark value 267 exceeds gate value 195 for chrome webpack home p95 getState Benchmark value 10 exceeds gate value 7 for chrome webpack home p95 initialActions Benchmark value 238 exceeds gate value 65 for chrome webpack home p95 setupStore Benchmark value 111 exceeds gate value 110 for firefox browserify home mean domInteractive Benchmark value 30 exceeds gate value 25 for firefox browserify home mean backgroundConnect Benchmark value 27 exceeds gate value 25 for firefox browserify home mean firstReactRender Benchmark value 4 exceeds gate value 1 for firefox browserify home mean initialActions Benchmark value 237 exceeds gate value 195 for firefox browserify home p95 domInteractive Benchmark value 13 exceeds gate value 2 for firefox browserify home p95 initialActions Benchmark value 1619 exceeds gate value 1615 for firefox webpack home mean uiStartup Benchmark value 1383 exceeds gate value 1380 for firefox webpack home mean load Benchmark value 1382 exceeds gate value 1380 for firefox webpack home mean domContentLoaded Benchmark value 109 exceeds gate value 100 for firefox webpack home mean domInteractive Benchmark value 30 exceeds gate value 26 for firefox webpack home mean backgroundConnect Benchmark value 61 exceeds gate value 38 for firefox webpack home mean firstReactRender Benchmark value 4 exceeds gate value 1 for firefox webpack home mean initialActions Benchmark value 1361 exceeds gate value 1360 for firefox webpack home mean loadScripts Benchmark value 260 exceeds gate value 156 for firefox webpack home p95 domInteractive Benchmark value 70 exceeds gate value 50 for firefox webpack home p95 firstReactRender Benchmark value 13 exceeds gate value 2 for firefox webpack home p95 initialActions Sum of mean exceeds: 301ms | Sum of p95 exceeds: 844.8ms Sum of all benchmark exceeds: 1145.8ms Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
a6e0aae to
68dca86
Compare
7 tasks
Gudahtt
added a commit
that referenced
this pull request
Sep 12, 2025
Add a new test workflow for E2E tests using `dist` builds. Currently this workflow has just one test suite in it, the vault decryption test. More will be added in later PRs. This was split out from #31435
7 tasks
Gudahtt
added a commit
that referenced
this pull request
Sep 12, 2025
Add a new test workflow for E2E tests using `dist` builds. Currently this workflow has just one test suite in it, the vault decryption test. More will be added in later PRs. This was split out from #31435
Gudahtt
added a commit
that referenced
this pull request
Sep 12, 2025
Add a new test workflow for E2E tests using `dist` builds. Currently this workflow has just one test suite in it, the vault decryption test. More will be added in later PRs. This was split out from #31435
github-merge-queue bot
pushed a commit
that referenced
this pull request
Sep 20, 2025
## **Description** The script responsible for determining how to split up tests in a given workflow was failing when I tried to add a new workflow. It didn't seem to handle the case where a workflow had zero past runs. It has been updated to use a naive fallback splitting algorithm in this case. [](https://codespaces.new/MetaMask/metamask-extension/pull/35905?quickstart=1) ## **Changelog** CHANGELOG entry: null ## **Related issues** This fixes a CI bug encountered on #31435 ## **Manual testing steps** See this PR for an example of the failures: #35906 See here for a rebased version of that PR on this branch, which works: #35913 ## **Screenshots/Recordings** N/A ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Extension Coding Standards](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I’ve included tests if applicable - [x] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-extension/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. --------- Co-authored-by: Howard Braham <howrad@gmail.com>
Add E2E test to ensure a window opens upon first install. This test needs to use a production-like build rather than a standard E2E build, so the existing "vault decryptor" job was repurposed to be more generically for E2E tests using a production-like build. A global function `reloadExtension` is added to `stateHooks` for use by this test. This function had to be enabled for production builds because the test uses a production build.
Gudahtt
force-pushed
the
first-time-install-e2e-test
branch
from
68dca86 to
690ca8f
Compare
Gudahtt
added a commit
that referenced
this pull request
Oct 17, 2025
Add a new test workflow for E2E tests using `dist` builds. Currently this workflow has just one test suite in it, the vault decryption test. More will be added in later PRs. This was split out from #31435
📊 Page Load Benchmark ResultsCurrent Commit: 📄 Localhost MetaMask Test DappSamples: 100 Summary
📈 Detailed Results
Results generated automatically by MetaMask CI |
Builds ready [690ca8f]
UI Startup Metrics (1249 ± 74 ms)
Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
](https://codespaces.new/MetaMask/metamask-extension/pull/34772?quickstart=1){kind=link}
](https://codespaces.new/MetaMask/metamask-extension/pull/35905?quickstart=1){kind=link}
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add E2E test to ensure a window opens upon first install.
This test needs to use a production-like build rather than a standard E2E build, so the existing "vault decryptor" job was repurposed to be more generically for E2E tests using a production-like build.
A global function
reloadExtensionis added tostateHooksfor use by this test. This function had to be enabled for production builds because the test uses a production build.Related issues
This is an E2E test for #31332
Manual testing steps
yarn dist)yarn test:e2e:single ./test/e2e/first-install.spec.ts --browser chrometo run the test on Chromeyarn dist:mv2)yarn test:e2e:single ./test/e2e/first-install.spec.ts --browser firefoxto run the test on FirefoxScreenshots/Recordings
N/A
Pre-merge author checklist
Pre-merge reviewer checklist