Skip to content

Commit 342c321

Browse files
christopherferreira9christopherferreira9
authored
test: improve test execution consistency (#20266)
<!-- Please submit this PR as a draft initially. Do not mark it as "Ready for review" until the template has been completely filled out, and PR status checks have passed at least once. --> ## **Description** This PR aims to improve the test execution time consistency. - It introduces a new tag to reduce the suite of ConfirmationsRedesigned tests - Moves tests from Smoke to Regression - Separates some suites into multiple `.spec` files for easy matrix split <!-- Write a short description of the changes included in this pull request, also include relevant motivation and context. Have in mind the following questions: 1. What is the reason for the change? 2. What is the improvement/solution? --> ## **Changelog** <!-- If this PR is not End-User-Facing and should not show up in the CHANGELOG, you can choose to either: 1. Write `CHANGELOG entry: null` 2. Label with `no-changelog` If this PR is End-User-Facing, please write a short User-Facing description in the past tense like: `CHANGELOG entry: Added a new tab for users to see their NFTs` `CHANGELOG entry: Fixed a bug that was causing some NFTs to flicker` (This helps the Release Engineer do their job more quickly and accurately) --> CHANGELOG entry: ## **Related issues** Fixes: ## **Manual testing steps** ```gherkin Feature: my feature name Scenario: user [verb for user action] Given [describe expected initial app state] When user [verb for user action] Then [describe expected outcome] ``` ## **Screenshots/Recordings** <!-- If applicable, add screenshots and/or recordings to visualize the before and after of your change. --> ### **Before** <!-- [screenshots/recordings] --> ### **After** <!-- [screenshots/recordings] --> ## **Pre-merge author checklist** - [ ] I’ve followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile Coding Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [ ] I've completed the PR template to the best of my ability - [ ] I’ve included tests if applicable - [ ] I’ve documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [ ] I’ve applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.
1 parent 6dda4f6 commit 342c321

File tree

12 files changed

+448
-280
lines changed

12 files changed

+448
-280
lines changed

‎.github/workflows/run-e2e-regression-tests-android.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,23 @@ jobs:
3737
total_splits: 4
3838
secrets: inherit
3939

40+
41+
regression-confirmations-redesigned-android:
42+
name: 'Confirmations Redesigned Regression (Android) - ${{ matrix.split }}'
43+
needs: [build-android-apks]
44+
strategy:
45+
matrix:
46+
split: [1, 2]
47+
fail-fast: false
48+
uses: ./.github/workflows/run-e2e-workflow.yml
49+
with:
50+
test-suite-name: regression-confirmations-redesigned-android-${{ matrix.split }}
51+
platform: android
52+
test_suite_tag: 'RegressionConfirmationsRedesigned'
53+
split_number: ${{ matrix.split }}
54+
total_splits: 2
55+
secrets: inherit
56+
4057
regression-trade-android:
4158
name: 'Trade Regression (Android) - ${{ matrix.split }}'
4259
needs: [build-android-apks]
@@ -171,6 +188,7 @@ jobs:
171188
if: always()
172189
needs:
173190
- regression-confirmations-android
191+
- regression-confirmations-redesigned-android
174192
- regression-trade-android
175193
- regression-wallet-platform-android
176194
- regression-identity-android

‎e2e/specs/confirmations-redesigned/signatures/signatures-typed.spec.ts‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
import Assertions from '../../../framework/Assertions';
2+
import Browser from '../../../pages/Browser/BrowserView';
3+
import FooterActions from '../../../pages/Browser/Confirmations/FooterActions';
4+
import FixtureBuilder from '../../../framework/fixtures/FixtureBuilder';
5+
import RequestTypes from '../../../pages/Browser/Confirmations/RequestTypes';
6+
import TabBarComponent from '../../../pages/wallet/TabBarComponent';
7+
import TestDApp from '../../../pages/Browser/TestDApp';
8+
import { loginToApp } from '../../../viewHelper';
9+
import { withFixtures } from '../../../framework/fixtures/FixtureHelper';
10+
import { SmokeConfirmationsRedesigned } from '../../../tags';
11+
import { buildPermissions } from '../../../framework/fixtures/FixtureUtils';
12+
import RowComponents from '../../../pages/Browser/Confirmations/RowComponents';
13+
import { DappVariants } from '../../../framework/Constants';
14+
import { Mockttp } from 'mockttp';
15+
import { setupRemoteFeatureFlagsMock } from '../../../api-mocking/helpers/remoteFeatureFlagsHelper';
16+
import { confirmationsRedesignedFeatureFlags } from '../../../api-mocking/mock-responses/feature-flags-mocks';
17+
18+
const SIGNATURE_LIST = [
19+
{
20+
specName: 'Typed V1 Sign',
21+
testDappBtn: TestDApp.tapTypedSignButton.bind(TestDApp),
22+
requestType: RequestTypes.TypedSignRequest,
23+
additionAssertions: async () => {
24+
await Assertions.expectElementToBeVisible(RowComponents.NetworkAndOrigin);
25+
},
26+
},
27+
{
28+
specName: 'Typed V3 Sign',
29+
testDappBtn: TestDApp.tapTypedV3SignButton.bind(TestDApp),
30+
requestType: RequestTypes.TypedSignRequest,
31+
additionAssertions: async () => {
32+
await Assertions.expectElementToBeVisible(RowComponents.OriginInfo);
33+
},
34+
},
35+
{
36+
specName: 'Typed V4 Sign',
37+
testDappBtn: TestDApp.tapTypedV4SignButton.bind(TestDApp),
38+
requestType: RequestTypes.TypedSignRequest,
39+
additionAssertions: async () => {
40+
await Assertions.expectElementToBeVisible(RowComponents.OriginInfo);
41+
},
42+
},
43+
];
44+
45+
describe(SmokeConfirmationsRedesigned('Typed Signature Requests'), () => {
46+
const testSpecificMock = async (mockServer: Mockttp) => {
47+
await setupRemoteFeatureFlagsMock(
48+
mockServer,
49+
Object.assign({}, ...confirmationsRedesignedFeatureFlags),
50+
);
51+
};
52+
53+
beforeAll(async () => {
54+
jest.setTimeout(2500000);
55+
});
56+
57+
for (const {
58+
specName,
59+
testDappBtn,
60+
requestType,
61+
additionAssertions,
62+
} of SIGNATURE_LIST) {
63+
it(`should sign ${specName} message`, async () => {
64+
await withFixtures(
65+
{
66+
dapps: [
67+
{
68+
dappVariant: DappVariants.TEST_DAPP,
69+
},
70+
],
71+
fixture: new FixtureBuilder()
72+
.withGanacheNetwork()
73+
.withPermissionControllerConnectedToTestDapp(
74+
buildPermissions(['0x539']),
75+
)
76+
.build(),
77+
restartDevice: true,
78+
testSpecificMock,
79+
},
80+
async () => {
81+
await loginToApp();
82+
83+
await TabBarComponent.tapBrowser();
84+
await Browser.navigateToTestDApp();
85+
86+
// cancel request
87+
await testDappBtn();
88+
await Assertions.expectElementToBeVisible(requestType);
89+
await FooterActions.tapCancelButton();
90+
await Assertions.expectElementToNotBeVisible(requestType);
91+
92+
await testDappBtn();
93+
await Assertions.expectElementToBeVisible(requestType);
94+
95+
// check different sections are visible
96+
await Assertions.expectElementToBeVisible(
97+
RowComponents.AccountNetwork,
98+
);
99+
await Assertions.expectElementToBeVisible(RowComponents.Message);
100+
101+
// any signature specific additional assertions
102+
if (additionAssertions) {
103+
await additionAssertions();
104+
}
105+
106+
// confirm request
107+
await FooterActions.tapConfirmButton();
108+
await Assertions.expectElementToNotBeVisible(requestType);
109+
},
110+
);
111+
});
112+
}
113+
});

‎e2e/specs/confirmations-redesigned/signatures/signatures.spec.ts‎

Lines changed: 0 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -34,30 +34,6 @@ const SIGNATURE_LIST = [
3434
);
3535
},
3636
},
37-
{
38-
specName: 'Typed V1 Sign',
39-
testDappBtn: TestDApp.tapTypedSignButton.bind(TestDApp),
40-
requestType: RequestTypes.TypedSignRequest,
41-
additionAssertions: async () => {
42-
await Assertions.expectElementToBeVisible(RowComponents.NetworkAndOrigin);
43-
},
44-
},
45-
{
46-
specName: 'Typed V3 Sign',
47-
testDappBtn: TestDApp.tapTypedV3SignButton.bind(TestDApp),
48-
requestType: RequestTypes.TypedSignRequest,
49-
additionAssertions: async () => {
50-
await Assertions.expectElementToBeVisible(RowComponents.OriginInfo);
51-
},
52-
},
53-
{
54-
specName: 'Typed V4 Sign',
55-
testDappBtn: TestDApp.tapTypedV4SignButton.bind(TestDApp),
56-
requestType: RequestTypes.TypedSignRequest,
57-
additionAssertions: async () => {
58-
await Assertions.expectElementToBeVisible(RowComponents.OriginInfo);
59-
},
60-
},
6137
];
6238

6339
describe(SmokeConfirmationsRedesigned('Signature Requests'), () => {

‎e2e/specs/confirmations-redesigned/transactions/contract-deployment.spec.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { SmokeConfirmationsRedesigned } from '../../../tags';
1+
import { RegressionConfirmationsRedesigned } from '../../../tags';
22
import { loginToApp } from '../../../viewHelper';
33
import Browser from '../../../pages/Browser/BrowserView';
44
import FixtureBuilder from '../../../framework/fixtures/FixtureBuilder';
@@ -17,7 +17,7 @@ import { setupMockRequest } from '../../../api-mocking/helpers/mockHelpers';
1717
import { setupRemoteFeatureFlagsMock } from '../../../api-mocking/helpers/remoteFeatureFlagsHelper';
1818
import { confirmationsRedesignedFeatureFlags } from '../../../api-mocking/mock-responses/feature-flags-mocks';
1919

20-
describe(SmokeConfirmationsRedesigned('Contract Deployment'), () => {
20+
describe(RegressionConfirmationsRedesigned('Contract Deployment'), () => {
2121
const testSpecificMock = async (mockServer: Mockttp) => {
2222
await setupMockRequest(mockServer, {
2323
requestMethod: 'GET',

‎e2e/specs/confirmations-redesigned/transactions/dapp-initiated-transfer.spec.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { SmokeConfirmationsRedesigned } from '../../../tags';
1+
import { RegressionConfirmationsRedesigned } from '../../../tags';
22
import { loginToApp } from '../../../viewHelper';
33
import Browser from '../../../pages/Browser/BrowserView';
44
import FixtureBuilder from '../../../framework/fixtures/FixtureBuilder';
@@ -47,7 +47,7 @@ const expectedEventNames = [
4747
expectedEvents.TRANSACTION_FINALIZED,
4848
];
4949

50-
describe(SmokeConfirmationsRedesigned('DApp Initiated Transfer'), () => {
50+
describe(RegressionConfirmationsRedesigned('DApp Initiated Transfer'), () => {
5151
const testSpecificMock = async (mockServer: Mockttp) => {
5252
await setupMockPostRequest(
5353
mockServer,

‎e2e/specs/confirmations-redesigned/transactions/send-max-transfer.spec.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { SmokeConfirmationsRedesigned } from '../../../tags';
1+
import { RegressionConfirmationsRedesigned } from '../../../tags';
22
import { loginToApp } from '../../../viewHelper';
33
import { withFixtures } from '../../../framework/fixtures/FixtureHelper';
44
import { buildPermissions } from '../../../framework/fixtures/FixtureUtils';
@@ -23,7 +23,7 @@ import { confirmationsRedesignedFeatureFlags } from '../../../api-mocking/mock-r
2323

2424
const RECIPIENT = '0x0c54fccd2e384b4bb6f2e405bf5cbc15a017aafb';
2525

26-
describe(SmokeConfirmationsRedesigned('Send Max Transfer'), () => {
26+
describe(RegressionConfirmationsRedesigned('Send Max Transfer'), () => {
2727
const testSpecificMock = async (mockServer: Mockttp) => {
2828
await setupMockRequest(mockServer, {
2929
requestMethod: 'GET',

‎e2e/specs/connections/evm-provider-events-regression.spec.ts‎

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
import { RegressionWalletPlatform } from '../../tags';
2+
import Assertions from '../../framework/Assertions';
3+
import FixtureBuilder, {
4+
DEFAULT_FIXTURE_ACCOUNT,
5+
DEFAULT_FIXTURE_ACCOUNT_2,
6+
DEFAULT_FIXTURE_ACCOUNT_CHECKSUM,
7+
} from '../../framework/fixtures/FixtureBuilder';
8+
import { withFixtures } from '../../framework/fixtures/FixtureHelper';
9+
import TestDApp from '../../pages/Browser/TestDApp';
10+
import TabBarComponent from '../../pages/wallet/TabBarComponent';
11+
import Browser from '../../pages/Browser/BrowserView';
12+
import ConnectBottomSheet from '../../pages/Browser/ConnectBottomSheet';
13+
import ConnectedAccountsModal from '../../pages/Browser/ConnectedAccountsModal';
14+
import { loginToApp } from '../../viewHelper';
15+
import { DappVariants } from '../../framework/Constants';
16+
import ToastModal from '../../pages/wallet/ToastModal';
17+
import AccountListBottomSheet from '../../pages/wallet/AccountListBottomSheet';
18+
19+
describe(RegressionWalletPlatform('EVM Provider Events'), () => {
20+
beforeAll(async () => {
21+
jest.setTimeout(150000);
22+
});
23+
24+
it('should notify account changes when adding and removing a permitted account for a dapp', async () => {
25+
await withFixtures(
26+
{
27+
dapps: [
28+
{
29+
dappVariant: DappVariants.TEST_DAPP,
30+
},
31+
],
32+
fixture: new FixtureBuilder()
33+
.withImportedHdKeyringAndTwoDefaultAccountsOneImportedHdAccountKeyringController()
34+
.withPermissionControllerConnectedToTestDapp()
35+
.build(),
36+
restartDevice: true,
37+
},
38+
async () => {
39+
await loginToApp();
40+
await TabBarComponent.tapBrowser();
41+
await Browser.navigateToTestDApp();
42+
await Browser.tapNetworkAvatarOrAccountButtonOnBrowser();
43+
await Assertions.expectElementToBeVisible(ConnectedAccountsModal.title);
44+
await Assertions.expectElementToNotBeVisible(
45+
ToastModal.notificationTitle,
46+
);
47+
48+
await ConnectedAccountsModal.tapManagePermissionsButton();
49+
await ConnectedAccountsModal.tapPermissionsSummaryTab();
50+
await ConnectedAccountsModal.tapNavigateToEditAccountsPermissionsButton();
51+
await AccountListBottomSheet.tapAccountByName('Account 2');
52+
await AccountListBottomSheet.tapConnectAccountsButton();
53+
const connectedAccountsAfterAdd = await TestDApp.getConnectedAccounts();
54+
if (
55+
connectedAccountsAfterAdd !==
56+
[DEFAULT_FIXTURE_ACCOUNT_CHECKSUM, DEFAULT_FIXTURE_ACCOUNT_2].join(
57+
',',
58+
)
59+
) {
60+
throw new Error(
61+
'connected accounts did not update when account was added',
62+
);
63+
}
64+
65+
await ConnectedAccountsModal.tapNavigateToEditAccountsPermissionsButton();
66+
await AccountListBottomSheet.tapAccountByName('Account 1');
67+
await AccountListBottomSheet.tapConnectAccountsButton();
68+
const connectedAccountsAfterRemove =
69+
await TestDApp.getConnectedAccounts();
70+
if (connectedAccountsAfterRemove !== DEFAULT_FIXTURE_ACCOUNT_2) {
71+
throw new Error(
72+
'connected accounts did not update when account was removed',
73+
);
74+
}
75+
},
76+
);
77+
});
78+
79+
it('should notify the dapp of the selected account after permitting a previously unpermitted dapp', async () => {
80+
await withFixtures(
81+
{
82+
dapps: [
83+
{
84+
dappVariant: DappVariants.TEST_DAPP,
85+
},
86+
],
87+
fixture: new FixtureBuilder().build(),
88+
restartDevice: true,
89+
},
90+
async () => {
91+
await loginToApp();
92+
await TabBarComponent.tapBrowser();
93+
await Browser.navigateToTestDApp();
94+
95+
const connectedAccountsBefore = await TestDApp.getConnectedAccounts();
96+
if (connectedAccountsBefore !== '') {
97+
throw new Error('accounts already connected');
98+
}
99+
100+
await TestDApp.connect();
101+
await ConnectBottomSheet.tapConnectButton();
102+
103+
const connectedAccountsAfter = await TestDApp.getConnectedAccounts();
104+
if (connectedAccountsAfter !== DEFAULT_FIXTURE_ACCOUNT) {
105+
throw new Error('account not connected');
106+
}
107+
},
108+
);
109+
});
110+
});

0 commit comments

Comments
 (0)