Skip to content

Commit

Permalink
fix: flaky test `Navigate transactions should reject and remove all u…
Browse files Browse the repository at this point in the history
…napproved transactions` (#25312)

<!--
Please submit this PR as a draft initially.
Do not mark it as "Ready for review" until the template has been
completely filled out, and PR status checks have passed at least once.
-->

## **Description**
This PR fixes the flaky test `Navigate transactions should reject and
remove all unapproved transactions`.
It fails with the error `TimeoutError: Waiting for element to be located
By(xpath, //button[contains(text(), "Reject all")])`.

The problem is that we click the Reject button, before the confirmation
screen has fully loaded (specifically, the total amount for that tx is
not there yet). This causes the click to don't have any effect and the
subsequent popup with the Reject All never appears.
The fix is simply await for the tx to be fully loaded (notice how the
total amount box is not loaded at the moment of the failure below).

Furthermore, it's been observed that the unapproved tx where trying to
load simulations (see loading spinner). This shouldn't prevent to
continue, but to further stabilize the test, the transactions have now
disabled the simulations.


- [ci
artifacts](https://circleci-tasks-prod.s3.us-east-1.amazonaws.com/forks/storage/artifacts/fb281712-ea41-446f-b28f-c0bea18842cc/798839675/3b956a72-4bc3-4ab4-8711-9d8786b90aa8/9/test-artifacts/chrome/Navigate%20transactions%20should%20reject%20and%20remove%20all%20unapproved%20transactions/test-failure-screenshot.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQVFQINEOCSPRCGLP%2F20240614%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240614T070310Z&X-Amz-Expires=60&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEAcaCXVzLWVhc3QtMSJGMEQCIHNThdyaubglBTAu8iEWtdQmyJDYsMnHS%2BZ8CmqpNjy1AiA5H6y8x2jtn0y%2FaqCSu1hD7orfORHWGN%2BlWiMbzXCVuiq0Agig%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAMaDDA0NTQ2NjgwNjU1NiIMdGPy5i2bqb9HiCvzKogCQMAcXnKXTqbbgJTWEmcA48k82rWrz3yFA3VtMSFHeb9VRIR9I9yfmm276VvqjjoOpXSa0I8LsuAaYBqZuGFCH7LNLSxpEEZVBjIsLjXMWetTaxbGcdRGyRth6K4B5GBTMxfqBKTGeE1qNWnh84MJpLgsNFVG3dqqGmFojTYrVCVdWVys%2FZfMPgC2WM8MgjtxlebGu9XW%2F9VQsq6A1mukjbox4%2F4ckwq65kPpXqJDvPd7jXLzcEByaU7kjlujipAobC4cOc6SLSQ%2BOVGjc3hXF9AHaKIl7BfS%2F3QmsNyXVrnnnExt3Uzqn1l%2Foen%2BXQHSz4sSmTeDf63fUeSfXZHi51pXSGwTBGnSMIbRr7MGOp4BPkSg3VCEXfzPxTABRXjbB2WTEBDMKXVF5HMgnYnfJ5snWqF6lXuztslav0q3zKbzUhEGg6OPF%2BsZHy%2FvdzPeaVGUoyAqHQjgmZynPsCbTCfv2GfNZkprts%2F9mfXGrGPqTtDzX%2B16Z1%2B13ev8ayTTDDHat8vdzlWkc8SzfX3h3NsZnmUnJA9EpfDTUGvg5rVtw%2BotbpmxJ%2BZJaLNP3Vc%3D&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=5bcdd4ded909d42f11151e5336da7d0e903aaf99438b44c37116884354b55a5f)

[![Open in GitHub
Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/MetaMask/metamask-extension/pull/25312?quickstart=1)

## **Related issues**

Fixes: #24661 and
#24641

## **Manual testing steps**

1. Check ci
2. Run test locally multiple times `yarn test:e2e:single
test/e2e/tests/transaction/navigate-transactions.spec.js
--browser=chrome --leave-running --retryUntilFailure --retries=10`

## **Screenshots/Recordings**


### **Before**
Notice how when the test failed, the total tx amount was yet not loaded
(3.000,..) and we only see the gas displayed.

Furthermore: we see the tx simulation element loading (this is not the
cause of this failure, but it has also disabled, to stabilize the
screen).


![image](https://github.com/MetaMask/metamask-extension/assets/54408225/792d069c-2ab8-41aa-8180-37c84b817a72)


### **After**
Notice that the tx simulations loading element is not there.
Notice how the correct amount is await until proceeding (3.0000315)

![Screenshot from 2024-06-14
11-33-56](https://github.com/MetaMask/metamask-extension/assets/54408225/7e6e999f-707b-445c-a94b-51734579e767)

## **Pre-merge author checklist**

- [x] I've followed [MetaMask Contributor
Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask
Extension Coding
Standards](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/CODING_GUIDELINES.md).
- [x] I've completed the PR template to the best of my ability
- [x] I’ve included tests if applicable
- [x] I’ve documented my code using [JSDoc](https://jsdoc.app/) format
if applicable
- [x] I’ve applied the right labels on the PR (see [labeling
guidelines](https://github.com/MetaMask/metamask-extension/blob/develop/.github/guidelines/LABELING_GUIDELINES.md)).
Not required for external contributors.

## **Pre-merge reviewer checklist**

- [ ] I've manually tested the PR (e.g. pull and build branch, run the
app, test code being changed).
- [ ] I confirm that this PR addresses all acceptance criteria described
in the ticket it closes and includes the necessary testing evidence such
as recordings and or screenshots.
  • Loading branch information
seaona authored Jun 14, 2024
1 parent 6163319 commit 4aa2fd0
Show file tree
Hide file tree
Showing 2 changed files with 121 additions and 0 deletions.
86 changes: 86 additions & 0 deletions test/e2e/fixture-builder.js
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,12 @@ class FixtureBuilder {
});
}

withPreferencesControllerTxSimulationsDisabled() {
return this.withPreferencesController({
useTransactionSimulations: false,
});
}

withAccountsController(data) {
merge(this.fixture.data.AccountsController, data);
return this;
Expand Down Expand Up @@ -814,8 +820,28 @@ class FixtureBuilder {
timestamp: 1631545992244,
value: false,
},
{
op: 'add',
path: '/simulationData',
value: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
note: 'TransactionController#updateSimulationData - Update simulation data',
timestamp: 1631545992244,
},
],
],
simulationData: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
id: '7087d1d7-f0e8-4c0f-a903-6d9daa392baf',
loadingDefaults: false,
origin: 'https://metamask.github.io',
Expand Down Expand Up @@ -869,8 +895,28 @@ class FixtureBuilder {
timestamp: 1631545994695,
value: false,
},
{
op: 'add',
path: '/simulationData',
value: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
note: 'TransactionController#updateSimulationData - Update simulation data',
timestamp: 1631545992244,
},
],
],
simulationData: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
id: '6eab4240-3762-4581-abc5-cd91eab6964e',
loadingDefaults: false,
origin: 'https://metamask.github.io',
Expand Down Expand Up @@ -924,8 +970,28 @@ class FixtureBuilder {
timestamp: 1631545996678,
value: false,
},
{
op: 'add',
path: '/simulationData',
value: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
note: 'TransactionController#updateSimulationData - Update simulation data',
timestamp: 1631545992244,
},
],
],
simulationData: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
id: 'c15eee26-11d6-4914-a70e-36ef9a3bcacb',
loadingDefaults: false,
origin: 'https://metamask.github.io',
Expand Down Expand Up @@ -979,8 +1045,28 @@ class FixtureBuilder {
timestamp: 1631545998677,
value: false,
},
{
op: 'add',
path: '/simulationData',
value: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
note: 'TransactionController#updateSimulationData - Update simulation data',
timestamp: 1631545992244,
},
],
],
simulationData: {
error: {
code: 'disabled',
message: 'Simulation disabled',
},
tokenBalanceChanges: [],
},
id: 'dfa9e5ad-d069-46b1-976e-a23734971d87',
loadingDefaults: false,
origin: 'https://metamask.github.io',
Expand Down
35 changes: 35 additions & 0 deletions test/e2e/tests/transaction/navigate-transactions.spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ describe('Navigate transactions', function () {
await withFixtures(
{
fixtures: new FixtureBuilder()
.withPreferencesControllerTxSimulationsDisabled()
.withTransactionControllerMultipleTransactions()
.build(),
ganacheOptions: generateGanacheOptions({ hardfork: 'london' }),
Expand All @@ -21,6 +22,12 @@ describe('Navigate transactions', function () {
async ({ driver }) => {
await unlockWallet(driver);

// Wait until total amount is loaded to mitigate flakiness on reject
await driver.findElement({
tag: 'span',
text: '3.0000315',
});

// navigate transactions
await driver.clickElement('[data-testid="next-page"]');
let navigationElement = await driver.findElement(
Expand Down Expand Up @@ -102,6 +109,7 @@ describe('Navigate transactions', function () {
dapp: true,
fixtures: new FixtureBuilder()
.withPermissionControllerConnectedToTestDapp()
.withPreferencesControllerTxSimulationsDisabled()
.withTransactionControllerMultipleTransactions()
.build(),
ganacheOptions: generateGanacheOptions({ hardfork: 'london' }),
Expand All @@ -110,6 +118,12 @@ describe('Navigate transactions', function () {
async ({ driver }) => {
await unlockWallet(driver);

// Wait until total amount is loaded to mitigate flakiness on reject
await driver.findElement({
tag: 'span',
text: '3.0000315',
});

await driver.clickElement('[data-testid="next-page"]');
let navigationElement = await driver.findElement(
'.confirm-page-container-navigation',
Expand Down Expand Up @@ -146,6 +160,7 @@ describe('Navigate transactions', function () {
await withFixtures(
{
fixtures: new FixtureBuilder()
.withPreferencesControllerTxSimulationsDisabled()
.withTransactionControllerMultipleTransactions()
.build(),
ganacheOptions: generateGanacheOptions({ hardfork: 'london' }),
Expand All @@ -154,6 +169,12 @@ describe('Navigate transactions', function () {
async ({ driver }) => {
await unlockWallet(driver);

// Wait until total amount is loaded to mitigate flakiness on reject
await driver.findElement({
tag: 'span',
text: '3.0000315',
});

// reject transaction
await driver.clickElement({ text: 'Reject', tag: 'button' });
const navigationElement = await driver.waitForSelector({
Expand All @@ -174,6 +195,7 @@ describe('Navigate transactions', function () {
await withFixtures(
{
fixtures: new FixtureBuilder()
.withPreferencesControllerTxSimulationsDisabled()
.withTransactionControllerMultipleTransactions()
.build(),
ganacheOptions: generateGanacheOptions({ hardfork: 'london' }),
Expand All @@ -182,6 +204,12 @@ describe('Navigate transactions', function () {
async ({ driver }) => {
await unlockWallet(driver);

// Wait until total amount is loaded to mitigate flakiness on reject
await driver.findElement({
tag: 'span',
text: '3.0000315',
});

// confirm transaction
await driver.clickElement({ text: 'Confirm', tag: 'button' });
const navigationElement = await driver.waitForSelector({
Expand All @@ -202,6 +230,7 @@ describe('Navigate transactions', function () {
await withFixtures(
{
fixtures: new FixtureBuilder()
.withPreferencesControllerTxSimulationsDisabled()
.withTransactionControllerMultipleTransactions()
.build(),
ganacheOptions: generateGanacheOptions({ hardfork: 'london' }),
Expand All @@ -210,6 +239,12 @@ describe('Navigate transactions', function () {
async ({ driver, ganacheServer }) => {
await unlockWallet(driver);

// Wait until total amount is loaded to mitigate flakiness on reject
await driver.findElement({
tag: 'span',
text: '3.0000315',
});

// reject transactions
await driver.clickElement({ text: 'Reject 4', tag: 'a' });
await driver.clickElement({ text: 'Reject all', tag: 'button' });
Expand Down

0 comments on commit 4aa2fd0

Please sign in to comment.