Patch: Compress transaction history

[Gudahtt]

Description

Compress transaction history down to 100 entries and truncate any existing transaction historys to just 100 entries. This should fix reports we've seen from production of extremely poor performance and crashes caused by long transaction histories, and will also prevent this problem from happening again.

Transaction history has been truncated because in the extreme cases, it would be prohibitively expensive to compress. The downside is that some state of how the transaction has changed may be lost. But this is unlikely to impact users because we only show a limited number of events from the transaction history in our UI, and these events are more likely to be at the beginning of long transaction histories. Even if a displayed event is lost, the impact on the UI is minimal (it's only shown on the transaction details page under "Activity log", and only for informational purposes).

For details on how the transaction compression works, and how it prevents history size from growing unbouned, see MetaMask/core#4555

The transaction controller change has been applied using a patch. The patch was generated from the core repository branch patch/transaction-controller-v32-compress-history. It will be made on develop at a later date because it is blocked by other controller updates at the moment.

The truncation is performed by a migration that was added in #26291 and cherry-picked into this branch

Related issues

Relates to #9372

Manual testing steps

To test the migration:

• Checkout v11.15.6 and create a dev build (unfortunately the repro steps don't work on MV3 due to snow being enabled even in dev builds, so we're using a pre-MV3 build)
• Install the dev build from the dist/chrome directory and proceed through onboarding
• Make a single transaction
  • Any chain, any transaction, as long as it's approved. We don't even need to wait for it to settle.
• Run this command in the background console to add a huge history to the transaction controller and restart the extension:

  chrome.storage.local.get(
    null,
    (state) => {
      state.data.TransactionController.transactions[0].history.push(
        ...[...Array(100000).keys()].map(() => [
          {
            value: '[ethjs-rpc] rpc error with payload {"id":[number],"jsonrpc":"2.0","params":["0x"],"method":"eth_getTransactionReceipt"} { "code": -32603, "message": "Internal JSON-RPC error.", "data": { "code": -32602, "message": "invalid argument 0: hex string has length 0, want 64 for common.Hash", "cause": null }, "stack": ...',
            path: '/warning/error',
            op: 'replace'
          },
          {
            note: 'transactions/pending-tx-tracker#event: tx:warning',
            value: 'There was a problem loading this transaction.',
            path: '/warning/message',
            op: 'replace'
          },
        ]),
      );
      chrome.storage.local.set(state, () => chrome.runtime.reload());
    }
  );
  

  • The extension should become extremely slow
• Disable the extension
• Switch to this branch and create a dev build
• Enable and reload the extension
  • The extension should no longer be slow

To test that the compression is working, we would want to get a transaction in a "stuck pending" state where an error is captured each iteration. I am not yet sure how to do that unfortunately.

Screenshots/Recordings

N/A

Pre-merge author checklist

• I've followed MetaMask Contributor Docs and MetaMask Extension Coding Standards.
• I've completed the PR template to the best of my ability
• I’ve included tests if applicable
• I’ve documented my code using JSDoc format if applicable
• I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

• I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
• I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

[socket-security]

Removed dependencies detected. Learn more about Socket for GitHub ↗︎

🚮 Removed packages: npm/@metamask/gas-fee-controller@17.0.0, npm/@metamask/transaction-controller@32.0.0

View full report↗︎

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.64%. Comparing base (c880e07) to head (7005ca2).
Report is 7 commits behind head on Version-v12.0.1.

Additional details and impacted files

@@                 Coverage Diff                 @@
##           Version-v12.0.1   #26236      +/-   ##
===================================================
+ Coverage            65.61%   65.64%   +0.03%     
===================================================
  Files                 1368     1369       +1     
  Lines                54600    54642      +42     
  Branches             14183    14193      +10     
===================================================
+ Hits                 35825    35867      +42     
  Misses               18775    18775              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[metamaskbot]

Builds ready [179086a]

• builds: chrome, firefox
• builds (beta): chrome
• builds (flask): chrome, firefox
• builds (MMI): chrome, firefox
• builds (test): chrome, firefox
• builds (test-flask): chrome, firefox
• build viz: Build System
• mv3: Background Module Init Stats
• mv3: UI Init Stats
• mv3: Module Load Stats
• mv3: Bundle Size Stats
• mv2: E2e Actions Stats
• code coverage: Report
• storybook: Storybook
• typescript migration: Dashboard
• all artifacts
• bundle viz:

  • background: 0, 1, 2, 3, 4, 5, 6
  • common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
  • content-script: 0
  • offscreen: 0
  • ui: 0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (52 ± 9 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	67	276	92	47	23
		domContentLoaded	9	69	14	15	7
		load	41	115	52	18	9
		domInteractive	9	69	14	15	7

[DDDDDanica]

Question: under which circumstance transactions is not an object?

[Gudahtt]

They should always be objects. This is meant to validate that the state is not corrupted somehow. We validate everything that the migration assumes, and in this case we are assuming it's an object so we validate that first.

[danjm]

There is both a 120.2 and 120.3 is this PR. The code is almost identical between the two... I assume this is not intentional?

[Gudahtt]

There is both a 120.2 and 120.3 is this PR. The code is almost identical between the two... I assume this is not intentional?

Correct, that was a mistake. 120.2 was the old copy, 120.3 was cherry-picked from develop. The old one has been deleted in 497d64f

I'm leaving a gap here for 120.2 because we still need to cherry-pick that in as well

[metamaskbot]

Builds ready [7005ca2]

• builds: chrome, firefox
• builds (beta): chrome
• builds (flask): chrome, firefox
• builds (MMI): chrome, firefox
• builds (test): chrome, firefox
• builds (test-flask): chrome, firefox
• build viz: Build System
• mv3: Background Module Init Stats
• mv3: UI Init Stats
• mv3: Module Load Stats
• mv3: Bundle Size Stats
• mv2: E2e Actions Stats
• code coverage: Report
• storybook: Storybook
• typescript migration: Dashboard
• all artifacts
• bundle viz:

  • background: 0, 1, 2, 3, 4, 5, 6
  • common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
  • content-script: 0
  • offscreen: 0
  • ui: 0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (47 ± 4 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	61	105	79	8	4
		domContentLoaded	9	18	10	2	1
		load	40	75	47	8	4
		domInteractive	9	18	10	2	1

[danjm]

I have successfully manually tested this. If the transaction history reaches length 100, then further additions to the history do not increase the length, but the latest additions are correctly added (whether those are errors or otherwise)

To test, I made 100 edits to gas price, and then confirmed the transaction. I also did the same, but with some code in the transaction controller edited, such that the next resubmission after first submission would fail.

[metamaskbot]

Builds ready [9661d84]

• builds: chrome, firefox
• builds (beta): chrome
• builds (flask): chrome, firefox
• builds (MMI): chrome, firefox
• builds (test): chrome, firefox
• builds (test-flask): chrome, firefox
• build viz: Build System
• mv3: Background Module Init Stats
• mv3: UI Init Stats
• mv3: Module Load Stats
• mv3: Bundle Size Stats
• mv2: E2e Actions Stats
• code coverage: Report
• storybook: Storybook
• typescript migration: Dashboard
• all artifacts
• bundle viz:

  • background: 0, 1, 2, 3, 4, 5, 6
  • common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
  • content-script: 0
  • offscreen: 0
  • ui: 0, 1, 10, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (54 ± 9 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	70	173	86	21	10
		domContentLoaded	9	66	13	12	6
		load	42	123	54	18	9
		domInteractive	9	66	13	12	6