Skip to content
This repository has been archived by the owner on Oct 7, 2024. It is now read-only.

chore(deps): replace superstruct imports with @metamask/superstruct #328

Closed
wants to merge 19 commits into from

Conversation

MajorLift
Copy link
Contributor

@MajorLift MajorLift commented May 30, 2024

Explanation

As part of the Wallet Framework Team's OKR (Q2 2024 O3KR4) for upgrading TypeScript to v5.0+ in the core monorepo, we are updating dependencies of the core repo so that they generate builds and type declarations for both CJS and ESM.

This requirement applies to nested dependencies as well, so we are also replacing superstruct with the ESM-compatible fork @metamask/superstruct in all core dependency packages.

Description

  • Replace superstruct dependency with @metamask/superstruct ^3.0.0.
    • 3.1.0
  • Replace all superstruct import statements with @metamask/superstruct
  • Bump @metamask/utils to ^8.5.0.
    • 9.0.0
  • If feasible without too much additional work:
    • Bump typescript to ~5.0.4
    • Set tsconfig options module and moduleResolution to NodeNext.
  • Using the create-release-pr github action, publish a new release containing these changes.

References

Changelog

## [8.0.1]
### Changed
- Re-categorize `@metamask/providers` from devDependency to dependency. ([#328](https://github.com/MetaMask/keyring-api/pull/328))
  - The imported type `MetaMaskInpageProvider` is required to use the exported classes `SnapRpcSender`, `KeyringSnapRpcClient`.
- Bump `@metamask/snaps-sdk` from `^4.2.0` to `^6.0.1` ([#328](https://github.com/MetaMask/keyring-api/pull/328))
- Bump `@metamask/utils` from `^8.4.0` to `^9.0.0` ([#328](https://github.com/MetaMask/keyring-api/pull/328))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#328](https://github.com/MetaMask/keyring-api/pull/328))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.

@MajorLift MajorLift self-assigned this May 30, 2024
@MajorLift MajorLift force-pushed the replace-superstruct-with-fork branch 2 times, most recently from 8bd8517 to bcc13d7 Compare May 30, 2024 21:47
Copy link

socket-security bot commented Jun 3, 2024

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package New capabilities Transitives Size Publisher
npm/@babel/code-frame@7.24.7 environment +10 194 kB nicolo-ribaudo
npm/@babel/core@7.24.8 environment, filesystem, unsafe Transitive: shell +22 4.88 MB nicolo-ribaudo
npm/@babel/generator@7.24.8 None +3 626 kB nicolo-ribaudo
npm/@babel/helper-plugin-utils@7.24.8 None 0 114 kB nicolo-ribaudo
npm/@babel/parser@7.24.8 None 0 1.89 MB nicolo-ribaudo
npm/@babel/template@7.24.7 None 0 69 kB nicolo-ribaudo
npm/@babel/traverse@7.24.8 None +5 723 kB nicolo-ribaudo
npm/@babel/types@7.24.8 environment +3 2.5 MB nicolo-ribaudo
npm/@eslint-community/regexpp@4.11.0 None 0 446 kB eslint-community-bot
npm/@jridgewell/sourcemap-codec@1.5.0 None 0 113 kB jridgewell
npm/@metamask/json-rpc-engine@8.0.2 Transitive: network +14 1.82 MB metamaskbot
npm/@metamask/providers@17.0.0 None +7 667 kB metamaskbot
npm/@noble/curves@1.4.2 None 0 1.5 MB paulmillr
npm/@scure/base@1.1.7 None 0 122 kB paulmillr
npm/@types/babel__traverse@7.20.6 None 0 84.1 kB types
npm/@types/istanbul-lib-coverage@2.0.6 None 0 5.45 kB types
npm/@types/json-schema@7.0.15 None 0 31.7 kB types
npm/@types/node@20.14.10 None +1 2.16 MB types
npm/acorn@8.12.1 None 0 538 kB marijn
npm/call-bind@1.0.7 None +4 87.8 kB ljharb
npm/debug@4.3.5 environment 0 42.4 kB qix
npm/define-data-property@1.1.4 None +2 50.4 kB ljharb
npm/define-properties@1.2.1 None +1 39.4 kB ljharb
npm/es-abstract@1.23.3 None +46 3.38 MB ljharb
npm/es-errors@1.3.0 None 0 12.3 kB ljharb
npm/escalade@3.1.2 filesystem 0 11.6 kB lukeed
npm/eslint-plugin-promise@6.4.0 None 0 71.1 kB eslint-community-bot
npm/esquery@1.6.0 None 0 1.04 MB michaelficarra
npm/fast-glob@3.3.2 filesystem +6 259 kB mrmlnc
npm/fsevents@2.3.3 None 0 173 kB pipobscure
npm/get-intrinsic@1.2.4 eval +1 73.1 kB ljharb
npm/glob@10.4.5 Transitive: environment, filesystem, shell +20 2.16 MB isaacs
npm/has-property-descriptors@1.0.2 None +1 22.7 kB ljharb
npm/has-proto@1.0.3 None 0 12 kB ljharb
npm/has-tostringtag@1.0.2 None 0 17.6 kB ljharb
npm/is-core-module@2.14.0 None 0 31.1 kB ljharb
npm/is-shared-array-buffer@1.0.3 None 0 18.7 kB ljharb
npm/istanbul-lib-coverage@3.2.2 None 0 34.4 kB oss-bot
npm/jest-it-up@3.2.0 filesystem Transitive: environment, shell +2 213 kB rbardini
npm/lru-cache@10.4.3 None 0 804 kB isaacs
npm/minipass@7.1.2 None 0 286 kB isaacs
npm/node-gyp@10.2.0 environment, shell Transitive: filesystem, network +43 3.65 MB nodejs-foundation
npm/picocolors@1.0.1 environment 0 5.15 kB alexeyraspopov
npm/rimraf@5.0.9 environment, filesystem 0 281 kB isaacs
npm/tsd@0.31.1 Transitive: environment, filesystem +38 47.9 MB sindresorhus
npm/typescript@5.0.4 None 0 39.2 MB typescript-bot
npm/which-typed-array@1.1.15 None +4 92.7 kB ljharb

🚮 Removed packages: npm/@babel/code-frame@7.24.2, npm/@babel/core@7.23.2, npm/@babel/generator@7.24.5, npm/@babel/helper-plugin-utils@7.22.5, npm/@babel/helper-split-export-declaration@7.24.5, npm/@babel/helper-validator-identifier@7.24.5, npm/@babel/parser@7.24.5, npm/@babel/template@7.22.15, npm/@babel/traverse@7.24.5, npm/@babel/types@7.24.5, npm/@eslint-community/regexpp@4.10.0, npm/@jridgewell/sourcemap-codec@1.4.15, npm/@metamask/providers@17.1.1, npm/@metamask/utils@8.4.0, npm/@scure/base@1.1.3, npm/@types/babel__traverse@7.20.3, npm/@types/istanbul-lib-coverage@2.0.4, npm/@types/json-schema@7.0.12, npm/@types/node@20.12.12, npm/acorn@8.11.3, npm/agent-base@6.0.2, npm/call-bind@1.0.2, npm/debug@4.3.4, npm/define-properties@1.2.0, npm/eslint-plugin-promise@6.1.1, npm/esquery@1.5.0, npm/fast-glob@3.2.12, npm/fsevents@2.3.2, npm/get-intrinsic@1.2.1, npm/glob@10.3.16, npm/is-array-buffer@3.0.2, npm/is-core-module@2.13.1, npm/is-typed-array@1.1.12, npm/istanbul-lib-coverage@3.2.0, npm/jest-it-up@3.1.0, npm/lru-cache@10.2.2, npm/minipass@7.1.1, npm/object-inspect@1.12.3, npm/rimraf@5.0.7, npm/superstruct@1.0.3, npm/tsd@0.31.0, npm/typescript@4.8.4

View full report↗︎

@MajorLift MajorLift force-pushed the replace-superstruct-with-fork branch from 030fec1 to a982011 Compare June 6, 2024 23:42
@MajorLift MajorLift marked this pull request as ready for review June 6, 2024 23:52
@MajorLift MajorLift requested a review from a team as a code owner June 6, 2024 23:52
@danroc danroc changed the title Replace 'superstruct' imports with '@metamask/superstruct' chore(deps): replace 'superstruct' imports with '@metamask/superstruct' Jun 7, 2024
danroc
danroc previously approved these changes Jun 7, 2024
package.json Outdated
@@ -38,11 +38,11 @@
"test:watch": "jest --watch"
},
"dependencies": {
"@metamask/snaps-sdk": "^4.2.0",
"@metamask/utils": "^8.4.0",
"@metamask/snaps-sdk": "^4.4.1",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to upgrade this one?

Copy link
Contributor Author

@MajorLift MajorLift Jun 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ccharly Thanks for checking in on this!

@metamask/snaps-sdk is also undergoing the same changes as this PR, which are due to be released in ^4.4.3.

The changes in snaps-sdk need to be reflected in this package because we need to replace superstruct in all nested dependencies of core. So we need to update the dependencies in all core packages so that both of the following are satisfied:

  • @metamask/snaps-sdk@^4.4.3 > @metamask/superstruct@^3.0.0
  • @metamask/keyring-api@^6.4.1 > @metamask/snaps-sdk@^4.4.3 > @metamask/superstruct@^3.0.0

What we're trying to avoid is a conflicting resolution like the following, because enabling the NodeNext option in core doesn't work if we have the ESM-incompatible superstruct anywhere in the dependency tree:

  • @metamask/snaps-sdk@^4.4.3 > @metamask/superstruct@^3.0.0
  • @metamask/keyring-api@^6.4.1 > @metamask/snaps-sdk@^4.2.0 > superstruct@^1.0.3

To achieve this, I'm first bumping @metamask/snaps-sdk to the latest released version in this PR to check for regressions or breakage that need fixing.

Next step is to bump to the unreleased version. I was planning to do this in a separate PR, but it looks like I might as well do it here, which is why I've switched this back to draft. I'll set this PR as ready to review once MetaMask/snaps#2445 is merged and released.

Hope this clarifies things!

Copy link

socket-security bot commented Jul 14, 2024

🚨 Potential security issues detected. Learn more about Socket for GitHub ↗︎

To accept the risk, merge this PR and you will not be notified again.

Alert Package NoteSourceCI
New author npm/istanbul-lib-report@3.0.1 🚫
New author npm/extension-port-stream@3.0.0 🚫
New author npm/diff@5.2.0 🚫
New author npm/superstruct@1.0.4 🚫
New author npm/micromatch@4.0.7 🚫
New author npm/node-gyp@10.2.0 🚫
Unstable ownership npm/node-gyp@10.2.0 🚫

View full report↗︎

Next steps

What is new author?

A new npm collaborator published a version of the package for the first time. New collaborators are usually benign additions to a project, but do indicate a change to the security surface area of a package.

Scrutinize new collaborator additions to packages because they now have the ability to publish code into your dependency tree. Packages should avoid frequent or unnecessary additions or changes to publishing rights.

What is unstable ownership?

A new collaborator has begun publishing package versions. Package stability and security risk may be elevated.

Try to reduce the amount of authors you depend on to reduce the risk to malicious actors gaining access to your supply chain. Packages should remove inactive collaborators with publishing rights from packages on npm.

Take a deeper look at the dependency

Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support [AT] socket [DOT] dev.

Remove the package

If you happen to install a dependency that Socket reports as Known Malware you should immediately remove it and select a different dependency. For other alert types, you may may wish to investigate alternative packages or consider if there are other ways to mitigate the specific risk posed by the dependency.

Mark a package as acceptable risk

To ignore an alert, reply with a comment starting with @SocketSecurity ignore followed by a space separated list of ecosystem/package-name@version specifiers. e.g. @SocketSecurity ignore npm/foo@1.0.0 or ignore all packages with @SocketSecurity ignore-all

  • @SocketSecurity ignore npm/istanbul-lib-report@3.0.1
  • @SocketSecurity ignore npm/extension-port-stream@3.0.0
  • @SocketSecurity ignore npm/diff@5.2.0
  • @SocketSecurity ignore npm/superstruct@1.0.4
  • @SocketSecurity ignore npm/micromatch@4.0.7
  • @SocketSecurity ignore npm/node-gyp@10.2.0

@MajorLift MajorLift changed the title chore(deps): replace 'superstruct' imports with '@metamask/superstruct' chore(deps): replace superstruct imports with @metamask/superstruct Jul 14, 2024
@danroc
Copy link
Contributor

danroc commented Jul 16, 2024

Replaced by #355

@danroc danroc closed this Jul 16, 2024
MajorLift added a commit to MetaMask/snaps that referenced this pull request Jul 17, 2024
#2445)

## Explanation

As part of the Wallet Framework Team's OKR ([Q2 2024
O3KR4](https://docs.google.com/document/d/1NYWepE--9HLG9NHAxmsHKQI7lfZL2nW0kv7UVd4q160/edit#bookmark=kix.h65bod9heydk),
[Q3 2024
O2KR4](https://docs.google.com/document/d/1JLEzfUxHlT8lw8ntgMWG0vQb5BAATcrYZDj0wRB2ogI/edit#bookmark=kix.yxz96lxhvjk3))
for upgrading TypeScript to v5.0+ in the core monorepo, we are updating
dependencies of the core repo so that they can be used with projects
that use `Node16` or `NodeNext` as their `moduleResolution` tsconfig
option.

To achieve this, all dependencies that are ESM packages need to be
updated so that they generate separate builds and type declarations that
are explicitly designated for CJS and ESM.

This requirement applies to nested dependencies as well, so we are also
replacing `superstruct` with the ESM-compatible fork
`@metamask/superstruct` in all dependencies of core that have
`superstruct` as a dependency.

## Description

- [x] Replace `superstruct` dependency with `@metamask/superstruct`
`^3.0.0`.
  - [x] `^3.1.0`
- [x] Replace all `superstruct` import statements with
`@metamask/superstruct`
- [x] Bump `@metamask/utils` to `^8.5.0`.
  - [x] `^9.1.0`
    - [x] remove yarn resolution to `@metamask/superstruct@npm:3.1.0`
- [ ] ~If feasible without too much additional work:~ -> create separate
PRs for these tasks
  - [ ] ~Bump `typescript` to `~5.0.4`~
  - [ ] ~#2514

Further context on why the `superstruct` and `utils` changes are
necessary:

- MetaMask/utils#144
- MetaMask/superstruct#1
- MetaMask/superstruct#18
- MetaMask/utils#182
- MetaMask/metamask-module-template#247

### `yarn resolutions`

`@metamask/utils` is pinned to `^9.1.0` via yarn resolutions, as there
are a large number of dependencies that are set to `^8.5.0` (see below),
and some of them (especially the core packages) are blocked by the merge
and release of this PR:

- core monorepo: `approval-controller`, `base-controller`,
`controller-utils`, `eth-json-rpc-provider`, `json-rpc-engine`,
`json-rpc-middleware-stream`, `permission-controller`
- standalone: `browser-passworder`, `create-release-branch`,
`eth-block-tracker`, `eth-json-rpc-middleware`, `eth-sig-util`,
`key-tree`, `post-message-stream`, `providers`, `snaps-registry`

Mixed usage of `utils` v8 and v9 anywhere in the monorepo's dependency
tree causes the following type errors:
-
https://github.com/MetaMask/snaps/actions/runs/9720788583/job/26900545288?pr=2445

### Release order roadmap

Due to interdependencies between the packages involved in this PR, we
will need to update and release them in a specific order:

- [ ] Merge this PR: #2445
- Request snaps team to hold off on releases until yarn resolution for
utils can be removed
- Bump `@metamask/utils` in dependencies of
`snaps-{sdk,utils,rpc-methods}`:
    - [x] `rpc-errors`: 
      - [x] MetaMask/rpc-errors#147
      - [x] MetaMask/rpc-errors#148
    - [x] `key-tree`
      - [x] MetaMask/key-tree#181
      - [x] MetaMask/key-tree#182
    - [x] `snaps-registry`
      - [x] MetaMask/snaps-registry#693
      - [x] MetaMask/snaps-registry#694
    - [x] `base-controller`, `permission-controller`
      - [x] MetaMask/core#4516
      - [x] MetaMask/core#4517
- [ ] Release `snaps-sdk`
- [x] Release `keyring-api`:
MetaMask/keyring-api#328
    - [ ] Release `snaps-utils`
      - [ ] Release `snaps-rpc-methods`
- [ ] Merge core PR: MetaMask/core#3645
- Before merging, first remove yarn resolutions for `snaps-sdk`,
`snaps-utils`, `keyring-api`
- [ ] Release all core pkgs (especially deps of `snaps-controllers` and
consumers of `utils`)
- Exclude core pkgs that have `snaps-controllers` as dependency:
`{accounts,chain,profile-sync}-controller`
- [ ] Bump and release all remaining `@metamask/utils@8.x.x` usage in
the `snaps-controllers` dependency tree
  - [x] `browser-passworder`
    - [x] MetaMask/browser-passworder#63
    - [x] MetaMask/browser-passworder#64
  - [x] `post-message-stream`
    - [x] MetaMask/post-message-stream#140
- ~Blocked by build error
https://github.com/MetaMask/post-message-stream/actions/runs/9863225191/job/27235524010?pr=140~
    - [x] MetaMask/post-message-stream#141
- [ ] Release `snaps-controllers`
- [ ] Release `{accounts,chain,profile-sync}-controller`,
`eth-snap-keyring`
(MetaMask/eth-snap-keyring#311)
- [ ] Bump and release all remaining `@metamask/utils@8.x.x` usage in
the snaps monorepo dependency tree
  - [ ] `create-release-branch`
    - [x] MetaMask/create-release-branch#150
    - [x] MetaMask/create-release-branch#149
  - [ ] `eth-block-tracker`
    - ~Blocked by `eth-json-rpc-provider`~
- [ ] Blocked by MetaMask/eth-block-tracker#252
  - [ ] `eth-json-rpc-middleware`
- Blocked by `eth-block-tracker`, ~`eth-json-rpc-provider`,~
`eth-sig-util`, ~`json-rpc-engine`~
  - [x] `abi-utils`
    - [x] MetaMask/abi-utils#80
    - [x] MetaMask/abi-utils#81
  - [ ] `eth-sig-util`
    - ~Blocked by `abi-utils`~
    - [x] MetaMask/eth-sig-util#381
    - [x] MetaMask/eth-sig-util#382
  - [x] `providers`
- ~Blocked by `json-rpc-engine`, `json-rpc-middleware-stream`,
`rpc-errors`~
    - [x] MetaMask/providers#345
    - [x] MetaMask/providers#347
- [ ] Remove yarn resolution for `@metamask/utils`
- [ ] Release all remaining snaps pkgs
  - New snaps monorepo releases are now safe to publish.

## References

- Closes #2444
- Followed by #2514
- Blocked by type errors caused by simultaneous usage of
`@metamask/utils` `9.0.0` and `8.5.0` in dependency tree.
- All tests passing with `@metamask/utils` fixed to `9.0.0` in yarn
resolutions:

https://github.com/MetaMask/snaps/actions/runs/9745954683/job/26895208572?pr=2445
- [ ] `snaps-sdk`, `snaps-utils` and their dependencies that use
`@metamask/utils@8.5.0` are blocking:
- [ ] core release via MetaMask/core#3645, which
is blocking:
      - [ ] `snaps-controllers` type errors
    - [ ] `snaps-rpc-methods` type errors
    - [ ] `{accounts,chain,profile-sync}-controller` as dependencies
- `@metamask/providers` update to `^17.1.0` is being blocked by
MetaMask/providers#340,
ts-bridge/ts-bridge#22
- Causes CI failure:
https://github.com/MetaMask/snaps/actions/runs/9783767567/job/27013136688?pr=2445
  - [x] Fix: MetaMask/providers#347

## Changelog

### `@metamask/snaps-cli` 

```md
### Changed
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#2445](#2445))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
```

### `@metamask/snaps-controllers`

```md
### Changed
- Bump `@metamask/base-controller` from `^6.0.0` to `^6.0.1` ([#2445](#2445))
- Bump `@metamask/permission-controller` from `^10.0.0` to `^10.0.1` ([#2445](#2445))
- Bump `@metamask/rpc-errors` from `^6.2.1` to `^6.3.1` ([#2445](#2445))
- Bump `@metamask/snaps-registry` from `^3.1.0` to `^3.2.1` ([#2445](#2445))
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))
```

### `@metamask/snaps-execution-environments`

```md
### Changed
- Bump `@metamask/rpc-errors` from `^6.2.1` to `^6.3.1` ([#2445](#2445))
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#2445](#2445))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
- Set `@metamask/providers` from `^17.0.0` to `17.0.0` ([#2445](#2445))
  - `17.1.0` and `17.1.1` introduce regressions.
```

### `@metamask/snaps-jest`

```md
### Changed
- Bump `@metamask/base-controller` from `^6.0.0` to `^6.0.1` ([#2445](#2445))
- Bump `@metamask/key-tree` from `^9.1.1` to `^9.1.2` ([#2445](#2445))
- Bump `@metamask/permission-controller` from `^10.0.0` to `^10.0.1` ([#2445](#2445))
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#2445](#2445))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
```

### `@metamask/snaps-rpc-methods`

```md
### Changed
- Bump `@metamask/key-tree` from `^9.1.1` to `^9.1.2` ([#2445](#2445))
- Bump `@metamask/permission-controller` from `^10.0.0` to `^10.0.1` ([#2445](#2445))
- Bump `@metamask/rpc-errors` from `^6.2.1` to `^6.3.1` ([#2445](#2445))
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#2445](#2445))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
```

### `@metamask/snaps-sdk`

```md
### Changed
- Bump `@metamask/key-tree` from `^9.1.1` to `^9.1.2` ([#2445](#2445))
- Bump `@metamask/rpc-errors` from `^6.2.1` to `^6.3.1` ([#2445](#2445))
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#2445](#2445))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
- Set `@metamask/providers` from `^17.0.0` to `17.0.0` ([#2445](#2445))
  - `17.1.0` and `17.1.1` introduce regressions.
```

### `@metamask/snaps-simulator` (major)

```md
### Changed
- **BREAKING:** Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))
  - Due to the return type of `bigIntToHex` being narrowed from `string` to `Hex`, the return type of `hexlifyTransactionData` is narrowed from an object of type `Record<keyof transaction, string>` to an object of type `Record<keyof transaction, Hex>`, where `transaction` is of type `Omit<TransactionFormData, 'transactionOrigin' | 'chainId'>`
- Bump `@metamask/base-controller` from `^6.0.0` to `^6.0.1` ([#2445](#2445))
- Bump `@metamask/key-tree` from `^9.1.1` to `^9.1.2` ([#2445](#2445))
- Bump `@metamask/permission-controller` from `^10.0.0` to `^10.0.1` ([#2445](#2445))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#2445](#2445))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
```

### `@metamask/snaps-utils`

```md
### Changed
- Bump `@metamask/base-controller` from `^6.0.0` to `^6.0.1` ([#2445](#2445))
- Bump `@metamask/key-tree` from `^9.1.1` to `^9.1.2` ([#2445](#2445))
- Bump `@metamask/permission-controller` from `^10.0.0` to `^10.0.1` ([#2445](#2445))
- Bump `@metamask/rpc-errors` from `^6.2.1` to `^6.3.1` ([#2445](#2445))
- Bump `@metamask/snaps-registry` from `^3.1.0` to `^3.2.1` ([#2445](#2445))
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))

### Fixed
- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#2445](#2445))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
```

### `@metamask/snaps-webpack-plugin`

```md
### Changed
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))
```

### `@metamask/test-snaps`

```md
### Changed
- Bump `@metamask/utils` from `^8.3.0` to `^9.1.0` ([#2445](#2445))

### Fixed
- Set `@metamask/providers` from `^17.0.0` to `17.0.0` ([#2445](#2445))
  - `17.1.0` and `17.1.1` introduce regressions.
```
MajorLift added a commit to MetaMask/core that referenced this pull request Jul 22, 2024
…ode16` (#3645)

## Explanation

### TypeScript v5.0

As part of the Wallet Framework Team's OKRs (Q2 2024 O3KR4, Q3 2024 O2KR4),
we are upgrading TypeScript to v5.0+ for all packages in the core
monorepo.

These upgrades will give us access to new features, aid us in writing
more type-safe and modern code, and also allow us to reach parity with
Extension and other MetaMask projects.

### `Node16`

In order to maximize the benefits of this upgrade, we are also enabling
the `Node16` setting for the `module` and `moduleResolution` tsconfig
options.

## Motivation

### Interop: CJS modules referencing ESM modules

The core monorepo is a collection of CJS packages, which use CJS module
resolution rules internally, and are treated as CJS modules by Node.js.
This is true despite that fact that these packages are authored in
TypeScript using ESM syntax (`import`/`export` statements) and are set
up to output dual builds for both CJS and ESM.

With `Node16` or `NodeNext` enabled, CJS modules are unable to reference
ESM modules via static/synchronous `import` statements, as TypeScript
assumes them to be compiled down to CJS-only `require` statements.

There are three solutions for this issue, of which we are utilizing the
first two:
1. Update the ESM-only dependency so that it outputs a CJS build and
type declaration as well.
2. Replace the static `import` statements with dynamic import syntax
(which, based on CJS emit rules, are not transformed to `require`
statements).
3. Migrate our module to ESM (by setting `"type": "module"` in
package.json and renaming all of our scripts to *.cjs)

> See
https://www.typescriptlang.org/docs/handbook/modules/reference.html#interoperability-rules

With dependencies that we control or use extensively, we pursue the
first option, as we're doing with `superstruct` and `@metamask/utils`
(and all of the many core dependencies that are downstream of either or
both).

With dependencies that see more limited usage, we are opting for either
the second option (e.g. `multiformats`) or, if available, using a
CJS-compatible alternative (e.g. replacing `lodash-es` with `lodash`).

The third solution of migrating to ESM is the most fundamental,
long-term measure, but we are refraining from it at this stage until we
can make a concerted effort to migrate our codebase as a whole. This is
because any individual ESM migration can cause a cascading effect
through the dependency tree where other packages are now required to
migrate as well.

### Reasons for moving away from `node`/`node10`

The following outlines the motivation for switching to `Node16`, backed
by relevant entries from the official TypeScript documentation.
- Starting with TypeScript v5.0, the previously used `moduleResolution`
setting `node` is renamed to `node10`, and strongly discouraged from
usage.
- "It [reflects the CommonJS module resolution algorithm as it existed
in Node.js versions earlier than
v12](https://www.typescriptlang.org/docs/handbook/modules/reference.html#node10-formerly-known-as-node).
**It should no longer be used.**"
- "Because `node16` and `nodenext` are the only module options that
reflect the complexities of Node.js’s dual module system, they are the
[**only correct module options** for all apps and libraries that are
intended to run in Node.js v12 or
later](https://www.typescriptlang.org/docs/handbook/modules/reference.html#node16-nodenext),
whether they use ES modules or not."
- The `node10` setting is unable to guarantee correct module resolution
for ESM-only dependencies.
- "[Because Node.js v12 introduced different module resolution rules for
ES
modules,](https://www.typescriptlang.org/docs/handbook/modules/theory.html#module-resolution)
though, it’s a very bad model of modern versions of Node.js. It should
not be used for new projects."
- "`node16` and `nodenext` describe the full range of behavior for
Node.js’s dual-format module system, and **emit files in either CommonJS
or ESM format**. This is different from every other `module` option,
which are runtime-agnostic and force all output files into a single
format, [leaving it to the user to ensure the output is valid for their
runtime.](https://www.typescriptlang.org/docs/handbook/modules/reference.html#node16-nodenext)"
- The `node10` setting does not support the package.json `"exports"`
field, which is used in our libraries to expose dual builds and type
declarations.
-
https://www.typescriptlang.org/docs/handbook/modules/reference.html#packagejson-exports
- The `Node16` and `NodeNext` settings maximize downstream
compatibility.
- "When compiling a library, you don’t know where the output code will
run, but you’d like it to run in as many places as possible. Using
"module": "nodenext" (along with the implied "moduleResolution":
"nodenext") is the best bet for maximizing the compatibility of the
output JavaScript’s module specifiers, since [it will force you to
comply with Node.js’s stricter rules for import module
resolution.](https://www.typescriptlang.org/docs/handbook/modules/theory.html#module-resolution-for-libraries)"
- "`"moduleResolution": "nodenext"` is only checking that the output
works in Node.js, but in most cases, [module code that works in Node.js
will work in other runtimes and in
bundlers](https://www.typescriptlang.org/docs/handbook/modules/theory.html#module-resolution-for-libraries)"
  
#### `Node16` vs. `NodeNext`

The two settings are currently identical, and `Node16` is intended to
work with all current node versions v16 or higher. If additional
capabilities are added to `NodeNext` or `Node18`/`Node20` that we want
to apply to our codebases, we will be able to introduce them after
checking for disruptive regressions or breaking changes.

#### Relationship with other tsconfig options

- `--module` `nodenext` or `node16` implies and enforces the
`moduleResolution` with the same name (and vice versa).
- `--module` `node16` implies (up to) `--target` `es2022`.
  - `--module` `nodenext` implies (up to) `--target` `esnext`.
>
https://www.typescriptlang.org/docs/handbook/modules/reference.html#implied-and-enforced-options

## Description

- [x] Replace `superstruct` dependency with `@metamask/superstruct`
`^3.0.0`.
  - [x] `^3.1.0`
- [x] Replace all `superstruct` import statements with
`@metamask/superstruct`
- [x] Bump `@metamask/utils` to `^8.5.0`.
  - [x] `^9.0.0`
    - [x] remove yarn resolution to `@metamask/superstruct@npm:3.1.0`
- [x] Bump `typescript` to `~5.0.4`
  - [x] Set `module` and `moduleResolution` tsconfig options to `Node16`
  - [ ] ~#4507
  
Further context on why the `superstruct` and `utils` changes are
necessary:
- MetaMask/utils#144
- MetaMask/superstruct#1
- MetaMask/superstruct#18
- MetaMask/utils#182
- MetaMask/metamask-module-template#247
-
https://www.typescriptlang.org/docs/handbook/modules/guides/choosing-compiler-options.html#considerations-for-bundling-libraries

## Release order roadmap

Due to interdependencies between the packages involved in this PR, we
will need to update and release them in a specific order:

- [x] #4516
- [x] Release `{base,permission}-controller`
- [x] (wait for releases: `snaps-sdk`, `snaps-utils`,
`snaps-controllers`, `keyring-api`)
  - [x] MetaMask/keyring-api#356
  - [x] MetaMask/snaps#2445
  - [x] MetaMask/snaps#2584
  - [x] MetaMask/snaps#2589
- [x] Remove yarn resolutions for `snaps-sdk`, `snaps-utils`,
~`keyring-api`~
- [ ] Merge this PR: #3645
- [x] Set yarn resolution for `@metamask/providers` via
`@metamask/snaps-sdk` to `17.1.1`
- [x] Leave messages in Changelog for affected packages
(`{accounts,chain,profile-sync}-controller`) to hold off on new releases
    ```md
    ### Uncategorized

- Please hold off on new releases of this package until the yarn
resolution for `@metamask/providers` is removed.
- This is blocked by a `@metamask/snaps-sdk` release with
`@metamask/providers` bumped to `>=17.1.1`.
- See: [Fix regressions introduced by
@metamask/providers@17.1.1](MetaMask/snaps#2579)
- Build error fixed by yarn resolution:
[MetaMask/core/actions/runs/10011688901/job/27675682526?pr=3645](https://github.com/MetaMask/core/actions/runs/10011688901/job/27675682526?pr=3645)
    ```
- [ ] Release all core pkgs (especially deps of `snaps-controllers` and
consumers of `utils`)
- Exclude core pkgs that have `snaps-controllers` as dependency, and are
affected by `@metamask/providers` yarn resolution
    - `{accounts,chain,profile-sync}-controller`
- [ ] Release `snaps-controllers`
- [ ] Release `{accounts,chain,profile-sync}-controller`,
`eth-snap-keyring`
(MetaMask/eth-snap-keyring#311)
- [ ] (release remaining `snaps` packages while releasing
`@metamask/utils@9.0.0` version bumps for all dependencies and nested
dependencies)

## References

- Contributes to #3651
- Blocked by:
  - `superstruct`
    - [x] MetaMask/superstruct#18
    - [x] MetaMask/superstruct#24
    - [x] MetaMask/superstruct#25
    - [x] MetaMask/superstruct#28
  - `utils`
    - [x] MetaMask/utils#185
    - [x] MetaMask/utils#191
    - [x] MetaMask/utils#194
    - [x] MetaMask/utils#196
- Blocked by downstream consumers of `superstruct`, `utils`:
  - `abi-utils`: 
    - [x] MetaMask/abi-utils#73
    - [x] MetaMask/abi-utils#78
    - [x] MetaMask/abi-utils#80
    - [x] MetaMask/abi-utils#81
  - `chain-api`: 
    - [x] MetaMask/accounts-chain-api#5
- [x] https://github.com/MetaMask/accounts-chain-api/releases/tag/v0.1.0
  - `eth-simple-keyring`: 
    - [x] MetaMask/eth-simple-keyring#177
    - [x] MetaMask/eth-simple-keyring#178
  - `providers`:
    - [x] MetaMask/providers#336
    - [x] MetaMask/providers#337
- Blocked by MetaMask/providers#340,
ts-bridge/ts-bridge#22
- Causes CI failure:
https://github.com/MetaMask/snaps/actions/runs/9783767567/job/27013136688?pr=2445
    - [x] MetaMask/providers#345
    - [x] MetaMask/providers#347
  - `rpc-errors`: 
    - [x] MetaMask/rpc-errors#147
    - [x] MetaMask/rpc-errors#148    
  - `snaps-registry`:
    - [x] MetaMask/snaps-registry#613
    - [x] MetaMask/snaps-registry#670
    - [x] MetaMask/snaps-registry#693
    - [x] MetaMask/snaps-registry#694
- Blocked by `snaps` monorepo releases
  - `snaps-sdk`, `snaps-utils`
    - `keyring-api`
      - [x] MetaMask/keyring-api#328
  - `snaps-controllers` 
    - `eth-snap-keyring`
      - [x] MetaMask/eth-snap-keyring#311

## Changelog

### `@metamask/accounts-controller`

```md
### Changed

- Bump `@metamask/keyring-api` from `^8.0.0` to `^8.0.1` ([#3645](#3645))
- Bump `@metamask/snaps-sdk` from `^4.2.0` to `^6.1.0` ([#3645](#3645))
- Bump `@metamask/snaps-utils` from `^7.4.0` to `^7.8.0` ([#3645](#3645))
- Bump peer dependency `@metamask/snaps-controllers` from `^8.1.1` to `^9.3.0` ([#3645](#3645))
```

### `@metamask/assets-controllers` (major)

```md
### Changed

- **BREAKING:** `getIpfsCIDv1AndPath`, `getFormattedIpfsUrl` are now async functions ([#3645](#3645))
- Add `immer` `^9.0.6` as a new dependenc. ([#3645](#3645))
- Bump `@metamask/abi-utils` from `^2.0.2` to `^2.0.3` ([#3645](#3645))
- Bump `multiformats` from `^9.5.2` to `^13.1.0` ([#3645](#3645))
```

### `@metamask/chain-controller`

```md
### Changed

- Bump `@metamask/chain-api` from `^0.0.1` to `^0.1.0` ([#3645](#3645))
- Bump `@metamask/keyring-api` from `^8.0.0` to `^8.0.1` ([#3645](#3645))
- Bump `@metamask/snaps-controllers` from `^8.1.1` to `^9.3.0` ([#3645](#3645))
- Bump `@metamask/snaps-sdk` from `^4.2.0` to `^6.1.0` ([#3645](#3645))
- Bump `@metamask/snaps-utils` from `^7.4.0` to `^7.8.0` ([#3645](#3645))
```

### `@metamask/keyring-controller`

```md
### Changed

- Bump `@metamask/eth-simple-keyring` from `^6.0.1` to `^6.0.2` ([#3645](#3645))
- Bump `@metamask/keyring-api` from `^8.0.0` to `^8.0.1` ([#3645](#3645))
- Set yarn resolution for `@metamask/snaps-sdk/@metamask/providers` to `17.1.1` ([#3645](#3645))
  - Remove once `@metamask/snaps-sdk` bumps its `@metamask/providers` version to `>=17.1.1`.
```
  
### `@metamask/network-controller` (minor)

```md
### Added

- Newly exports the following types: `AutoManagedNetworkClient`, `InfuraNetworkClientConfiguration`, `CustomNetworkClientConfiguration` ([#3645](#3645))
```

### `@metamask/profile-sync-controller`

```md
### Changed

- Bump dependency and peer dependency `@metamask/snaps-controllers` from `^8.1.1` to `^9.3.0` ([#3645](#3645))
- Bump `@metamask/snaps-sdk` from `^4.2.0` to `^6.1.0` ([#3645](#3645))
- Bump `@metamask/snaps-utils` from `^7.4.0` to `^7.8.0` ([#3645](#3645))
```

### `@metamask/transaction-controller`

```md
### Changed

- Bump `@metamask/keyring-api` from `^8.0.0` to `^8.0.1` ([#3645](#3645))
```

### `@metamask/user-operation-controller`

```md
### Fixed

- Replace `superstruct` with ESM-compatible `@metamask/superstruct` `^3.1.0` ([#3645](#3645))
  - This fixes the issue of this package being unusable by any TypeScript project that uses `Node16` or `NodeNext` as its `moduleResolution` option.
```

## Checklist

- [x] I've updated the test suite for new or updated code as appropriate
- [x] I've updated documentation (JSDoc, Markdown, etc.) for new or
updated code as appropriate
- [x] I've highlighted breaking changes using the "BREAKING" category
above as appropriate

---------

Co-authored-by: Elliot Winkler <elliot.winkler@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace superstruct with ESM-compatible fork @metamask/superstruct
3 participants