Skip to content

Commit

Permalink
chore: Patch jest to not use JSON serialization in message passing (#…
Browse files Browse the repository at this point in the history
…5883)

We often get "Do not know how to serialize a BigInt" as an output from
our failing tests (see
[here](https://github.com/AztecProtocol/aztec-packages/actions/runs/8756999966/job/24035011971#step:5:465)
for an example). This is caused by jest using node `child_process` to
spawn multiple runners, and using json serialization for message
passing, so when a child wants to report a failure in a test, if that
failure involved a bigint, it triggers this error.

One solution is to migrate to jest's experimental worker threads, but
I'm not sure how memory is handled if we move from subprocesses to
threads. This solution is dirtier but allows us to keep using child
processes: it patches jest so it uses node's [advanced
serialization](https://nodejs.org/api/child_process.html#advanced-serialization)
which "is generally more powerful and supports more built-in JavaScript
object types, such as BigInt, Map and Set, ArrayBuffer and TypedArray,
Buffer, Error, RegExp etc".

I tested it on the same PR as I linked above and indeed we can now see
[failed bigint
expectations](https://github.com/AztecProtocol/aztec-packages/actions/runs/8757912954/job/24037750048#step:5:739)
as error messages.
  • Loading branch information
spalladino authored Apr 22, 2024
1 parent cec8191 commit 1c24c8e
Show file tree
Hide file tree
Showing 3 changed files with 45 additions and 2 deletions.
13 changes: 13 additions & 0 deletions yarn-project/.yarn/patches/jest-runner-npm-29.7.0-3bc9f82b58.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff --git a/build/index.js b/build/index.js
index 65c0ed180a1f44a5095f80d572aacb68be1db3da..3bb4938110a50a2eca1b2f01466b7be16c9c8145 100644
--- a/build/index.js
+++ b/build/index.js
@@ -124,7 +124,7 @@ class TestRunner extends _types.EmittingTestRunner {
enableWorkerThreads: this._globalConfig.workerThreads,
exposedMethods: ['worker'],
forkOptions: {
- serialization: 'json',
+ serialization: 'advanced',
stdio: 'pipe'
},
// The workerIdleMemoryLimit should've been converted to a number during
3 changes: 2 additions & 1 deletion yarn-project/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
"@aztec/bb.js": "portal:../barretenberg/ts",
"@noir-lang/acvm_js": "portal:../noir/packages/acvm_js",
"@noir-lang/types": "portal:../noir/packages/types",
"@noir-lang/noirc_abi": "portal:../noir/packages/noirc_abi"
"@noir-lang/noirc_abi": "portal:../noir/packages/noirc_abi",
"jest-runner@^29.7.0": "patch:jest-runner@npm%3A29.7.0#./.yarn/patches/jest-runner-npm-29.7.0-3bc9f82b58.patch"
}
}
31 changes: 30 additions & 1 deletion yarn-project/yarn.lock
Original file line number Diff line number Diff line change
Expand Up @@ -8951,7 +8951,7 @@ __metadata:
languageName: node
linkType: hard

"jest-runner@npm:^29.7.0":
"jest-runner@npm:29.7.0":
version: 29.7.0
resolution: "jest-runner@npm:29.7.0"
dependencies:
Expand Down Expand Up @@ -8980,6 +8980,35 @@ __metadata:
languageName: node
linkType: hard

"jest-runner@patch:jest-runner@npm%3A29.7.0#./.yarn/patches/jest-runner-npm-29.7.0-3bc9f82b58.patch::locator=%40aztec%2Faztec3-packages%40workspace%3A.":
version: 29.7.0
resolution: "jest-runner@patch:jest-runner@npm%3A29.7.0#./.yarn/patches/jest-runner-npm-29.7.0-3bc9f82b58.patch::version=29.7.0&hash=a79dea&locator=%40aztec%2Faztec3-packages%40workspace%3A."
dependencies:
"@jest/console": ^29.7.0
"@jest/environment": ^29.7.0
"@jest/test-result": ^29.7.0
"@jest/transform": ^29.7.0
"@jest/types": ^29.6.3
"@types/node": "*"
chalk: ^4.0.0
emittery: ^0.13.1
graceful-fs: ^4.2.9
jest-docblock: ^29.7.0
jest-environment-node: ^29.7.0
jest-haste-map: ^29.7.0
jest-leak-detector: ^29.7.0
jest-message-util: ^29.7.0
jest-resolve: ^29.7.0
jest-runtime: ^29.7.0
jest-util: ^29.7.0
jest-watcher: ^29.7.0
jest-worker: ^29.7.0
p-limit: ^3.1.0
source-map-support: 0.5.13
checksum: 8345cba67695e45a6cc1fddca470ba61d981c6fa3565b53728ba36678acf6ce7657868e7483ab34b75e433efd42f57fd2894031069c06ac48b7b22ac9950ff1d
languageName: node
linkType: hard

"jest-runtime@npm:^29.7.0":
version: 29.7.0
resolution: "jest-runtime@npm:29.7.0"
Expand Down

0 comments on commit 1c24c8e

Please sign in to comment.