[Bug]: Modified ArrayBuffer exception when running Jest with SES enabled

[samsiegart]

Version

27.2.5

Steps to reproduce

1. Clone my repo at https://github.com/samsiegart/jest-ses-repro
2. yarn install
3. yarn test

Expected behavior

The test runs and passes.

Actual behavior

Test suite fails to run with:

 FAIL  ./sum.test.js
  ● Test suite failed to run

    Unexpected intrinsic intrinsics.ArrayBuffer.__proto__ at %FunctionPrototype%

Additional context

I am using SES in my application. Part of what SES does is freezes certain objects and types, like ArrayBuffer. It does this by invoking a lockdown function, which happens when doing import '@agoric/install-ses' in setupTests.js in my repo. It appears that Jest is modifying the ArrayBuffer constructor in some way, causing this to break, and I would like to figure out where and why, and hopefully fix it so that Jest is compatible with SES.

I have also tried running lockdown before Jest is loaded and executed, and I get a similar, but more opaque error:

TypeError: Cannot assign to read only property 'constructor' of object '[object Object]'

I believe this happens for the same reason. After ArrayBuffer is frozen, when Jest tries to modify the constructor, it throws an error.

Any pointers to where this modification could be happening, or how to get a more informative stack trace to appear, would be greatly appreciated.

Environment

npx: installed 1 in 0.652s

  System:
    OS: Linux 5.10 Ubuntu 20.04.3 LTS (Focal Fossa)
    CPU: (16) x64 AMD Ryzen 7 5800X 8-Core Processor
  Binaries:
    Node: 14.16.0 - ~/.nvm/versions/node/v14.16.0/bin/node
    Yarn: 1.22.11 - /usr/local/bin/yarn
    npm: 6.14.11 - ~/.nvm/versions/node/v14.16.0/bin/npm
  npmPackages:
    jest: ^27.2.5 => 27.2.5

[erights]

Hi @samsiegart , your second symptom

TypeError: Cannot assign to read only property 'constructor' of object '[object Object]'

might be the JavaScript Standard's Override Mistake. SES has a mitigation for that. Try setting the lockdown overrideTaming option to 'severe'

lockdown({ overrideTaming: 'severe' });

See https://github.com/endojs/endo/blob/master/packages/ses/lockdown-options.md

[erights]

Please let us know if that changes the symptoms. Thanks.

[samsiegart]

@erights That did in fact fix the issue and the test was able to run. However, I discovered that the way I'm calling lockdown before the test is not effective. When I try to import a library requiring ses, such as eventual-send, in the component being tested, I get ReferenceError: harden is not defined. It appears that Jest is not running tests in the same context as the script that invokes it. Here's my testing script that calls lockdown and runs jest:

/* global lockdown */
require('ses');
lockdown({ errorTaming: 'unsafe', overrideTaming: 'severe' });

// Do this as the first thing so that any code reading it knows the right env.
process.env.BABEL_ENV = 'test';
process.env.NODE_ENV = 'test';
process.env.PUBLIC_URL = '';

// Makes the script crash on unhandled rejections instead of silently
// ignoring them. In the future, promise rejections that are not handled will
// terminate the Node.js process with a non-zero exit code.
process.on('unhandledRejection', err => {
  throw err;
});

// Ensure environment variables are read.
require('../config/env.cjs');
const jest = require('jest');
const execSync = require('child_process').execSync;
let argv = process.argv.slice(2);
function isInGitRepository() {
  try {
    execSync('git rev-parse --is-inside-work-tree', { stdio: 'ignore' });
    return true;
  } catch (e) {
    return false;
  }
}

function isInMercurialRepository() {
  try {
    execSync('hg --cwd . root', { stdio: 'ignore' });
    return true;
  } catch (e) {
    return false;
  }
}

// Watch unless on CI or explicitly running all tests
if (
  !process.env.CI &&
  argv.indexOf('--watchAll') === -1 &&
  argv.indexOf('--watchAll=false') === -1
) {
  // https://github.com/facebook/create-react-app/issues/5210
  const hasSourceControl = isInGitRepository() || isInMercurialRepository();
  argv.push(hasSourceControl ? '--watch' : '--watchAll');
}

jest.run(argv);

I'm not sure if it's possible at all to call lockdown before Jest in this manner, we may just need to call it after the testing environment is set up, in which case we get Unexpected intrinsic intrinsics.ArrayBuffer.__proto__ at %FunctionPrototype% currently.

[kriskowal]

This seems to be a symptom of Jest’s general approach to fixing instanceof when it blesses child VM contexts (Realms) with certain Node.js API’s from the main Node.js Realm (https://github.com/facebook/jest/blob/8c35c4d5581d0a27952851fb94723dc73842c97a/packages/jest-environment-node/src/index.ts#L42) as in response to #3186

However, fixing “identity discontinuity hazards” in this fashion leads to whacking moles, if not forever, for a very long time. For example, I expect that calling fetch() and then response.json() in a the child realm results in instanceof checks for Object and Array failing in the test realm. Every time you fix one instanceof check by punching another intrinsic from the incubating realm down to the child realm, it creates a new failure for any methods of that object that return non-primitives. Here’s another: new TextEncoder().encodeInto(buffer) instanceof Object likely fails.

It might be worthwhile to consider using vm2 instead, which provides a full “membrane” between realms.

🙏 @mhofman for isolating the issues above.

[SimenB]

Hi! Yeah, this sounds like a symptom of #2549. I think the real solution at some point is the combination of nodejs/node#28823 and nodejs/node#31852, but there's not much movement on either, unfortunately.

It might be worthwhile to consider using vm2 instead, which provides a full “membrane” between realms.

I doubt we can do that as we need the ESM APIs of vm. If it's just the context that might not be an issue? We'd also need JSDOM to use it though (we use their getInternalVMContext API)

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[erights]

This is worth re-examining

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity. Please open a new issue if the issue is still relevant, linking to this one.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.