Evaluate node core modules within a vm.Context

[SimenB]

Is your feature request related to a problem? Please describe.

One of Jest's longest-standing issue are about globals in tests being different from Node's, see jestjs/jest#2549.

Short example illustrating the problem:

const {createContext, compileFunction} = require('vm');

const context = createContext({});

const code = `
  const fs = require('fs');
  const assert = require('assert');

  try {
    fs.readFileSync('/some/faulty/path', 'utf8')
  } catch (error) {
    assert(error instanceof Error)
  }
`;

const compiledFunction = compileFunction(code, ['require'], {
  parsingContext: context,
});

compiledFunction(require);

Run this example, and the instanceof test will fail. Remove parsingContext option, and it will pass.

Describe the solution you'd like

Jest implements its own require etc, which works great most of the time, but in the case of node core modules, it fallbacks to requiring the module outside of the context. AFAIK the JS source of the node core modules are embedded in the binary of Node, so we cannot read them and pass them though compileFunction, SourceTextModule or some such ourselves so the globals would be the same.

It would be great if Node provided some API allowing the JS of core modules to be evaluated within a vm.Context.

(I had originally given up on this, but seeing as #30709 solves the super old #855 I thought maybe we might be lucky enough that this is possible as well? 😀)

Describe alternatives you've considered

We have tried to mess around with setting Symbol.hasInstance and just injecting globals from the outside (although that breaks the encapsulation we're aiming for by using separate Contexts in the first place) without luck. It's also a moving target as we'd have to add that property to all globals.

---

FWIW the Jest issue has a $500 bounty which I'd be happy to give to someone solving this in core, or the JS Foundation if that's more correct.

[devsnek]

I think we could expose this somewhat painlessly, whether we want to or not being a separate question.

[SimenB]

Ah, that's promising! What would be the reasons to not add this (beyond a larger api surface)?

[addaleax]

I’ve talked about this recently with other people, partially in the context of #28823. This is definitely doable, but it’s also going to be a lot of work.

[SimenB]

Thanks for that link, that's also something that would be great for us! We currently inject them manually, but they pose the same issues (instanceof often failing and the fact people can mutate those globals and it'll leak between tests). Current code: https://github.com/facebook/jest/blob/9fbe3c5bd07002d569a1b4037d53556244a728cd/packages/jest-environment-node/src/index.ts#L38-L64

[joyeecheung]

If cross-context support is not necessary, this can be done with an explicit option to include these globals during context creation, which tells Node.js to run the corresponding setups and create a different set of builtins for the context (from the top of my head, it should also be possible to make the option switch on/off Node.js-specific non browser globals). I believe it would be more difficult, or at least require more serialization/deserialization overhead, if we want the builtins from different contexts to interoperate with each other, however.

AFAIK the JS source of the node core modules are embedded in the binary of Node, so we cannot read them and pass them though compileFunction, SourceTextModule or some such ourselves so the globals would be the same.

The source code is actually available through one of the process.bindings but they cannot be evaluated properly in the user land anyway because they need a set of internals to be passed into the compiled function to finish the setup.

[SimenB]

You're referring to the node globals part (#28823) right? I'm not 100% sure what you mean with cross-context. I'm reasonably sure we wouldn't want any cross context stuff, as long as we could still override them from the outside. For example we can install fake timers and interact with them from the outside. But we wouldn't want 2 context's messing with each other, beyond the place creating a context being able to access it. Is that considered cross-context and would make it harder?

[joyeecheung]

@SimenB By cross-context operations I mean for example, passing a sandbox into the createContext() call and access what's in sandbox from the inside, or returning something from an invocation of a compiled function evaluated on the context and use that returned result from the outside. That could introduce subtle issues when these objects are passed into builtins (on either side) as things like brand checks and instanceof checks may fail and most builtins aren't really prepared to handle objects created from a different context gracefully. But from the description in the OP and #31852 (comment) it seems the use case of Jest does not require something like this, I think an option for this kind of use case would be fairly doable (compared to the kind that needs cross-context interop), though it would still need some internal cleanups as some internal hooks in Node.js currently assume they are evaluated in the main context (most ENVIRONMENT_STRONG_PERSISTENT_VALUES, I think) and depend on references to the set of things in the main context.

[SimenB]

Ah okay, thanks for elaborating! The use case in the OP should be perfectly fine with that limitation, then.

As for globals in user code, that limitation would present issues for us I think. For example our require implementation is passed from the parent context. The example in the OP is a bit too simplified perhaps, instead compiledFunction(require) what we really do is compiledFunction(ourCustomRequireImplementation), and that function needs to be created from the outside. We'll also want to control (and replace dynamically) setTimeout and friends from the parent context for mocked timers. I'm personally fine with the function we pass in not being instanceof Function (as it's not today anyways), but that might not be a trade-off that core can make or limitation it can accept if adding a new API for this?

However, the code snippet I linked to would not be affected as in that case we just want what's inside the context to look like what's outside - we don't actually need to know what they are or access them.

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[SimenB]

Seeing as it was seen as (possibly) painless back in #31852 (comment), I'd still love to see this land! This would solve a long-standing issue in Jest which is the cause of unnecessary friction and causes people to think Jest is not fitting for testing plain Node apps (including messaging from projects in the nodejs organization on GH, which is... unfortunate)

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[joyeecheung]

By the way with the recent progress on ShadowRealms integration we should be closer to getting separate set of builtins implemented in vm contexts now. ShadowRealm mandates that each shadow realm has a completely isolated set of builtins, in the case of Node.js, that includes both JS builtins like Error or Array and Node.js builtins like Buffer and setTimeout. In the case of vm contexts we could consider making the requirement less strict (e.g. allowing side-effect-full builtins like fs to be replicated in a new context), the underlying work can be shared nonetheless, though I think there is still a substantial amount of refactoring to do to make the builtins replicatable i.e. it can't be painless if what you want is e.g. that fs.readFileSync() throws an Error that belongs to a new context - in many cases, the native layer of fs and almost all the infrastructure it depends on in Node.js assume that they only need to deal with the main context, so they always just use the main context to invoke callbacks or throw errors, these all need to be updated. Much of that churn need to be done for ShadowRealms integration anyway, but there is additional churn to update builtins that aren't supposed to be available in ShadowRealms (e.g. fs).

[SimenB]

Thanks for the update @joyeecheung! I'm very excited to hear about progress on this.

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[SimenB]

No bot - this might still be possible down the line when shadowrealms is further along

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[SimenB]

😢

[bnoordhuis]

Yeah... I suppose the question is: is someone volunteering to do the work? It's definitely possible to implement this feature request but it's also definitely a lot of work.

I'd like to see it happen but If no one is stepping up, then we might as well close this instead of letting it languish and pollute everyone's inbox every 6 months.

[github-actions]

There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[github-actions]

There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

For more information on how the project manages feature requests, please consult the feature request management document.

[yetithefoot]

Also interested in these changes. I came here from the issue with testing transformers.js with jest.
xenova/transformers.js#57
https://backend.cafe/should-you-use-jest-as-a-testing-library