Update react-native Jest preset to more closely match an actual device

[dstaley]

Introduction

In the Jest documentation, the react-native preset provided by React Native is described as follows:

The preset is a node environment that mimics the environment of a React Native app.

However, this is only true on a very basic level. The execution environment that tests are run in is much closer to Node than it is to JSC/Hermes. For example, code that relies on Node libraries such as crypto will pass in the test environment, but fail when run on an actual device.

I would love to see some effort around updating the Jest preset to more closely match the environment that code will actually run in. As a stretch goal, I'd really love to see unit tests actually run inside JSC and/or Hermes.

Details

There's two main points of compatibility that I think can affect users: APIs and features. As far as APIs go, I actually think the Jest preset matches fairly close. It polyfills nearly all of the APIs described in the React Native JavaScript Environment documentation.

Prevent the use of Node system modules

In terms of capabilities, the main one I'd like to see resolved is the ability of a user to import arbitrary modules. In the current setup.js file, one of the easiest solutions would be to add the following lines:

require('module').builtinModules.forEach(m => {
  jest.mock(m, function() { throw new Error(`Cannot require system module`)});
});

This would resut in a test error anytime the user attempted to import one of the modules included with Node. However, we probably need a slightly less blunt solution. (As far as specific messaging goes, it looks like Expo reports this error when attempting to import a Node standard library.)

Resolve packages using the react-native field

When bundling a dependency, Metro will use the react-native field from package.json. However, in the test environment, main is used. This results in loading a library that might use code/packages that aren't compatible with React Native.

I'm not sure whether Metro or Jest is responsible for resolving modules during testing, but if it's Jest, we can probably just update the preset to use a custom resolver.

Discussion points

1. What's the best way to fail tests that require() modules that won't be available on a device?
2. Are there other significant areas where Jest deviates from the on-device environment?
3. Is running tests in Hermes/JSC something that's possible without massive changes? (Out of scope given the difficulty and the fact that Jest is more focused on unit tests.)

[kelset]

I think that @cpojer is probably the best person to answer this

[SimenB]

RN should probably have its own test environment that's neither "node" or "browser". If that's actually JSC/Hermes or not would then be up to the env. It'd need to support the vm API so I'm not sure how feasible it is, but that's a possibility at least.

Polyfills could live in the env as well rather than using setupFiles or whatever RN is currently using in its preset.

---

As for Node core modules, I think an environment saying "don't support node core modules" makes sense and we can throw "module not found" for core modules in Jest. That's cleaner than you setting up mocks for them I believe. Might be mistaken, tho 🙂

[cpojer]

Thanks for bringing up this discussion. I will merge Pull Requests that improve the Jest Preset or sensible changes to Jest.

1. What's the best way to fail tests that require() modules that won't be available on a device?

I think we should likely overwrite the resolver and throw if it is a builtin module (use is-builtin-module). Note, though, that if people do things with fs or path or similar just for setting up tests, that would also break. A solution for that could be an optional global function part of the preset to enable requiring fs for tests only.

2. Are there other significant areas where Jest deviates from the on-device environment?

Yes, Jest diverges a ton from what happens on device. Jest does not run any native code for either Android, iOS or any out-of-tree platforms. This is out of scope for unit testing. For end-to-end tests, a platform-specific solution should be used instead of a JavaScript framework (unless, of course, you are interested in building a Jest driver for an e2e testing setup for iOS and Android, at which point you could write tests for React Native and native screens).

3. Is running tests in Hermes/JSC something that's possible without massive changes?

This is unfortunately not easy without very big changes. However, consider that Jest is meant for unit tests, not for end-to-end tests. This is similar to UI programming for the web where we emulate parts of the DOM via jsdom. The React Native environment for Jest should be what jsdom is for browsers.

[dstaley]

Thanks for bringing up this discussion. I will merge Pull Requests that improve the Jest Preset or sensible changes to Jest.

Awesome! 😍 (Speaking of, I did open this PR to add fetch.)

I think we should likely overwrite the resolver and throw if it is a builtin module (use is-builtin-module).

Before we go that route, is there another option that might give the result of only disallowing those modules in code under test as opposed to actual test declaration code? Does the Jest resolver run for the actual unit test declarations?

Yes, Jest diverges a ton from what happens on device. Jest does not run any native code for either Android, iOS or any out-of-tree platforms. This is out of scope for unit testing.

Yeah, I think this is probably one of the easier differences to communicate to users, so I agree that it should be outside of the scope of the Jest preset.

This is unfortunately not easy without very big changes.

Roger that! I figured if it was easy it could be a quick shortcut to ensuring the test environment is pretty close to an actual device. I'll remove it from the first post to keep discussion focused.

I am curious what your thoughts are on modifying the Jest preset to prefer the react-native field in package.json if present. Can you think of any roadblocks there?

Thank you for your thoughts! I really appreciate them.

[cpojer]

I am curious what your thoughts are on modifying the Jest preset to prefer the react-native field in package.json if present. Can you think of any roadblocks there?

Yes, we should do that. It would be good if Jest had support for customizing the name of main fields like Metro does.

[SimenB]

It would be good if Jest had support for customizing the name of main fields like Metro does.

It does - just provide a custom resolver. There's an example in the docs which uses module instead of main: https://jestjs.io/docs/en/configuration#resolver-string

[vhakulinen]

3. Is running tests in Hermes/JSC something that's possible without massive changes? (Out of scope given the difficulty and the fact that Jest is more focused on unit tests.)

Wouldn't it be possible to boostrap hermes VM (e.g. load turbo native modules) and point it to a entry point for javascript tests?

This would avoid the need to run a device emulator for non-ui and non-platform specific tests (for example, database tests).