Stryker with multiple Jest projects

[giacomoforlani]

Question
I created an application with multiple Jest projects because I need to tests node and jsdom environments.
When I launch stryker run, I encountered this exception:

ERROR DryRunExecutor One or more tests resulted in an error:
        Test runner crashed. Tried twice to restart it without any luck. Last time the error message was: Error: Error: Jest: Cannot use configuration as an object without a file path.
Error: Jest: Cannot use configuration as an object without a file path.
    at readConfig (/Users/user/Examples/next-test/node_modules/jest-config/build/index.js:188:13)

I saw that problem is due by line /Users/user/Examples/next-test/node_modules/jest-config/build/index.js:406:32
const parsedConfig = await readConfig(argv, projects[0]);

readConfig function expects a string as the second argument but, in my case, projects is an array of objects instead of an array of strings.

Anyone have an idea to fix or bypass this exception?

Environment

 "@stryker-mutator/core": "^4.0.0",
 "@stryker-mutator/jest-runner": "^4.0.0",
 "@types/jest": "^26.0.15",
 "jest": "^26.6.0",

stryker.config.js

/**
 * @type {import('@stryker-mutator/api/core').StrykerOptions}
 */
module.exports = {
  mutate: [
    "**/*.ts",
    "**/*.tsx",
    "!**/*.spec.*",
  ],
  packageManager: "yarn",
  reporters: ["html", "clear-text", "progress", "dashboard"],
  testRunner: "jest",
  coverageAnalysis: "all",
};

jest.config.js

const testPathIgnorePatternsDefault = [
  '<rootDir>[/\\\\](build|docs|node_modules|.next|.stryker-tmp)[/\\\\]'
];

module.exports = {
  roots: [
    '<rootDir>'
  ],
  moduleFileExtensions: [
    'js',
    'ts',
    'tsx',
    'json',
  ],
  transform: {
    '^.+\\.(ts|tsx)$': 'babel-jest',
  },
  transformIgnorePatterns: [
    '[/\\\\]node_modules[/\\\\].+\\.(ts|tsx)$',
  ],
  projects: [
    {
      name: 'jsdom',
      testPathIgnorePatterns: [
        ...testPathIgnorePatternsDefault,
        '/__tests__/node/',
      ],
      testRegex: '.*.spec.*',
      testEnvironment: 'jsdom',
    },
    {
      name: 'node',
      testPathIgnorePatterns: [
        ...testPathIgnorePatternsDefault,
      ],
      testRegex: '__tests__/node/.*.spec.*',
      testEnvironment: 'node',
    },
  ],
};

[nicojs]

Thanks for opening this issue. I personally don't have a lot of experience with jest. Probably the fastest way to debug this is to create a small reproduction project and add it here. You can either push it to git and give the link or attach a .zip file here.

[giacomoforlani]

Thank you to your reply @nicojs.
Here are the zip file containing the project: next-test.zip
It is already a small project with the only purpose to test this issue.

[textbook]

I just migrated a project of mine over to using Jest projects (for the same reason, to split jsdom and node environments) and that caused Strkyer to fail with the same error:

16:28:54 (2812) ERROR DryRunExecutor One or more tests resulted in an error:
	Test runner crashed. Tried twice to restart it without any luck. Last time the error message was: Error: Error: Jest: Cannot use configuration as an object without a file path.
Error: Jest: Cannot use configuration as an object without a file path.
    at readConfig (/home/runner/work/starter-kit/starter-kit/node_modules/jest-config/build/index.js:188:13)
    at /home/runner/work/starter-kit/starter-kit/node_modules/jest-config/build/index.js:447:18
    at Array.map (<anonymous>)
    at readConfigs (/home/runner/work/starter-kit/starter-kit/node_modules/jest-config/build/index.js:441:10)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Object.runCLI (/home/runner/work/starter-kit/starter-kit/node_modules/@jest/core/build/cli/index.js:230:59)
    at async JestGreaterThan25Adapter.run (/home/runner/work/starter-kit/starter-kit/node_modules/@stryker-mutator/jest-runner/src/jest-test-adapters/jest-greater-than-25-adapter.js:16:24)
    at async JestTestRunner.run (/home/runner/work/starter-kit/starter-kit/node_modules/@stryker-mutator/jest-runner/src/jest-test-runner.js:58:21)
    at async ChildProcessTestRunnerWorker.dryRun (/home/runner/work/starter-kit/starter-kit/node_modules/@stryker-mutator/core/src/test-runner/ChildProcessTestRunnerWorker.js:23:30)
    at async ChildProcessProxyWorker.handleCall (/home/runner/work/starter-kit/starter-kit/node_modules/@stryker-mutator/core/src/child-proxy/ChildProcessProxyWorker.js:62:28)

Digging a little, I figured out that the problem is to do with Jest's handling of the --config argument as JSON rather than a file path, and recreated it without Stryker involved. That let me find the bug report on Jest itself: jestjs/jest#7415.

From Stryker's perspective it seems that, to support Jest configurations containing projects, the --projects argument needs to be passed to the CLI too. Looking at Jest's docs this would need to be the project strings from this example:

{
  "projects": ["<rootDir>", "<rootDir>/examples/*"]
}

the displayNames from this example:

{
  "projects": [
    {
      "displayName": "test"
    },
    {
      "displayName": "lint",
      "runner": "jest-runner-eslint",
      "testMatch": ["<rootDir>/**/*.js"]
    }
  ]
}

and I don't know what you'd be able to pass if a project's an object without a displayName, which seems to be accepted (though discouraged):

Note: When using multi-project runner, it's recommended to add a displayName for each project. This will show the displayName of a project next to its tests.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[nicojs]

Reopening because of #2780 (comment)

[nicojs]

Note: I had to remove the 7.0 milestone, because we are blocked by jestjs/jest#14062