Circular references hang jest when assertions fail on node 14

[voces]

🐛 Bug Report

When an assertion fails where either the expected or actual value is circular, and both values are objects, jest encounters an error stating it failed to convert a circular structure to JSON, resulting in the test run not completing.

To Reproduce

it("test", () => {
  const foo = {};
  foo.ref = foo;

  expect(foo).toEqual({});
});

Running jest gives me the following error:

(node:11685) UnhandledPromiseRejectionWarning: TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    --- property 'ref' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (internal/child_process/serialization.js:117:20)
    at process.target._send (internal/child_process.js:804:17)
    at process.target.send (internal/child_process.js:702:19)
    at reportSuccess (/Users/verit/basic-jsx/node_modules/jest-worker/build/workers/processChild.js:67:11)

Jest continues running indefinitely (I only tested up to ten minutes) and reports nothing regarding the test suite.

I traced this to the added failureDetails property on error messages, landed in 26.3.0.

Expected behavior

I'd expect the test to fail and jest to complete running.

envinfo

I only tested two versions. The above error occurs on 14.9.0, but does not on 12.16.1.

  System:
    OS: macOS 10.15.6
    CPU: (8) x64 Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
  Binaries:
    Node: 14.9.0 - ~/.nodenv/versions/14.9.0/bin/node
    npm: 6.14.8 - ~/.nodenv/versions/14.9.0/bin/npm
  npmPackages:
    jest: ^26.4.2 => 26.4.2 

[Lonli-Lokli]

Is there any workaround?

[joelcoxokc]

@Lonli-Lokli Looks like the only workaround for now is --detectOpenHandles

However, this causes a massive decrease in performance.

I hope this can be fixed soon. I have migrated several of our projects to jest... and now it causes hiccups all throughout our build system when one test breaks.

It would be nice if --testTimeout worked in this scenario... but it still allows the test to just hang until Jenkins or circle ci times out.

[ziacik]

I can reproduce this when I have two such tests (in separate files - but not sure if this matters) and I have to run it with --watch. Here, however, it can be reproduced even without --watch: https://repl.it/@Frantiekiaik/jest-playground-1

[rimunroe]

I just ran into what I assume is the same issue on Node 10.16.0 and Jest 26.6.2. I can't reproduce it as written, but if I make a file containing two copies of @voces's example test and run it in watch mode, I get the same error.

The test:

it('test', () => {
  const foo = {};
  foo.ref = foo;

  expect(foo).toEqual({});
});
it('test 2', () => {
  const foo = {};
  foo.ref = foo;

  expect(foo).toEqual({});
});

The error:

(node:58951) ExperimentalWarning: The fs.promises API is experimental
(node:58951) UnhandledPromiseRejectionWarning: TypeError: Converting circular structure to JSON
    at JSON.stringify (<anonymous>)
    at process.target._send (internal/child_process.js:735:23)
    at process.target.send (internal/child_process.js:634:19)
    at reportSuccess (/Users/richardmunroe/analytics_ui/node_modules/jest-worker/build/workers/processChild.js:67:11)
(node:58951) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:58951) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Running with --detectOpenHandles makes the problem go away for me too.

[ziacik]

This is happening since #9496
Commit 9186361

And it seems to happen because of some inter-process serialization. After adding failureDetails property, the serialization fails on cyclic references as it contains the objects being under test. Maybe some sanitization of the failureDetails property would help.

[ziacik]

Note that after commit 5f6f2ec which changes default runner to circus, the error message is

"messageParent" can only be used inside a worker

      at messageParent (packages/jest-worker/build/workers/messageParent.js:46:11)

with JEST_JASMINE=1 the error is as before.

[ziacik]

Added a PR as an attempt to fix this.

Also, please note that the messageParent error mentioned above is due to this line which swallows the real error message which is also about circular references.

[SimenB]

Added a PR as an attempt to fix this.

🎉

Also, please note that the messageParent error mentioned above is due to this line which swallows the real error message which is also about circular references.

We should not swallow errors like that...

[piotrl]

We confirm it happens in Node 12, and it’s more common when using Angular Dependency Injection (I think they have cyclic structures in some error-states).

The process hangs in such scenario, but this can be improved slightly be applying --unhandled-rejection=strict to nodejs script, instead running jest as separate binary. It helps jest to recover and fail suite (but it does not resolve cyclic reference of course).

[daton89]

I can confirm this problem as very common with Angular DI with Node 12 as @piotrl mentioned.
Using Jest 26.6.3

How can I run it with --unhandled-rejection=strict flag?

    //package.json > scripts
    "test:unit": "node --unhandled-rejections=strict $(npm bin)/jest --env=jest-environment-jsdom-sixteen ",

this is actually stopping the execution, is that the right workaround?

[obalilty]

i was getting the same error for ZoneJs object with circular structure
was able to workaround this issue by adding to following to testWorker.js (line 127)

const sendMessageToJest = (eventName, args) => {
  // start here
  if (args[1].failureDetails) {
    args[1].failureDetails.forEach((fd) => {
      fd.rejection = null;
      fd.promise = null;
      fd.zone = null;
      fd.task = null;
    })
  };
  // end here
  (0, _jestWorker().messageParent)([eventName, args]);
};

now getting the real error with stacktrace message and without any "Converting circular structure to JSON" errors
you may need to make adjustments depending on your code

A more correct approach might be to set null for each property of failureDetails item except 'stack' and 'message' props

jest v28.1.0 or 29.5.0
node v16.19.0

[tim-sh]

With jest@29.5.0, the following generic workaround helped me:

function deCircle(o, seen = new Set()) {
  if (!o || typeof o !== 'object') return;
  Object.entries(o).forEach(([k,v]) => {
    if (seen.has(v)) {
      o[k] = null;
      return;
    }
    const s = new Set(seen);
    s.add(v);
    deCircle(v, s);
  })
}
function reportSuccess(result) {
  if (!process || !process.send) {
    throw new Error('Child can only be used on a forked process');
  }
  deCircle(result); // ← patching the object
  process.send([_types.PARENT_MESSAGE_OK, result]);
}

You might need to patch on error as well, depending on the circumstances.

[felipebutcher]

I had this problem a few minutes ago. Found out it happened when I had two test files with same name in different folders. Once I renamed one of the files the problem was gone.

[danielo515]

Glad to see this is not a general problem with testing eslint rules. In my case, when a test fails, because almost all eslint nodes have circular references this problem is quite common.

[kasir-barati]

Still happening even though I am just expecting a number to be there and even thought it is but still tests fails 😭
Look here: https://github.com/kasir-barati/nestjs-materials/blob/main/typeorm/src/modules/talent-e2e/get-all.e2e-spec.ts#L9

The Solution that worked for me

#10577 (comment)

[kasir-barati]

Coming back again to the same issue, it seems that this time adding --detectOpenHandles flag is not gonna help me, I cannot deduce the issue from the logs and when I add that flag test suit fails immediately after it reaches the point that it should make a http req via axios (so no room to log anything in my test suit) and just logs that axios request failed with 400 http status code but in reality it even won't enter the catch block, any idea?

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'res' -> object with constructor 'Object'
    --- property 'req' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:159:20)
    at process.target._send (node:internal/child_process:852:17)
    at process.target.send (node:internal/child_process:752:19)
    at reportSuccess (/home/kasir/projects/you-say/node_modules/jest-worker/build/workers/processChild.js:82:11)

Node.js v20.10.0
 FAIL   backend-e2e  apps/backend-e2e/src/auth/auth-business.spec.ts
  ● Test suite failed to run

    Jest worker encountered 4 child process exceptions, exceeding retry limit

      at ChildProcessWorker.initialize (../../node_modules/jest-worker/build/workers/ChildProcessWorker.js:181:21)

[sawvox]

Coming back again to the same issue, it seems that this time adding --detectOpenHandles flag is not gonna help me, I cannot deduce the issue from the logs and when I add that flag test suit fails immediately after it reaches the point that it should make a http req via axios (so no room to log anything in my test suit) and just logs that axios request failed with 400 http status code but in reality it even won't enter the catch block, any idea?

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'res' -> object with constructor 'Object'
    --- property 'req' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:159:20)
    at process.target._send (node:internal/child_process:852:17)
    at process.target.send (node:internal/child_process:752:19)
    at reportSuccess (/home/kasir/projects/you-say/node_modules/jest-worker/build/workers/processChild.js:82:11)

Node.js v20.10.0
 FAIL   backend-e2e  apps/backend-e2e/src/auth/auth-business.spec.ts
  ● Test suite failed to run

    Jest worker encountered 4 child process exceptions, exceeding retry limit

      at ChildProcessWorker.initialize (../../node_modules/jest-worker/build/workers/ChildProcessWorker.js:181:21)

Trying to stringify the req object from Axios will result in the circular reference: req -> res -> req
As your error states:

--> starting at object with constructor 'Object'
| property 'res' -> object with constructor 'Object'
--- property 'req' closes the circle

Try comparing the properties of the request or response object that you care about rather than the whole object itself.

I.e. expect(res.body?.myProperty).toEqual("foo")

[kasir-barati]

So @sawvox what you're saying is that if I just compare what I wanted in the request or response object I would not face this issue but I guess that is not something to compromise, But thanks for uncovering one more mystery for me. I did not know that it is happening because of my expects.

[franzos]

Today I've started seeing similar errors on a repository that I've been working on for 4 years. Funny thing is, it's really random - sometimes it works, sometimes it doesn't. Flags like --runInBand seem the increase the likelihood that it works, but it's no guarantee.

node:internal/child_process/serialization:159
    const string = JSONStringify(message) + '\n';
                   ^

TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'Object'
    |     property 'socket' -> object with constructor 'Object'
    --- property '_httpMessage' closes the circle
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:159:20)
    at process.target._send (node:internal/child_process:838:17)
    at process.target.send (node:internal/child_process:738:19)
    at reportSuccess (/usr/src/app/node_modules/.pnpm/jest-worker@29.7.0/node_modules/jest-worker/build/workers/processChild.js:82:11)

Node.js v18.16.1

Also tried the suggested workerThreads: true but this will lead to a different error:

    DataCloneError: function transformRequest(data, headers) {
        const contentType = headers.getContentType() || '';
        const ha...<omitted>... } could not be cloned.

      at messageParent (../../node_modules/.pnpm/jest-worker@29.7.0/node_modules/jest-worker/build/workers/messageParent.js:24:34)

This is on Jest 29.5.12.

[steebchen]

Also running into this, the worst thing is that the stacktrace only shows files in node_modules, which makes it extremely hard to debug where the error exactly came from in your tests. Running into this with node@v20.10.0 and jest@29.7.0

[franzos]

which makes it extremely hard to debug where the error exactly came from in your tests

If you have a rough idea where it's coming from, you can wrap the section with try/catch, to get the actual error message:

try {
   // TypeError: Converting circular structure to JSON
} catch(err) {
   console.error(err) // actual error message
}

It's important not to expose the raw error to jest.

[tim-sh]

Seems to be fixed with latest jest@29.
Tested with the original test cited by @voces on Node.js versions 14, 18, and 20.

[kpervin]

We are getting this issue when an Axios error is thrown and we run the following:

await service.fnThatCallsAxios().catch( (e) => expect(e?.message).toBeUndefined() );

Currently using jest: ^29.7.0

[stefanonardo]

You don't actually need to expect. You can just throw an AxiosError (or any Error with a circular reference) and the child process will fail to report.

It still happens in jest@29.7.0

[SimenB]

https://github.com/jestjs/jest/releases/tag/v30.0.0-alpha.6

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.