Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfaults #2347

Closed
revmischa opened this issue Nov 17, 2022 · 19 comments
Closed

Segfaults #2347

revmischa opened this issue Nov 17, 2022 · 19 comments
Labels
help wanted Extra attention is needed upstream

Comments

@revmischa
Copy link

revmischa commented Nov 17, 2022

Describe the bug

Vitest runs fine for me but once all my tests pass and it's done running, it segfaults.
It does not segfault for simple tests, but for more complex tests in my application, it segfaults. I suspect aws-sdk and/or prisma libraries. Segfaults definitely happen with prisma tests on linux. On macOS there is a stack trace which points at aws-crt-node. These might be unrelated segfaults, I don't know. Upgrading to latest aws-sdk libs didn't help (this helped a previous time I had segfaults #1488)

Setting threads: false in vite.config.ts makes no difference whatsoever as far as I'm able to observe

Reproduction

I don't have an easy reproduction right now, I will see if it's possible to make one. In the meantime I have two stack traces.

Linux (running under lldb)

https://paste.debian.net/1261041/

Mac

 ✓ backend/src/api/resolver/vacancy/get.test.ts (4) 908ms
Test Files  1 passed (1)
     Tests  1 passed | 3 skipped (4)
  Start at  20:43:08
  Duration  2.95s (transform 479ms, setup 13ms, collect 1.54s, tests 908ms)
Signal received: 11, errno: 0
  SIGSEGV @ 0x0x170
################################################################################
Stack trace:
################################################################################
1   aws-crt-nodejs.node                 0x000000010430fe44 s_print_stack_trace + 24
2   libsystem_platform.dylib            0x00000001a59342a4 _sigtramp + 56
3   aws-crt-nodejs.node                 0x000000010430fecc s_napi_context_finalize + 128
4   aws-crt-nodejs.node                 0x000000010430fecc s_napi_context_finalize + 128
5   node                                0x0000000100f5c658 _ZN10napi_env__14CallIntoModuleIRZN15node_napi_env__13CallFinalizerILb1EEEvPFvPS_PvS4_ES4_S4_EUlS3_E_ZNS1_18CallbackIntoModuleILb1ES7_EEvOT0_EUlS3_N2v85LocalINSC_5ValueEEEE_EEvOT_SB_ + 60
6   node                                0x0000000100f599d0 _ZN15node_napi_env__13CallFinalizerILb1EEEvPFvP10napi_env__PvS3_ES3_S3_ + 104
7   node                                0x0000000100f47088 _ZN6v8impl7RefBase8FinalizeEb + 68
8   node                                0x0000000100f597e4 _ZN10napi_env__11FinalizeAllEv + 40
9   node                                0x0000000100f5979c _ZN15node_napi_env__D2Ev + 40
10  node                                0x0000000100f59878 _ZN15node_napi_env__D0Ev + 12
11  node                                0x0000000100f5bef8 _ZN6v8impl12_GLOBAL__N_118ThreadSafeFunctionD1Ev + 92
12  node                                0x0000000100f5bf40 _ZN6v8impl12_GLOBAL__N_118ThreadSafeFunctionD0Ev + 12
13  node                                0x0000000100f5c1d0 _ZZN4node11Environment11CloseHandleI11uv_handle_sZN6v8impl12_GLOBAL__N_118ThreadSafeFunction26CloseHandlesAndMaybeDeleteEbEUlPS2_E_EEvPT_T0_ENUlS6_E_8__invokeES6_ + 436
14  libuv.1.dylib                       0x000000010335a8cc uv_run + 504
15  node                                0x0000000100f35134 _ZN4node11Environment14CleanupHandlesEv + 252
16  node                                0x0000000100f35764 _ZN4node11Environment10RunCleanupEv + 432
17  node                                0x0000000100edb398 _ZN4node15FreeEnvironmentEPNS_11EnvironmentE + 108
18  node                                0x0000000101003358 _ZN4node16OnScopeLeaveImplIZNS_6worker6Worker3RunEvE3$_0ED1Ev + 112
19  node                                0x0000000101002f70 _ZN4node6worker6Worker3RunEv + 1208
20  node                                0x00000001010058a4 _ZZN4node6worker6Worker11StartThreadERKN2v820FunctionCallbackInfoINS2_5ValueEEEEN3$_38__invokeEPv + 56
21  libsystem_pthread.dylib             0x00000001a590606c _pthread_start + 148
22  libsystem_pthread.dylib             0x00000001a5900e2c thread_start + 8
Signal received: 11, errno: 0
  SIGSEGV @ 0x0x0
################################################################################
Stack trace:
################################################################################
1   aws-crt-nodejs.node                 0x000000010430fe44 s_print_stack_trace + 24
2   libsystem_platform.dylib            0x00000001a59342a4 _sigtramp + 56
3   libcrypto.1.1.dylib                 0x00000001038027e8 CRYPTO_THREAD_write_lock + 12
4   libcrypto.1.1.dylib                 0x00000001037e13e4 RAND_get_rand_method + 76
5   libcrypto.1.1.dylib                 0x00000001037e18b4 RAND_status + 12
6   node                                0x0000000101069554 _ZN4node6crypto6CSPRNGEPvm + 24
7   node                                0x0000000100f5862c _ZZN4node24InitializeOncePerProcessEiPPcNS_27InitializationSettingsFlagsENS_12ProcessFlags5FlagsEEN3$_18__invokeEPhm + 12
8   node                                0x00000001017a9860 _ZN2v84base21RandomNumberGeneratorC2Ev + 64
9   node                                0x0000000101194880 _ZN2v88internal7Isolate23random_number_generatorEv + 76
10  node                                0x00000001011e2b18 _ZN2v88internal4Heap18InitializeHashSeedEv + 48
11  node                                0x0000000101475a24 _ZN2v88internal20ReadOnlyDeserializer22DeserializeIntoIsolateEv + 224
12  node                                0x000000010122f614 _ZN2v88internal12ReadOnlyHeap21DeseralizeIntoIsolateEPNS0_7IsolateEPNS0_12SnapshotDataEb + 116
13  node                                0x000000010122f468 _ZN2v88internal12ReadOnlyHeap5SetUpEPNS0_7IsolateEPNS0_12SnapshotDataEb + 160
14  node                                0x0000000101193474 _ZN2v88internal7Isolate4InitEPNS0_12SnapshotDataES3_b + 1184
15  node                                0x0000000101193c40 _ZN2v88internal7Isolate16InitWithSnapshotEPNS0_12SnapshotDataES3_b + 12
16  node                                0x000000010147e364 _ZN2v88internal8Snapshot10InitializeEPNS0_7IsolateE + 292
17  node                                0x00000001010c54fc _ZN2v87Isolate10InitializeEPS0_RKNS0_12CreateParamsE + 308
18  node                                0x0000000101005584 _ZN4node6worker16WorkerThreadDataC2EPNS0_6WorkerE + 452
19  node                                0x0000000101002c3c _ZN4node6worker6Worker3RunEv + 388
20  node                                0x00000001010058a4 _ZZN4node6worker6Worker11StartThreadERKN2v820FunctionCallbackInfoINS2_5ValueEEEEN3$_38__invokeEPv + 56
21  libsystem_pthread.dylib             0x00000001a590606c _pthread_start + 148
22  libsystem_pthread.dylib             0x00000001a5900e2c thread_start + 8

System Info

System:
    OS: Linux 5.19 Ubuntu 22.10 22.10 (Kinetic Kudu)
    CPU: (8) x64 Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
    Memory: 21.66 GB / 31.07 GB
    Container: Yes
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 18.12.0 - ~/.nvm/versions/node/v18.12.0/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v18.12.0/bin/yarn
    npm: 8.19.2 - ~/.nvm/versions/node/v18.12.0/bin/npm
  npmPackages:
    vite: ^3.0.8 => 3.0.9 
    vitest: ^0.25.2 => 0.25.2 


### Used Package Manager

pnpm

### Validations

- [X] Follow our [Code of Conduct](https://github.com/vitest-dev/vitest/blob/main/CODE_OF_CONDUCT.md)
- [X] Read the [Contributing Guidelines](https://github.com/vitest-dev/vitest/blob/main/CONTRIBUTING.md).
- [X] Read the [docs](https://vitest.dev/guide/).
- [X] Check that there isn't [already an issue](https://github.com/vitest-dev/vitest/issues) that reports the same bug to avoid creating a duplicate.
- [X] Check that this is a concrete bug. For Q&A open a [GitHub Discussion](https://github.com/vitest-dev/vitest/discussions) or join our [Discord Chat Server](https://chat.vitest.dev).
- [X] The provided reproduction is a [minimal reproducible example](https://stackoverflow.com/help/minimal-reproducible-example) of the bug.
@pestaa
Copy link

pestaa commented Nov 20, 2022

Experiencing the same. Do you by any chance use TRPC?

This is the line it segfaults on:

  const caller = appRouter.createCaller(ctx);

like in https://github.com/trpc/examples-next-prisma-starter/blob/next/src/server/routers/post.test.ts#L10

I also have AWS SDKs and Prisma, so couldn't pinpoint it better yet.

@revmischa
Copy link
Author

I'm not using TRPC

@sheremet-va sheremet-va added help wanted Extra attention is needed upstream labels Nov 21, 2022
@revmischa
Copy link
Author

I do not get the error if I use <=0.24.3 but if I use any newer version it segfaults. Presumably the problem was introduced in 0.24.4

@revmischa
Copy link
Author

revmischa commented Dec 2, 2022

I have a reproduction: https://github.com/jetbridge/sst-prisma/tree/vitest-segfault - branch vitest-segfault

Instructions:

  1. be on linux
  2. clone repo and check out vitest-segfault
  3. npm i -g pnpm
  4. pnpm i
  5. pnpm run db:migrate:test
  6. pnpm exec vitest run backend/src/db/client.test.ts
    output:
 FAIL  backend/src/db/client.test.ts > Prisma client > Prisma client synchronous
AssertionError: expected 1 to deeply equal 2
 ❯ backend/src/db/client.test.ts:25:24
     23| 
     24|     const res = await prismaSync.$queryRaw<Array<{ one: number }>>`SELECT 1 as one`;
     25|     expect(res[0].one).toEqual(2);
       |                        ^
     26|   });
     27| });

  - Expected   "2"
  + Received   "1"

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

 Test Files  1 failed (1)
      Tests  1 failed | 1 passed (2)
   Start at  22:31:37
   Duration  2.01s (transform 557ms, setup 25ms, collect 921ms, tests 35ms)

 ERR_PNPM_RECURSIVE_EXEC_FIRST_FAIL  Command was killed with SIGSEGV (Segmentation fault): vitest run backend/src/db/client.test.ts

As mentioned above, it does not segfault for vitest <=0.24.3

As an aside, if you do downgrade vitest and run the test, it will fail, not segfault, but the exit code will be 0 even though the test failed (ref #2363)

@mjswensen
Copy link

I've got a reproduction as well (vitest version 0.26.2): themerdev/themer at commit 20e8bc0b5dfb4970228ba2ac6be5f0d1bba2f104

Instructions:

  1. Clone the repo — I've got a devcontainer.json set up so you can use Codespaces or VS Code container development extension if desired
  2. Check out commit 20e8bc0b5dfb4970228ba2ac6be5f0d1bba2f104
  3. (If you're using dev containers, you may need to rebuild the container at this point since the container definition is different than main branch)
  4. npm install --workspace packages/wallpaper-diamonds
  5. npm run test --workspace packages/wallpaper-diamonds -- run

Sometimes it will pass without error. Other times, you'll see something like this:

> @themerdev/wallpaper-diamonds@3.0.0 test
> vitest run


 RUN  v0.25.1 /workspaces/themer/cli/packages/wallpaper-diamonds

 ✓ lib/index.spec.mjs (2)

 Test Files  1 passed (1)
      Tests  2 passed (2)
   Start at  16:16:32
   Duration  750ms (transform 321ms, setup 0ms, collect 32ms, tests 123ms)

Segmentation fault
npm ERR! Lifecycle script `test` failed with error: 
npm ERR! Error: command failed 
npm ERR!   in workspace: @themerdev/wallpaper-diamonds@3.0.0 
npm ERR!   at location: /workspaces/themer/cli/packages/wallpaper-diamonds 

I've tried using the segfault-handler package to get a better stack trace, but haven't had any luck no matter where I place the handler—perhaps it's happening after the node process executing the test code has already completed or something.

@mrspartak
Copy link

mrspartak commented Dec 28, 2022

Added one more test suite yesterday and started to get segfault in the CI, locally everything works fine.
Vitest: 0.26.2
Local env: Mac m1, node 14.18.1
CI env: node 14.18.1
Immediately after showing that everything passed, I get an error. Sometimes on a retry is works fine, but very rare.
image

The last test suite I added was about TCP communication between the local server and mocked APP

@revmischa
Copy link
Author

revmischa commented Jan 11, 2023

Still getting a segfault after running all tests on linux, using vitest 0.27.1

Current executable set to '/usr/bin/sh' (x86_64).
(lldb) settings set -- target.run-args  "node_modules/.bin/vitest" "backend/src/db/client.test.ts"
...
Process 29844 resuming
Process 29844 stopped and restarted: thread 1 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 1 received signal: SIGCHLD

 DEV  v0.27.1 /home/cyber/dev/jbp

Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
Process 29844 stopped and restarted: thread 12 received signal: SIGCHLD
 ✓ backend/src/db/client.test.ts (2)

 Test Files  1 passed (1)
      Tests  2 passed (2)
   Start at  14:22:08
   Duration  5.50s (transform 327ms, setup 25ms, collect 4.32s, tests 30ms)


 PASS  Waiting for file changes...
       press h to show help, press q to quit
Process 29844 stopped
* thread #12, name = 'node', stop reason = signal SIGSEGV: invalid address (fault address: 0x7ff2df7cb620)
    frame #0: 0x00007ff2df7cb620
error: memory read failed for 0x7ff2df7cb600
(lldb) bt
* thread #12, name = 'node', stop reason = signal SIGSEGV: invalid address (fault address: 0x7ff2df7cb620)
  * frame #0: 0x00007ff2df7cb620
    frame #1: 0x00007ffff788cfb1 libc.so.6`__GI___nptl_deallocate_tsd at nptl_deallocate_tsd.c:73:29
    frame #2: 0x00007ffff788cf3a libc.so.6`__GI___nptl_deallocate_tsd at nptl_deallocate_tsd.c:22:1
    frame #3: 0x00007ffff789025e libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:453:3
    frame #4: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
(lldb) bt all
warning: This version of LLDB has no plugin for the language "assembler". Inspection of frame variables will be limited.
  thread #1, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x00007fffed409990, expected=29908, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff7891ee3 libc.so.6`__pthread_clockjoin_ex(threadid=140737173821120, thread_return=0x0000000000000000, clockid=0, abstime=0x0000000000000000, block=true) at pthread_join_common.c:102:14
    frame #4: 0x00007ffff7891d83 libc.so.6`___pthread_join(threadid=<unavailable>, thread_return=<unavailable>) at pthread_join.c:24:10 [artificial]
    frame #5: 0x000000000165b1de node`uv_thread_join(tid=<unavailable>) at thread.c:273:10
    frame #6: 0x0000000000c30b6e node`node::worker::Worker::JoinThread() + 46
    frame #7: 0x0000000000aefa38 node`node::Environment::RunAndClearNativeImmediates(bool) + 1080
    frame #8: 0x0000000000aefd1c node`node::Environment::InitializeLibuv()::'lambda'(uv_async_s*)::_FUN(uv_async_s*) + 60
    frame #9: 0x000000000164ca06 node`uv__async_io(loop=0x0000000005255da0, w=<unavailable>, events=<unavailable>) at async.c:163:5
    frame #10: 0x000000000165f144 node`uv__io_poll(loop=0x0000000004e95da0, timeout=<unavailable>) at epoll.c:374:11
    frame #11: 0x000000000164d358 node`uv_run(loop=0x0000000004e95da0, mode=UV_RUN_DEFAULT) at core.c:389:5
    frame #12: 0x0000000000aad9e5 node`node::SpinEventLoop(node::Environment*) + 325
    frame #13: 0x0000000000bb20fc node`node::NodeMainInstance::Run() + 236
    frame #14: 0x0000000000b2abd2 node`node::LoadSnapshotDataAndRun(node::SnapshotData const**, node::InitializationResult const*) + 194
    frame #15: 0x0000000000b2e6bf node`node::Start(int, char**) + 735
    frame #16: 0x00007ffff7823510 libc.so.6`__libc_start_call_main(main=(node`main), argc=3, argv=0x00007fffffffdd08) at libc_start_call_main.h:58:16
    frame #17: 0x00007ffff78235c9 libc.so.6`__libc_start_main_impl(main=(node`main), argc=3, argv=0x0000000004e95da0, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffdcf8) at libc-start.c:381:3
    frame #18: 0x0000000000aaa86e node`_start + 46
  thread #2, name = 'node'
    frame #0: 0x00007ffff791eb66 libc.so.6`epoll_wait(epfd=9, events=0x00007ffff77fbd60, maxevents=1024, timeout=-1) at epoll_wait.c:30:10
    frame #1: 0x000000000165f4e3 node`uv__io_poll(loop=0x00000000050189e8, timeout=<unavailable>) at epoll.c:236:14
    frame #2: 0x000000000164d358 node`uv_run(loop=0x00000000050189e8, mode=UV_RUN_DEFAULT) at core.c:389:5
    frame #3: 0x0000000000bdc39b node`node::WorkerThreadsTaskRunner::DelayedTaskScheduler::Start()::'lambda'(void*)::_FUN(void*) + 123
    frame #4: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #5: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #3, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000005018824, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x00000000050187f8, mutex=0x00000000050187d0) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=<unavailable>, mutex=<unavailable>) at thread.c:780:7
    frame #6: 0x0000000000bd728b node`node::(anonymous namespace)::PlatformWorkerThread(void*) + 267
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #4, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000005018820, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x00000000050187f8, mutex=0x00000000050187d0) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=<unavailable>, mutex=<unavailable>) at thread.c:780:7
    frame #6: 0x0000000000bd728b node`node::(anonymous namespace)::PlatformWorkerThread(void*) + 267
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #5, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000005018824, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x00000000050187f8, mutex=0x00000000050187d0) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=<unavailable>, mutex=<unavailable>) at thread.c:780:7
    frame #6: 0x0000000000bd728b node`node::(anonymous namespace)::PlatformWorkerThread(void*) + 267
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #6, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000005018824, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x00000000050187f8, mutex=0x00000000050187d0) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=<unavailable>, mutex=<unavailable>) at thread.c:780:7
    frame #6: 0x0000000000bd728b node`node::(anonymous namespace)::PlatformWorkerThread(void*) + 267
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #7, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000004e8c340, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff7898440 libc.so.6`__new_sem_wait_slow64(sem=0x0000000004e8c340, abstime=<unavailable>, clockid=<unavailable>) at sem_waitcommon.c:183:10
    frame #4: 0x000000000165b702 node`uv_sem_wait at thread.c:626:9
    frame #5: 0x000000000165b6f0 node`uv_sem_wait(sem=0x0000000004e8c340) at thread.c:682:5
    frame #6: 0x0000000000c72015 node`node::inspector::(anonymous namespace)::StartIoThreadMain(void*) + 53
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #8, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000004e95d6c, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at thread.c:780:7
    frame #6: 0x0000000001647bc4 node`worker(arg=0x0000000000000000) at threadpool.c:76:7
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #9, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000004e95d68, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at thread.c:780:7
    frame #6: 0x0000000001647bc4 node`worker(arg=0x0000000000000000) at threadpool.c:76:7
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #10, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000004e95d68, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at thread.c:780:7
    frame #6: 0x0000000001647bc4 node`worker(arg=0x0000000000000000) at threadpool.c:76:7
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #11, name = 'node'
    frame #0: 0x00007ffff788c9f6 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:57:12
    frame #1: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64 at futex-internal.c:87:9
    frame #2: 0x00007ffff788c9c9 libc.so.6`__GI___futex_abstimed_wait_cancelable64(futex_word=0x0000000004e95d68, expected=0, clockid=<unavailable>, abstime=0x0000000000000000, private=<unavailable>) at futex-internal.c:139:10
    frame #3: 0x00007ffff788f338 libc.so.6`___pthread_cond_wait at pthread_cond_wait.c:503:10
    frame #4: 0x00007ffff788f260 libc.so.6`___pthread_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at pthread_cond_wait.c:627:10
    frame #5: 0x000000000165b8c9 node`uv_cond_wait(cond=0x0000000004e95d40, mutex=0x0000000004e95d00) at thread.c:780:7
    frame #6: 0x0000000001647bc4 node`worker(arg=0x0000000000000000) at threadpool.c:76:7
    frame #7: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #8: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
* thread #12, name = 'node', stop reason = signal SIGSEGV: invalid address (fault address: 0x7ff2df7cb620)
  * frame #0: 0x00007ff2df7cb620
    frame #1: 0x00007ffff788cfb1 libc.so.6`__GI___nptl_deallocate_tsd at nptl_deallocate_tsd.c:73:29
    frame #2: 0x00007ffff788cf3a libc.so.6`__GI___nptl_deallocate_tsd at nptl_deallocate_tsd.c:22:1
    frame #3: 0x00007ffff789025e libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:453:3
    frame #4: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #13, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff791eb66 libc.so.6`epoll_wait(epfd=29, events=0x00007ff84c41ce10, maxevents=1024, timeout=-1) at epoll_wait.c:30:10
    frame #1: 0x00007ff347715cc0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25236 + 536
    frame #2: 0x00007ff347715977 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25234 + 48
    frame #3: 0x00007ff3477158e7 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25233 + 404
    frame #4: 0x00007ff347714541 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 590
    frame #5: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #6: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #7: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #8: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #9: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #10: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #11: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #14, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #15, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #16, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #17, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #18, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #19, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #20, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #21, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #22, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #23, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #24, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #25, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #26, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #27, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #28, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #29, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #30, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #31, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #32, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #33, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #34, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #35, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #36, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9365 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 1006
    frame #2: 0x00007ff3477144ad libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25197 + 442
    frame #3: 0x00007ff34771ddac libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25447 + 2709
    frame #4: 0x00007ff347713918 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25169 + 18
    frame #5: 0x00007ff347713253 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 310
    frame #6: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #37, name = 'opentelemetry-e'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3476d1390 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24194 + 208
    frame #2: 0x00007ff3476d2899 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24223 + 73
    frame #3: 0x00007ff3472b2bde libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol11992 + 182
    frame #4: 0x00007ff3472b4be0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12046 + 146
    frame #5: 0x00007ff3472afb03 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol11924 + 1856
    frame #6: 0x00007ff3472b9c66 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12183 + 151
    frame #7: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #8: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #9: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #38, name = 'futures-timer'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3476d1390 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24194 + 208
    frame #2: 0x00007ff3476d2899 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24223 + 73
    frame #3: 0x00007ff34701c1d8 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol5707 + 1604
    frame #4: 0x00007ff34701dd63 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol5783 + 177
    frame #5: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #6: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #7: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #39, name = 'tokio-runtime-w'
    frame #0: 0x00007ffff7916c4d libc.so.6`syscall at syscall.S:38
    frame #1: 0x00007ff3472c9239 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol12400 + 706
    frame #2: 0x00007ff3477132f5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25162 + 472
    frame #3: 0x00007ff347716ae0 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol25265 + 142
    frame #4: 0x00007ff3476fcbd5 libquery_engine-debian-openssl-3.0.x.so.node`___lldb_unnamed_symbol24550 + 21
    frame #5: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #6: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81
  thread #40, name = 'node'
    frame #0: 0x00000000011bcfe7 node`v8::internal::StringTable::Data::Resize(v8::internal::PtrComprCageBase, std::unique_ptr<v8::internal::StringTable::Data, std::default_delete<v8::internal::StringTable::Data>>, int) (.constprop.115) + 151
    frame #1: 0x00000000011bd18c node`v8::internal::StringTable::EnsureCapacity(v8::internal::PtrComprCageBase, int) (.constprop.112) + 108
    frame #2: 0x00000000011bf134 node`v8::internal::Handle<v8::internal::String> v8::internal::StringTable::LookupKey<v8::internal::StringTableInsertionKey, v8::internal::Isolate>(v8::internal::Isolate*, v8::internal::StringTableInsertionKey*) + 228
    frame #3: 0x0000000001316312 node`v8::internal::SharedHeapDeserializer::DeserializeStringTable() + 162
    frame #4: 0x0000000001316384 node`v8::internal::SharedHeapDeserializer::DeserializeIntoIsolate() + 84
    frame #5: 0x0000000000eb0288 node`v8::internal::Isolate::Init(v8::internal::SnapshotData*, v8::internal::SnapshotData*, v8::internal::SnapshotData*, bool) + 2952
    frame #6: 0x00000000013181ee node`v8::internal::Snapshot::Initialize(v8::internal::Isolate*) (.part.48) + 526
    frame #7: 0x0000000000d6c52a node`v8::Isolate::Initialize(v8::Isolate*, v8::Isolate::CreateParams const&) + 362
    frame #8: 0x0000000000c352a6 node`node::worker::Worker::Run() + 1030
    frame #9: 0x0000000000c363c8 node`node::worker::Worker::StartThread(v8::FunctionCallbackInfo<v8::Value> const&)::'lambda'(void*)::_FUN(void*) + 56
    frame #10: 0x00007ffff7890402 libc.so.6`start_thread(arg=<unavailable>) at pthread_create.c:442:8
    frame #11: 0x00007ffff791f590 libc.so.6`__clone3 at clone3.S:81

@zackheil
Copy link

zackheil commented Feb 17, 2023

I too am getting segfaults when running a test file that involves the aws-sdk and only on Linux with node 16 and 18. Interestingly, this issue only manifested for me recently in a GitHub action run (with vitest@0.28.4 and @aws-sdk/*@3.264.0) and I have been running vitest tests with my packages just fine as far back as v0.15.x and @aws-sdk/*@3.95.0.

Edit: Looks like they were failing silently like others described.

Looks like the tests run just fine, I get all my tests passing and I put an afterAll call to log when it is hit, but it segfaults right before exit somewhere:

image

I also tried using the segfault handler several times at the top of the file, but nothing triggers it to run, so I don't have (nor do I know how) to gather any useful logs for this issue.

All of my other tests that don't involve aws-sdk pass without crashing vitest.

@nnyegaard
Copy link

I just upgraded to 0.29.2, and I don't experience the problem anymore

@sheremet-va
Copy link
Member

sheremet-va commented Mar 1, 2023

--no-threads now uses child_process to spawn tests. If the problem before was with workers, it should be fixed in 0.29.0.

@timbrinded
Copy link

timbrinded commented Mar 8, 2023

Still happening for me on 0.29.2, Ubuntu running node18.

I get this when i run some heavy test packs (with their own idiosyncratic setups) in quick succession to one another e.g.

vitest heavyTest1 ; vitest heavyTest2; vitest heavyTest3

This is with threads enabled. SingleThread mode is a bit of a no-go for me due to the requirement for separation.


Stacktrace:

=========== Caught a Segmentation Fault [pid=1275871] ===========
-----[ Native Stacktraces ]-----
[pc=0x00007fad0abc3aa8, sp=0x00007fa563ff33e0] in segfault_handler(int)+0x58
[pc=0x00007fad0a442520, sp=0x00007fa563ff3400] in __sigaction+0x50
[pc=0x0000000000000000, sp=0x00007fa563ff4108] in +0x50
[pc=0x00007fad00c0b73c, sp=0x00007fa563ff4118] in Secp256k1Addon::Init(Napi::Env)+0xe9c
[pc=0x00007fad00c05d56, sp=0x00007fa563ff4860] in Init(Napi::Env, Napi::Object)+0x16
[pc=0x00007fad00c05e3e, sp=0x00007fa563ff4890] in __napi_Init(napi_env__*, napi_value__*)+0x5e
[pc=0x0000000000b3dfb4, sp=0x00007fa563ff48c0] in napi_module_register_by_symbol(v8::Local<v8::Object>, v8::Local<v8::Value>, v8::Local<v8::Context>, napi_value__* (*)(napi_env__*, napi_value__*))+0x3c4
[pc=0x0000000000b44382, sp=0x00007fa563ff4ed0] in std::_Function_handler<bool (node::binding::DLib*), node::binding::DLOpen(v8::FunctionCallbackInfo<v8::Value> const&)::{lambda(node::binding::DLib*)#1}>::_M_invoke(std::_Any_data const&, node::binding::DLib*&&)+0x132
[pc=0x0000000000b003e3, sp=0x00007fa563ff4f30] in node::Environment::TryLoadAddon(char const*, int, std::function<bool (node::binding::DLib*)> const&)+0x73
[pc=0x0000000000b43944, sp=0x00007fa563ff4f80] in node::binding::DLOpen(v8::FunctionCallbackInfo<v8::Value> const&)+0x254
[pc=0x0000000000dc0f90, sp=0x00007fa563ff5430] in v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments)+0x380
[pc=0x0000000000dc24cf, sp=0x00007fa563ff5570] in v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*)+0xaf
[pc=0x00000000017002f9, sp=0x00007fa563ff55e0] in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit+0x39
[pc=0x0000000001684250, sp=0x00007fa563ff55f0] in Builtins_InterpreterEntryTrampoline+0xd0
[pc=0x000000000f5b33d2, sp=0x00007fa563ff5600] in +0xd0
[pc=0x000000000f5b2888, sp=0x00007fa563ff5610] in +0xd0
[pc=0x000000000f5aef8d, sp=0x00007fa563ff5620] in +0xd0
[pc=0x000000000f5a9bcf, sp=0x00007fa563ff5630] in +0xd0
[pc=0x0000000001684250, sp=0x00007fa563ff5640] in Builtins_InterpreterEntryTrampoline+0xd0
[pc=0x0000000001684250, sp=0x00007fa563ff5650] in Builtins_InterpreterEntryTrampoline+0xd0
[pc=0x000000000f5b526c, sp=0x00007fa563ff5660] in +0xd0
[pc=0x000000000f5c6455, sp=0x00007fa563ff5670] in +0xd0

@mrspartak
Copy link

Our issue with segfault was resolved, when we upgraded node from 14 to 18

@mclean25
Copy link

mclean25 commented Mar 10, 2023

I was having the same issue, but was able to get around it by mocking out any modules that referenced AWS SDK modules, for example:

beforeAll(() => {
  // Mocking this out is required to prevent segfaults which are likely
  // coming from the AWS SDK.
  vi.mock("../path/to/aws-module", () => {
    return {};
  });
});

I guessed the AWS SDKs were causing the issue because it was segfaulting after that test suite (which was importing AWS modules indirectly) was the last logged when running in tests in GH actions on ubuntu-latest

@RobbeCl
Copy link
Contributor

RobbeCl commented Mar 16, 2023

Same issue here. Node 16/18 and also using AWS sdk v3

@revmischa
Copy link
Author

Vitest 0.29.x with node 18 is working okay for me now

@RobbeCl
Copy link
Contributor

RobbeCl commented Mar 30, 2023

Vitest 0.29.x still gives me segfaults with aws-sdk. When enabling --no-threads, they are gone.

@davidrhoderick
Copy link

Having the same problem with vitest v0.31.0 + @aws-sdk/client-lambda v3.328.0. As per @RobbeCl's suggestion, using --no-threads gets rid the errors.

@davidrhoderick
Copy link

Having the same problem with vitest v0.31.0 + @aws-sdk/client-lambda v3.328.0. As per @RobbeCl's suggestion, using --no-threads gets rid the errors.

As per @mclean25's suggestion, I mocked the @aws-sdk/client-lambda to get this working all the way because --no-threads made one of my sub-repos when running all the mono-repo tests together.

@adrianisk
Copy link

We're also hitting this issue which happened when we introduced an @aws-sdk dependency. I happened to stumble upon awslabs/aws-crt-nodejs#452 while looking into something else, might be related?

@github-actions github-actions bot locked and limited conversation to collaborators Jun 14, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
help wanted Extra attention is needed upstream
Projects
None yet
Development

No branches or pull requests