Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

There are X handle(s) keeping the process running; FILEHANDLE #4415

Closed
6 tasks done
gajus opened this issue Oct 31, 2023 · 14 comments
Closed
6 tasks done

There are X handle(s) keeping the process running; FILEHANDLE #4415

gajus opened this issue Oct 31, 2023 · 14 comments

Comments

@gajus
Copy link

gajus commented Oct 31, 2023

Describe the bug

This might be not a bug, but where does one even start debugging this?

contra-api:test:vitest: There are 6601 handle(s) keeping the process running
contra-api:test:vitest: 
contra-api:test:vitest: # FILEHANDLE
contra-api:test:vitest: node:internal/async_hooks:202
contra-api:test:vitest: 
contra-api:test:vitest: # FILEHANDLE
contra-api:test:vitest: node:internal/async_hooks:202
contra-api:test:vitest: 
contra-api:test:vitest: # Tinypool
contra-api:test:vitest: node:internal/async_hooks:202                                                                                                                         
contra-api:test:vitest: node:internal/async_hooks:505                                                                                                                         
contra-api:test:vitest: file:///home/github/actions-runner/_work/gaia/gaia/node_modules/.pnpm/tinypool@0.7.0/node_modules/tinypool/dist/esm/index.js:36                       
contra-api:test:vitest: file:///home/github/actions-runner/_work/gaia/gaia/node_modules/.pnpm/tinypool@0.7.0/node_modules/tinypool/dist/esm/index.js:57                       
contra-api:test:vitest: file:///home/github/actions-runner/_work/gaia/gaia/node_modules/.pnpm/tinypool@0.7.0/node_modules/tinypool/dist/esm/index.js:799                      
contra-api:test:vitest: file:///home/github/actions-runner/_work/gaia/gaia/node_modules/.pnpm/vitest@0.34.5_jsdom@21.1.1/node_modules/vitest/dist/vendor-node.81dd929c.js:7314
contra-api:test:vitest: file:///home/github/actions-runner/_work/gaia/gaia/node_modules/.pnpm/vitest@0.34.5_jsdom@21.1.1/node_modules/vitest/dist/vendor-node.81dd929c.js:7676
contra-api:test:vitest: 
contra-api:test:vitest: # FILEHANDLE
contra-api:test:vitest: node:internal/async_hooks:202

Reproduction

Running our entire test suite.

System Info

System:
    OS: macOS 13.2.1
    CPU: (10) arm64 Apple M1 Max
    Memory: 955.88 MB / 64.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 21.1.0 - ~/.nvm/versions/node/v21.1.0/bin/node
    Yarn: 1.22.19 - ~/Library/pnpm/yarn
    npm: 10.2.0 - ~/.nvm/versions/node/v21.1.0/bin/npm
    pnpm: 8.9.0 - ~/.nvm/versions/node/v21.1.0/bin/pnpm
  Browsers:
    Brave Browser: 116.1.57.64
    Chrome: 118.0.5993.117
    Edge: 114.0.1823.43
    Safari: 16.3
  npmPackages:
    vitest: ^0.34.5 => 0.34.5

Used Package Manager

pnpm

Validations

@AriPerkkio
Copy link
Member

AriPerkkio commented Oct 31, 2023

Reproduction

Running our entire test suite.

Happy to look into this but we need a reproduction case.

Copy link

Hello @gajus. Please provide a minimal reproduction using a GitHub repository or StackBlitz. Issues marked with need reproduction will be closed if they have no activity within 3 days.

@gajus
Copy link
Author

gajus commented Oct 31, 2023

Reproduction

Running our entire test suite.

Happy to look into this but we need a reproduction case.

I wouldn't even know where to start. This happens in our large repository with thousands of tests, and doesn't happen every time.

@AriPerkkio
Copy link
Member

Seeing Tinypool in the listed resources indicates that some test file is preventing it from shutting down. Without hanging-process reporter, do you see errors indicating which file that is? In #3031 there's an example of such error logging.

What's the reason why you are using hanging-process reporter in the first place? What errors do you see without it?

@gajus
Copy link
Author

gajus commented Oct 31, 2023

What's the reason why you are using hanging-process reporter in the first place? What errors do you see without it?

Without it it just occasionally times out. The test suite never completes.

It is not a single file. Keeps popping up on random files, e.g.

contra-api:test:vitest: close timed out after 10000ms
contra-api:test:vitest: Failed to terminate worker while running /home/github/actions-runner/_work/gaia/gaia/apps/contra-api/testV2/integration/contraMagic/generateContraMagicOneLinerSuggestions.test.ts.
contra-api:test:vitest: Tests closed successfully but something prevents Vite server from exiting
contra-api:test:vitest: close timed out after 10000ms
contra-api:test:vitest: Failed to terminate worker while running /home/github/actions-runner/_work/gaia/gaia/apps/contra-api/test/contra-api/integration/queries/productizedServiceById.test.ts.
contra-api:test:vitest: Tests closed successfully but something prevents Vite server from exiting

I am trying to replicate this issue locally.

I installed why-is-node-running and running this test suite on repeat:

while [ true ]; do vitest run queries/productizedServiceById.test.ts; done

But so far nothing.

@gajus
Copy link
Author

gajus commented Oct 31, 2023

Is it at all possible that this is somehow hardware/OS specific?

These tests have ran now at least 50 times on my machine without a single hitch.

On CI I keep seeing them fail every other build.

Will try to create a VM that replicates the CI env.

@AriPerkkio
Copy link
Member

AriPerkkio commented Oct 31, 2023

Without seeing the test cases, dependencies and Vitest configuration it's impossible to point out the root cause. But here's some things to try at least:

@gajus
Copy link
Author

gajus commented Nov 3, 2023

Starting to suspect that we are hitting this nodejs/undici#2026

@AriPerkkio
Copy link
Member

You could try if using node-fetch fixes the issue. Something like this should be enough: https://github.com/dfinity/feedback/pull/151/files#diff-84f6be72ed45f1bf4cdbd441d0bd824e38270a7599bd4eac2981b05eb7ad9d29R8-R14

@gajus
Copy link
Author

gajus commented Nov 3, 2023

ah, literally just did this a second ago.

@mcollina
Copy link

mcollina commented Nov 3, 2023

Note that you could be hitting nodejs/node#49344.

@gajus
Copy link
Author

gajus commented Nov 3, 2023

Note that you could be hitting nodejs/node#49344.

This would be only an issue if coverage is enabled, right?

We have coverage explicitly disabled in our tests.

Our config, for reference:

export default defineConfig({
  resolve: {
    alias: {
      '#': path.resolve(__dirname, './test'),
      $: path.resolve(__dirname, './testV2'),
      '@': path.resolve(__dirname, './src'),
    },
  },
  test: {
    coverage: {
      enabled: false,
    },
    environment: 'node',
    // Note that we are intentionally using *.test.ts suffix because
    // *.test.{ts,tsx} suffix is used by ESLint to identify vitest files.
    // Do not add non-*.test.{ts,tsx} files to the list below.
    include: ['./{src,test,testV2}/**/*.test.ts'],
    maxThreads: isCi ? Math.ceil(os.cpus().length * 0.8) : 4,
    minThreads: 1,
    mockReset: true,
    outputFile: './src/__generated__/vitest/results.xml',
    reporters: ['verbose'],
    setupFiles: path.resolve(__dirname, './test/vitest.setup.ts'),
    testTimeout: 60_000,
  },
});

@mcollina
Copy link

mcollina commented Nov 3, 2023

Something more to try, set UV_USE_IO_URING=0.

@AriPerkkio
Copy link
Member

AriPerkkio commented Nov 4, 2023

Note that you could be hitting nodejs/node#49344.

Vitest moved from using NODE_V8_COVERAGE to use node:inspect directly a while ago. It was causing test hangs as described in the linked node issue.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Nov 8, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Nov 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants