Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reported Memory Leak #235

Closed
mgifford opened this issue Aug 19, 2024 · 3 comments
Closed

Reported Memory Leak #235

mgifford opened this issue Aug 19, 2024 · 3 comments

Comments

@mgifford
Copy link

mgifford commented Aug 19, 2024

Describe the bug

I am getting this error with unlighthouse/0.13.2 darwin-arm64 node-v20.16.0:

✔ Completed inspectHtmlTask for /webinar-vbid-model-cy-2025-application-process-office-hours. (Time Taken: 9.2s 196.08 KB 51% complete)           Unlighthouse 2:26:06 p.m.
✔ Completed inspectHtmlTask for /sf5510-authorization-agreement-preauthorized-payments. (Time Taken: 23.1s 214.02 KB 52% complete)                Unlighthouse 2:26:08 p.m.
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 unpipe listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene
(Use `node --trace-warnings ...` to show where the warning was created)
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListener
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListener
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 unpipe listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene 
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListener
(node:7707) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene
/Users/mgifford/.npm/_npx/b17dc4389256a785/node_modules/puppeteer-cluster/dist/util.js:69
                throw new Error(`Timeout hit: ${millis}`);
                      ^

Error: Timeout hit: 300000
    at /Users/mgifford/.npm/_npx/b17dc4389256a785/node_modules/puppeteer-cluster/dist/util.js:69:23
    at Generator.next (<anonymous>)
    at fulfilled (/Users/mgifford/.npm/_npx/b17dc4389256a785/node_modules/puppeteer-cluster/dist/util.js:5:58)
    at runNextTicks (node:internal/process/task_queues:60:5)
    at listOnTimeout (node:internal/timers:545:9)
    at process.processTimers (node:internal/timers:519:7)

Node.js v20.16.0

I am unclear why this would be an issue.

Sites scan normally, then the amount of memory consumed on the device skyrockets until the script crashes.

I don't know if it is a puppeteer or unlighthouse.dev problem. I have done nothing custom to either. Just a config.

Also, in the instance where it doesn't die on a different site:

✔ Completed inspectHtmlTask for /publications/. (Time Taken: 18.9s 174.3 KB 50% complete)                                                         Unlighthouse 2:32:19 p.m.
✔ Completed inspectHtmlTask for /od/. (Time Taken: 22.9s 78.14 KB 53% complete)                                                                   Unlighthouse 2:32:19 p.m.
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 unpipe listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListen
(Use `node --trace-warnings ...` to show where the warning was created)
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListen
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 unpipe listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListen
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListene
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListenr
(node:15947) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added to [WriteStream]. MaxListeners is 10. Use emitter.setMaxListen
✔ Completed runLighthouseTask for /oirm/. (Time Taken: 28.3s Score: 0.76 Samples: 1 56% complete)                                                 Unlighthouse 2:32:46 p.m.
✔ Completed runLighthouseTask for /staff/. (Time Taken: 29.2s Score: 0.87 Samples: 1 61% complete)                                                Unlighthouse 2:32:47 p.m.

Reproduction

Yes...

System / Nuxt Info

System:
    OS: macOS 14.5
    CPU: (10) arm64 Apple M1 Max
    Memory: 169.72 MB / 32.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.16.0 - ~/.nvm/versions/node/v20.16.0/bin/node
    Yarn: 1.22.22 - /opt/homebrew/bin/yarn
    npm: 10.8.1 - ~/.nvm/versions/node/v20.16.0/bin/npm
    pnpm: 9.5.0 - /opt/homebrew/bin/pnpm
  Browsers:
    Brave Browser: 118.1.59.117
    Chrome: 127.0.6533.120
    Chrome Canary: 129.0.6666.1
    Safari: 17.5
    
    

npx unlighthouse-ci --site https://www.whitehouse.gov --config unlighthouse.config-wh.ts
@KitLykos
Copy link

I have been working on this for a little bit now and while other jobs complete, we have this one still running consistently into this memory issue. We're running a github action on Ubuntu latest with node v.22. It always fails at the same point and reports as out of memory. I added some tracing and am attaching the logs from the most recent run. The code snippet below is the same for all three jobs, working in two and not in the one mentioned above.
logs_28210731332.zip

steps:
- uses: actions/checkout@v4
env:
NODE_OPTIONS: --max_old_space_size=8192
- run: npm init -y
- run: npm install
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
- run: npm install -g @unlighthouse/cli puppeteer
- run: strace npx unlighthouse-ci

@Hahlh
Copy link

Hahlh commented Oct 5, 2024

Still happening for me as well:

/home/user/.cache/pnpm/dlx/nyebsawq2skhwbmv3wuy7tb4uu/1925b7eae8d-33c875/node_modules/.pnpm/puppeteer-cluster@0.24.0_puppeteer@23.5.0/node_modules/puppeteer-cluster/dist/util.js:69
                throw new Error(`Timeout hit: ${millis}`);
                      ^

Error: Timeout hit: 300000
    at /home/user/.cache/pnpm/dlx/nyebsawq2skhwbmv3wuy7tb4uu/1925b7eae8d-33c875/node_modules/.pnpm/puppeteer-cluster@0.24.0_puppeteer@23.5.0/node_modules/puppeteer-cluster/dist/util.js:69:23
    at Generator.next (<anonymous>)
    at fulfilled (/home/user/.cache/pnpm/dlx/nyebsawq2skhwbmv3wuy7tb4uu/1925b7eae8d-33c875/node_modules/.pnpm/puppeteer-cluster@0.24.0_puppeteer@23.5.0/node_modules/puppeteer-cluster/dist/util.js:5:58)
    at runNextTicks (node:internal/process/task_queues:60:5)
    at process.processTimers (node:internal/timers:511:9)

Node.js v20.12.2

@luketowell
Copy link

Hi, was there ever a resolution to this issue as it is happening repeatedly with one of our sites?

Node.js: v20.18.0
unlighthouse-cli: v0.14.0

Seems to get to 99% and then throws the following error:

<path>/node_modules/puppeteer-cluster/dist/util.js:69
                throw new Error(`Timeout hit: ${millis}`);
                      ^

Error: Timeout hit: 300000
    at <path>/node_modules/puppeteer-cluster/dist/util.js:69:23
    at Generator.next (<anonymous>)
    at fulfilled (<path>/node_modules/puppeteer-cluster/dist/util.js:5:58)

Node.js v20.18.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants