Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EventEmitter memory leak #757

Closed
amsibamsi opened this issue Sep 23, 2019 · 5 comments
Closed

EventEmitter memory leak #757

amsibamsi opened this issue Sep 23, 2019 · 5 comments
Assignees
Labels
api: pubsub Issues related to the googleapis/nodejs-pubsub API. external This issue is blocked on a bug with the actual product. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. status: duplicate Duplicate. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.

Comments

@amsibamsi
Copy link

Environment details

  • OS: Debian 9.11
  • Node.js version: 10.16.3
  • npm version: 6.9.0
  • @google-cloud/pubsub version: 1.0.0

Steps to reproduce

Run the following sample program with GCP project myproject and Pub/Sub topic mytopic set up and observe warnings like:

(node:22056) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added. Use emitter.setMaxListeners() to increase limit

const projectId = 'myproject';
const topicName = 'mytopic';
const {PubSub} = require('@google-cloud/pubsub');
const pubsub = new PubSub({projectId});
const topic = pubsub.topic(`projects/${projectId}/topics/${topicName}`);

async function sleep(ms) {
       return new Promise(resolve => {
               setTimeout(resolve, ms);
       });
}

function publish() {
       const data = Buffer.from(JSON.stringify({date: Date.now()}));
       topic.publish(data, function(err, mId) {
               console.log(err, mId);
       });
}

async function main() {
       for (let i = 0; i < 1000; i++) {
               await publish();
               await sleep(20);
       }
}

main();

Watching this program run for a while it indeed grows to 300 MB resident memory quickly. Is there indeed a memory leak, and if yes, how can it be fixed?

Also tested this with older pubsub versions like 0.30.0, 0.29.0, etc. and newest release of Nodejs 12, same issue. Is this indeed a bug in the library or related to my code?

@callmehiphop
Copy link
Contributor

👋 @amsibamsi we've been seeing this issue pop up across several of our Node.js clients, I believe this is a grpc issue (grpc/grpc-node#1027) which may or may not be resolved already.

Can you check to see if @grpc/grpc-js@0.5.4 is installed? If it isn't, you may need to delete any lock files you might have and/or re-install your dependencies.

@callmehiphop callmehiphop added external This issue is blocked on a bug with the actual product. priority: p2 Moderately-important priority. Fix may not be included in next release. status: duplicate Duplicate. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns. labels Sep 23, 2019
@kamalaboulhosn kamalaboulhosn added priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. and removed priority: p2 Moderately-important priority. Fix may not be included in next release. labels Sep 23, 2019
@amsibamsi
Copy link
Author

Thanks, this indeed fixed the problem.

@callmehiphop
Copy link
Contributor

@amsibamsi awesome, thanks for letting us know!

@ctavan
Copy link
Contributor

ctavan commented Sep 24, 2019

@callmehiphop wouldn't it be appropriate to upgrade the dependency in package.json to "@grpc/grpc-js": "^0.5.4", then?

@callmehiphop
Copy link
Contributor

@ctavan yep, absolutely! We'll need to make that update across the board

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the googleapis/nodejs-pubsub API. external This issue is blocked on a bug with the actual product. priority: p1 Important issue which blocks shipping the next release. Will be fixed prior to next release. status: duplicate Duplicate. type: bug Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Projects
None yet
Development

No branches or pull requests

4 participants