Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse connection with lazyConnect on worker #281

Open
nmrgt opened this issue Oct 7, 2020 · 2 comments
Open

Reuse connection with lazyConnect on worker #281

nmrgt opened this issue Oct 7, 2020 · 2 comments

Comments

@nmrgt
Copy link

nmrgt commented Oct 7, 2020

If I use a shared redis connection with lazy connect option in a worker, like this (pseudo code)

const client = new Redis(connectionString, { lazyConnect: true });
await client.connect();
const worker = new Worker('myworker', () => {}, { connection: client });

Worker dont process any job exept those already in queue when starting.
blockingConnection.client promise is never resolved which block every new job processing.

In Worker constructor :

    this.blockingConnection = new RedisConnection(
      isRedisInstance(opts.connection)
        ? (<Redis>opts.connection).duplicate()
        : opts.connection,
    );

I think that the bug come from redis duplicating (clone redis object with options - like lazyconnect) but don't initialize connection. Maybe we could fix this by :

  • adding documentation (don't pass lazyconnect option) ;
  • removing duplicate() method call, just reuse connection ;
  • test if lazyconnect is on and handle connection in init method of RedisConnection class if so

The quick fix for me was to remove lazyconnect :)

Thanks for your great work !

@manast
Copy link
Contributor

manast commented Oct 7, 2020

Could be a bug in ioredis when duplicating a connection that has the lazyConnect: true option set. Best would be to just warn about it I think.

@toverux
Copy link

toverux commented Aug 28, 2023

I am also using lazyConnect and I cannot observe this behavior, it's probably been fixed upstream?


I stumbled on this issue as I was trying to understand why BullMQ duplicates connections. Probably a design choice, but it might cause certain problems.

The use case is repeated creation and destruction of QueueEvents objects.
My app leaked connections as I passed customized, ready-to-use client instances to Bull, without realising that Bull duplicated connections. I closed my own ioredis connections instead of using Bull APIs, so 1) I leaked a ton of connections (fixed by using BullMQ's close() methods) and 2) I connected two times to the server (fixed with lazyConnect).

I would prefer that Bull did not duplicated connections implictly, this goes against the principle of least astonishment, and I prefer to manage my ioredis connections. At least, an option to control this would be nice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants