Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] asynq can not follow the right redis master when using sentinel mode. #959

Open
hoorayman opened this issue Nov 7, 2024 · 1 comment
Assignees
Labels
bug Something isn't working investigate

Comments

@hoorayman
Copy link

Describe the bug
asynq can not follow the right redis master.

Environment:

  • OS: [Linux]
  • asynq package version [e.g. v0.25.0]
  • Redis/Valkey version

To Reproduce
use helm to install a redis cluster, with sentinel be set to true.
configure the asynq using sentinel:
image

Expected behavior
asynq can follow the right master and running normally.

Screenshots
go-redis can find the right master and change:
image
But asynq report error:
asynq: pid=1 2024/11/07 08:23:58.539202 ERROR: Dequeue error: UNKNOWN: redis eval error: READONLY You can't write against a read only replica. script: a8153319360adc71cdc370107f8ab9786f204765, on @user_script:3.
asynq: pid=1 2024/11/07 08:23:58.557608 ERROR: Failed to write server state data: UNKNOWN: redis command error: SADD failed: READONLY You can't write against a read only replica.

@hoorayman hoorayman added the bug Something isn't working label Nov 7, 2024
@hoorayman hoorayman changed the title [BUG] Description of the bug: asynq can not follow the right redis master when using sentinel mode. [BUG] asynq can not follow the right redis master when using sentinel mode. Nov 7, 2024
@kamikazechaser
Copy link
Collaborator

This is most likely an issue with misconfiguration when using go-redis with Redis sentinel (Also, because you are running this in k8s, you will most likely have only 1 service that points to the redis sentinel endpoint). Could you debug a simple go-redis application first https://redis.uptrace.dev/guide/go-redis-sentinel.html#redis-server-client?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working investigate
Projects
None yet
Development

No branches or pull requests

3 participants