Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[apparent bug] uuid appended to job_id makes duplicate next execution scheduling #202

Closed
gabriels1234 opened this issue Dec 6, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@gabriels1234
Copy link
Contributor

gabriels1234 commented Dec 6, 2024

Describe the bug
Using more than one worker for the same queue is somehow duplicating(or more) jobs when scheduling next execution.
EDIT (it stills happens with one worker)

image

(from v2.10)

   def is_scheduled(self) -> bool:
        """Check whether a next job for this task is queued/scheduled to be executed"""
        if self.job_id is None:  # no job_id => is not scheduled
            return False
        # check whether job_id is in scheduled/queued/active jobs
        scheduled_jobs = self.rqueue.scheduled_job_registry.get_job_ids()
        enqueued_jobs = self.rqueue.get_job_ids()
        active_jobs = self.rqueue.started_job_registry.get_job_ids()
        res = (self.job_id in scheduled_jobs) or (self.job_id in enqueued_jobs) or (self.job_id in active_jobs)
        # If the job_id is not scheduled/queued/started,
        # update the job_id to None. (The job_id belongs to a previous run which is completed)
        if not res:
            self.job_id = None
            super(BaseTask, self).save()
        return res
    def _next_job_id(self):
        addition = uuid.uuid4().hex[-10:]
        name = self.name.replace("/", ".")
        return f"{self.queue}:{name}:{addition}"

    def _enqueue_args(self) -> Dict:
        """Args for DjangoQueue.enqueue.
        Set all arguments for DjangoQueue.enqueue/enqueue_at.
        Particularly:
        - set job timeout and ttl
        - ensure a callback to reschedule the job next iteration.
        - Set job-id to proper format
        - set job meta
        """
        res = dict(
            meta=dict(
                task_type=self.TASK_TYPE,
                scheduled_task_id=self.id,
            ),
            on_success=success_callback,
            on_failure=failure_callback,
            job_id=self._next_job_id(),  # <------ makes the job id not identical
        )
        if self.at_front:
            res["at_front"] = self.at_front
        if self.timeout:
            res["job_timeout"] = self.timeout
        if self.result_ttl is not None:
            res["result_ttl"] = self.result_ttl
        return res

To Reproduce
Steps to reproduce the behavior:

  • start a worker twice for the same queue (I used & to start 2 daemons)

Screenshots
Can provide screenshots if needed

Additional context
@cunla Do you see the bug? Do you want me to propose a PR with a fix? (also, could that be in the 2x version and not in v3? We're not ready for the switch. Moreover, doing away with RQ would mean also being Backend-agnostic? (redis, database))

Thanks so much!

@gabriels1234 gabriels1234 added the bug Something isn't working label Dec 6, 2024
@gabriels1234 gabriels1234 changed the title [apparent bug] uuid appended to job_id makes multiple workers [apparent bug] uuid appended to job_id makes duplicate next execution scheduling Dec 6, 2024
@gabriels1234
Copy link
Contributor Author

gabriels1234 commented Dec 6, 2024

I made a temporary fix, job that runs every minute and dedupes the ScheduledRegistry of all the queues ignoring the final random value.
[EDIT: this seems to be an issue running locally, since I reload the worker (automatically) every time there's a code change saved, it might bring duplicates. The issue is still present in non-local environments]

cunla added a commit that referenced this issue Dec 22, 2024
@cunla cunla closed this as completed in 2231190 Dec 22, 2024
cunla added a commit that referenced this issue Dec 22, 2024
cunla added a commit that referenced this issue Dec 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant