RedisBeater

RedisBeater is a Celery Beat Scheduler that stores the scheduled tasks and runtime metadata in Redis. It is a fork of RedBeat

Why RedisBeater?

1. Dynamic live task creation and modification, without lengthy downtime
2. Externally manage tasks from any language with Redis bindings
3. Shared data store; Beat isn't tied to a single drive or machine
4. Fast startup even with a large task count
5. Prevent accidentally running multiple Beat servers
6. Work with any schedule class which provide required interface

For more background on the genesis of RedisBeater see this blog post

Getting Started

Install with pip:

pip install celery-redisbeater

Configure RedisBeater settings in your Celery configuration file:

redisbeater_redis_url = "redis://localhost:6379/1"

Then specify the scheduler when running Celery Beat:

celery beat -S redisbeater.RedisBeaterScheduler

RedisBeater uses a distributed lock to prevent multiple instances running. To disable this feature, set:

redisbeater_lock_key = None

More details available on Read the Docs

You can initialize and use RedisBeater just as use forked project. You just need to replace RedBeat with RedisBeater. For instance:

RedisBeaterSchedulerEntry(
    'task-name',
    'tasks.some_task',
    interval,
    args=['arg1', 2],
).save()

Custom Schedule

If you want to use your custom schedule class, you must define encode_beater method and return fields that your class needs when initialized by RedisBeaterScheduler later. For instance:

class customecrontab(BaseSchedule):
    def __init__(self, minute='*', hour='*', day_of_week='*',
             day_of_month='*', month_of_year='*', **kwargs):
    self.hour = hour
    self.minute = minute
    self.day_of_week = day_of_week
    self.day_of_month = day_of_month
    self.month_of_year = month_of_year
    super(crontab, self).__init__(**kwargs)

    def encode_beater(self):
        return {
            'minute': self.minute,
            'hour': self.hour,
            'day_of_week': self.day_of_week,
            'day_of_month': self.day_of_month,
            'month_of_year': self.month_of_year,
        }

Development

RedisBeater is available on GitHub

Once you have the source you can run the tests with the following commands:

pip install -r requirements.dev.txt
py.test tests

You can also quickly fire up a sample Beat instance with:

celery beat --config exampleconf