Working with Celery on CourtListener

If you are working with celery tasks, it's helpful to use celery events which is a simple curses monitor displaying task and worker history, you can inspect the result and traceback of tasks, and it also supports some management commands like rate limiting and shutting down workers.

1. First, you need to set CELERY_TASK_ALWAYS_EAGER to False under DEVELOPMENT Settings: cl.settings.third_party.celery, to allow running your task into the async worker: CELERY_TASK_ALWAYS_EAGER = False Remember to set it TRUE once debugging is finished, otherwise, tests aren't going to pass.

2. To use celery events you need to run a celery worker that sends events, adding the -E option: docker exec -it cl-celery celery -A cl worker -c 1 -E

3. For concurrent workers, the -c option should be changed in the previous command: -c {NUMBER_OF_CONCURRENT_WORKERS}. Also, may need to update CELERY_WORKER_CONCURRENCY value in cl.settings.third_party.celery

4. Note that if the Celery tasks you are testing are assigned to a special queue, such as "batch1", you will need to specify the queue name to the celery worker mentioned in 2.: docker exec -it cl-celery celery -A cl worker --queues batch1. Otherwise, you would see the tasks being sent, but no worker executing them.

5. Then start celery events monitor: docker exec -it cl-celery celery -A cl events Now, when executing a task you'll be able to monitor it with celery events.