Add the ability to schedule repeating / recurring / cron-like jobs

[inopinatus]

Is repeating/recurrence planned?

Within delayed_job I currently use a cron-like scheduling job whose task is solely to launch outstanding recurrences and then repeat itself. It works, it's stable, it's a little clunky. A threaded backend might have more options.

However, I realise there's no recurrence construct in Active Job to build on, and from the notes I have a sense that good_job includes a design objective to leverage AJ. So a possible corollary might be, not to extend conceptually much beyond Active Job. In which case I'd be interesting in building up a extension (good_job_cron?) as a related project.

[tomasc]

@inopinatus since GoodJob is compatible with ActiveJob, I assume ActiveJob Scheduler should just work?

[bensheldon]

@inopinatus thanks for asking! That has crossed my mind.

Cron-like scheduling is outside of scope for GoodJob because it's not a native ActiveJob convention.

In my own projects, I'll reach for rufus-scheduler if I'm not on Heroku and able to use Heroku Scheduler. I have thought that Rufus-Scheduler could also probably be greatly slimmed down if it was re-written to target modern Rails with Concurrent Ruby (Good...Time?), but I have my my hands full with GoodJob as it is.

@tomasc thank you for suggesting ActiveJob Scheduler. I wasn't aware of that project, but it looks like it would be compatible with GoodJob because it's compatible with ActiveJob. That seems like a good recommendation.

[inopinatus]

@bensheldon thanks, that was what I figured.

@tomasc I wasn't aware of activejob-scheduler and it looks like it's never been used in production. I'd put a number of questionmarks alongside its design as well. I don't think this is something to recommend.

[tomasc]

@inopinatus thanks for looking into it. I had it on my list to potentially switch from Sidekiq Scheduler. You are right that, upon closer inspection, it does not look very solid.

[bensheldon]

I'm re-opening this issue for visibility and discussion; the current status is still "out of scope but curious".

[bensheldon]

Closing again for backlog grooming ✨

[bensheldon]

I have reconsidered my objections, described in #255, and now believe that this would be a good capability to integrate into GoodJob regardless of whether ActiveJob defines a core interface for it. I've added this Issue to the prioritized backlog. Let's do this 🚀

[bensheldon]

Regarding sequencing, I think first building out concurrency controls (#206) would greatly simplify cron/scheduling functionality.

[nicbet]

Leaving a comment here for anyone else interested in repeated/scheduled job execution with GoodJob:

I'm currently solving this using the clockwork gem (https://github.com/adamwiggins/clockwork), defining my scheduler as scheduler.rb:

require 'clockwork'
require './config/boot'
require './config/environment'

module Clockwork
  every(60.seconds, 'ldap.sync') do
    LdapSyncJob.perform_now if ServerConfig.ldap_enabled
  end
end

paired with a Procfile

web: bundle exec rails server -b 0.0.0.0 -p ${PORT:-5000}
worker: bundle exec good_job start
scheduler: bundle exec clockwork scheduler.rb

That said, native scheduling, and in turn eliminating a gem dependency would obviously be great!

[bensheldon]

I wanted to sketch out an interface for configuring cron, as an imagined Readme section. I was inspired by sidekiq-cron and que-scheduler.

Your feedback is requested 🙏

---

Cron-style repeating / recurring jobs

GoodJob can enqueue jobs on a recurring basis that can be used as a replacement for cron.

# config/environments/production.rb
config.good_job = {
  # ... other configuration
  cron: {
   # Every 15 minutes, enqueue `ExampleJob.set(priority: -10).perform_later(52, name: "Alice")`
    frequent_task: { # each recurring job must have a unique key
       cron: "*/15 * * * *", # cron-style scheduling format by fugit gem
       class: "ExampleJob", # reference the Job class with a string
       args: [42, { name: "Alice" }], # arguments to pass.  Could also allow a Proc for dynamic args, but problematic?
       set: { priority: -10 }, # additional ActiveJob properties. Could also allow a Proc for dynamic args, but problematic?
       description: "Something helpful", # optional description that appears in Dashboard
    }
  }
}

[bensheldon]

I've created a PR for cron-like repeating/recurring functionality in #297. Your feedback is requested.

cc @newbenji