Delay an existing job or schedule a new job

[cbisnett]

Add functionality to find previously delayed jobs and update their timestamp or queue a new job if no previously delayed jobs match. This is similar to the find_or_create_by functionality of ActiveRecord.

The current implementation doesn't care about the timestamps of the existing jobs and will update all matching jobs to the specified timestamp. This means that some jobs may be delayed further into the future and others may be executed sooner. Example:

     Job1                  Job2
------^----------*----------^
            New Timestamp

Background
Several times I've found myself needing a solution for a batching up some background work. One of the most common patterns is notifying users when some event happens. The most naive solution is to send an email when an event is created. In some situations this is fine, except when several events happen within a short period of time. Users will quickly get annoyed that you're filling their inbox with notification emails. So the obvious step is to batch up the emails and only send a few notification emails containing multiple events.

I've implemented this previously by creating a scheduled job that runs every X minutes and finds all the notifications that haven't been sent, marks them as sent, and sends the notification email. This works OK except it's hard to control the trade-off between sending the notification in a timely manner and not sending too many emails. For example, if the scheduled job is set to run every 10 minutes and an event is created within the first minute, the notification won't be sent for 9 minutes or more. This isn't ideal because the notification took quite a while to be sent. On the other hand if 50 events are created within the 10 minute window then we've saved the user 49 emails.

With this functionality it's really easy to implement a better solution. When an event is created it will either create a new background job or will delay an existing scheduled job. So we can implement a system that will send a notification email within X minutes of the last event. If a bunch of events are created in quick succession the background job's delayed timer will keep getting reset but will still run within X minutes of the last event. This way we can decrease the window and send the notification emails with less of a delay but still avoid sending too many emails when several events trigger in quick succession.

[iloveitaly]

@cbisnett Thanks for the detailed description! I'll take a deeper look a bit later, but it looks good to me.

Couple things before merging:

• Could you rebase on master so CI runs?
• Could you add a changelog entry?

Would love to get this merged into master!

[cbisnett]

Let me see if I can rebase this on the latest master.

[cbisnett]

@iloveitaly Rebased and added a changelog entry. Let me know if there is anything to be done to get this merged.

[iloveitaly]

Thank you! This is a neat feature.

[matthewhively]

Any ETA when this will be tagged for release?