[3.x] Allow trimming completed jobs

[themsaid]

High memory consumption was reported in #715. Between v3.2.2 and the most recent version, I couldn't locate a change that may cause this.

However, the proposed solution in the issue seems useful. Basically, this PR adds the ability to purge completed jobs separately, so for example you may keep your recent jobs fo 60 minutes, failed jobs for 7 days, but purge completed jobs every 5 minutes.

One side effect of this change is that the list of recent jobs may include a lot of completed jobs that were purged but still exist in the recent_jobs set and thus paginated results will look strange.

One thing we can do is to have a separate list for:

• Pending Jobs
• Completed Jobs
• Failed jobs

[taylorotwell]

Personally I'm not comfortable with the way this basically breaks pagination. I'm open to other suggestions though.

[mkantautas]

@themsaid This undermines recent. If you have a config from earlier versions with e.g. 'recent' => 1440, // 24 hours , then it is being ignored (recent jobs won't be visible for 24 hours). It's a breaking change and should have been added to a major release only.

[mkantautas]

completed is also not mentioned in the docs @taylorotwell

[themsaid]

@neorganic please upgrade to Horizon 4.0 where the issue doesn't exist.

[mkantautas]

@themsaid Well that's not possible because my project is on 5.8 Laravel, but will have this in mind.