Feature Ideas

[kmarsh]

I have some wacky feature ideas for good_job and was wondering if you had anything like them planned and, if not, what the appetite would be for merging them if I took them on...

1. Store a job's output. I see there already is a error column... what about an output one that would store a copy of stdout/stderr of a job?
2. Scheduled tasks with a cron-like syntax
3. View job execution history/stats in the web UI
4. Manually triggering a job (with optional params) from the Web UI

The Web UI stuff might be better done as a separate gem that could leverage good_job/ActiveJob's semantics without cluttering the core too much, but 1–2 would probably need to be in good_job proper.

A lot of these features are around an idea I have to replace a Jenkins instance we run just for running one-off and scheduled tasks in our Rails app. (I'm cribbing some of these ideas from RocketJob which I like, but don't care for the MongoDB backend.)

... crazy or genius?

[bensheldon]

Hey @kmarsh! Thanks for thinking of GoodJob!

Overall, I am of the opinion that GoodJob should focus purely on being a backend for ActiveJob. Any additional functionality, that isn't purely backend configuration, should instead be pushed upstream into Rails.

Lemme work through your suggestions one by one:

1. Store a job's output...

I think this is really interesting, though I have some reservation: particularly, how to achieve a reasonable implementation (e.g. intercepting stdout vs logs vs everything). I also think this is something that I would recommend someone build themselves with an around_perform.

2. Scheduled tasks with a cron-like syntax

I strongly believe that background jobs and cron are the two tablestakes operational needs of a Rails app after getting web set up. I also don't believe they're related. This is discussed a little in #53. Overall, I think you should propose an ActiveScheduler upstream in Rails.

3. View job execution history/stats in the web UI

Totally. Do it! 🙌

4. Manually triggering a job (with optional params) from the Web UI

My vision for the web UI is to give some insights and debugging. But generally I think this borders on something that belongs more in application space than backend.

The Web UI stuff might be better done as a separate gem...

I'm of mind that any part of GoodJob I'm developing be part of a monolithic gem. Obviously I can't control if someone wants to ship their own add-on gem, but I'd encourage folks to contribute improvements to core. I'm not too worried (right now) that adding to the Web UI will significantly impact the gem size or maintainability. I'd really appreciate any improvements to the web UI.

[kmarsh]

Thanks for your consideration! (and I apologize this could be a discussion instead of an issue, so I'll go ahead and close this!)

1. Store a job's output...

I think this is really interesting, though I have some reservation: particularly, how to achieve a reasonable implementation (e.g. intercepting stdout vs logs vs everything). I also think this is something that I would recommend someone build themselves with an around_perform.

I wonder if adding an output column would be enough to let jobs conform to their own definition of what "output" is, similar to sidekiq-status. Some may want to trap stdout/stderr in an around_perform, others might want to dump some JSON status there, most would probably be empty.

I guess apps are free to add their own columns to the good_jobs table in app-land migrations? I'll do some experimentation.

I'd really appreciate any improvements to the web UI.

I'll see what I can do!

Thanks so much! 🙌🏻