Add ability to schedule scripts & reports

[ryanmerolle]

NetBox version

v3.1.5

Feature type

New functionality

Proposed functionality

This feature request is to add the ability to schedule report and script runs.

One idea is to create a new page/section for these schedules and possibly associated results that allows users to view all scheduled operations.

Another idea is to just add this functionality right into the existing scripts and reports page/section

Use case

As a user, I want to be able to schedule recurring scripts and reports in NetBox itself.

Alternatively, a user could setup a schedule outside of UI using an external tool to execute an API call / POST or via the cli. The challenge is that if you are not a user who is aware of the NetBox setup, its hard to follow how and what triggered the script or report run.

Database changes

No response

External dependencies

No response

[jeremystretch]

Alternatively, a user could setup a schedule outside of UI using an external tool to execute an API call / POST or via the cli.

The recommended approach would be to execute the runscript and runreport management commands provided by NetBox as cron jobs.

[ryanmerolle]

Alternatively, a user could setup a schedule outside of UI using an external tool to execute an API call / POST or via the cli.

The recommended approach would be to execute the runscript and runreport management commands provided by NetBox as cron jobs.

Yep, I forgot to mention that particular approach, but with all those "non-native" schedulers the downside would be that you have no way to see or update the schedule (for users not able to ssh to the host).

[ziggekatten]

I have spent some time looking at external tools to do this (currently using cron), but they are either overly complex, not python based and the more important, separate authentication and authorization from NetbBox.

The result is a napkin with some scribbles for a plugin using the Django extension for job scheduling somehow. Will maybe be my first real project coding Django/NetBox plugin. Alternatively I would go the Celery route, though my experience is that it is rather fiddly, at least with Flask. DOnt know yet if Django is more straight forward. Key is to have a single process manage the schedule, like the house keeping in netbox, or beat in Celery.

But I believe this is a valid candidate for a plugin.

[jeremystretch]

It looks like django-rq provides a couple options for scheduling jobs.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Please see our contributing guide.

[kkthxbye-code]

@jeremystretch - I might take a stab at this soonish if it's accepted. I think it would be a shame to auto-close at least. It would be blocked until #8957 is merged though in my opinion. It would be confusing to trigger a job at night, and then missing the result because someone ran the same script in the morning.

[jeremystretch]

@kkthxbye-code Ok, I'll mark this as blocked for now.

[kkthxbye-code]

Not blocked anymore.

I have been experimenting a little with this. The scheduling part is easy and works fine. My worry is that the current script/report system is already a little messy and a rework might be in order.

The split between reports and scripts is already a little thin and I think Jeremy has mentioned merging them before.

Then we have the JobResults class which I'm not really sure is aptly named. Isn't it already more of a JobExecution, JobTask or just Job? Do we want to add a new status to JobResultStatusChoices for scheduled tasks? Otherwise we only know a task has been scheduled by asking django-rq. Do we want to duplicate the scheduled time and add it to the model, or are we satisfied only having that in redis? All this might be fine, but if a user wants to cancel the scheduled task, what does that do to the JobResult?

I can for sure implement a solution with just adding a field for scheduling scripts/reports and moving job results out of the admin panel. Then creating a new view for scheduled tasks, which pretty much just asks django-rq for a list of these, with the option of canceling a task. Not sure if it's the right way to go though.

[DanSheps]

The split between reports and scripts is already a little thin and I think Jeremy has mentioned merging them before.

It has been brought up before. We could definitely open an issue for this, do you think it would be wise to again blocked this issue by the merge issue?

Then we have the JobResults class which I'm not really sure is aptly named. Isn't it already more of a JobExecution, JobTask or just Job? Do we want to add a new status to JobResultStatusChoices for scheduled tasks? Otherwise we only know a task has been scheduled by asking django-rq. Do we want to duplicate the scheduled time and add it to the model, or are we satisfied only having that in redis? All this might be fine, but if a user wants to cancel the scheduled task, what does that do to the JobResult?

So, I played with this in my backup plugin. There are so many things that can be impacted by django-rq to cause the schedule to not kick off correctly. If we do that, we would almost need a "watcher" job to run to make sure it doesn't get dropped into the canceled queue or something equally weird doesn't happen to it.

I can for sure implement a solution with just adding a field for scheduling scripts/reports and moving job results out of the admin panel. Then creating a new view for scheduled tasks, which pretty much just asks django-rq for a list of these, with the option of canceling a task. Not sure if it's the right way to go though.

I will let Jeremy weigh in, but I think this might be the correct course:

1. Merge Scripts/Reports into a single "Job" workflow
2. Move "Jobs" to the main UI and removed from the admin UI
3. Add the scheduling piece
4. Update any existing views/management commands

This likely should be fleshed out in a new issue. Should probably also be done in a 3.x release and not a patch.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

[kkthxbye-code]

I'll give this a shot, see if I can come up with a good solution. If anyone has any additional input, feel free to share.

As a first step the implementation might just be the addition of a time/date schedule field to scripts/reports and then we'll rely on the built-in admin panel views for managing scheduled tasks. FR's can then be created for moving tasks management out of the admin panel if deemed necessary.