Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add support for scheduling Jobs #1248

Closed
1 of 2 tasks
gaurav274 opened this issue Oct 2, 2023 · 3 comments · Fixed by #1308
Closed
1 of 2 tasks

feat: Add support for scheduling Jobs #1248

gaurav274 opened this issue Oct 2, 2023 · 3 comments · Fixed by #1308
Assignees
@dungnmaster

Comments

@gaurav274
Copy link
Member

Search before asking

  • I have searched the EvaDB issues and found no similar feature requests.

Description

Users want to schedule recurring jobs like retraining the model every 2 hours, running predictions every day, and appending to the table.

Use case

No response

Are you willing to submit a PR?

  • Yes I'd like to help by submitting a PR!
@gaurav274
Copy link
Member Author

@xzdandy @jiashenC For adding support for jobs, is it a good idea to begin by saying that we only support jobs for server/client mode and not the serverless mode?

@gaurav274
Copy link
Member Author

gaurav274 commented Oct 7, 2023 •
edited by dungnmaster
Loading

  • Add support for adding a job
  • Add support for deleting a job
  • Implement a scheduler to fetch and run the jobs
  • Add support for storing the execution history of the triggered jobs

@xzdandy
Copy link
Collaborator

xzdandy commented Oct 19, 2023

An idea just coming to my mind is that we start a background process to manage all background jobs, when bootstraping evadb. And provide a evadb.stop() to kill all of them. This background process can also take care heuristics update and collection in the future.

@dungnmaster dungnmaster mentioned this issue Oct 20, 2023
Merged
@dungnmaster dungnmaster self-assigned this Oct 20, 2023
@xzdandy xzdandy linked a pull request Nov 11, 2023 that will close this issue
Job scheduler implementation #1308
Merged
xzdandy pushed a commit that referenced this issue Nov 14, 2023
@dungnmaster @gaurav274
Job scheduler implementation (#1308)
1fbb74f 
This PR adds support for creating and dropping jobs in evadb based on
this [task](#1248).

1.  Jobs can be created using the create job query:

   
> CREATE JOB {job_name} AS {
>             {job_queries; ...}
>     }
>     START {start_time}
>     END {end_time}
>     EVERY {repeat_period} {repeat_unit}

2. Created jobs can be dropped using:

> DROP JOB {job_name}

3. The scheduled jobs will only be triggered if the job scheduler
process is started explicitly using:

> EvaDBConnection.start_jobs()

4. The job scheduler process can be stopped using:

> EvaDBConnection.stop_jobs()

---------

Co-authored-by: Gaurav Tarlok Kakkar <gaurav21776@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dungnmaster dungnmaster

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Job scheduler implementation georgia-tech-db/evadb
3 participants
@xzdandy @gaurav274 @dungnmaster