Job scheduler implementation

[dungnmaster]

This PR adds support for creating and dropping jobs in evadb based on this task.

1. Jobs can be created using the create job query:

CREATE JOB {job_name} AS {
{job_queries; ...}
}
START {start_time}
END {end_time}
EVERY {repeat_period} {repeat_unit}

2. Created jobs can be dropped using:

DROP JOB {job_name}

3. The scheduled jobs will only be triggered if the job scheduler process is started explicitly using:

EvaDBConnection.start_jobs()

4. The job scheduler process can be stopped using:

EvaDBConnection.stop_jobs()

[xzdandy]

Hi @dungnmaster, thanks for looking into Job scheduler. What is the status of the PR? Could you summarize the design which helps us understanding the implementation. Thanks!

[dungnmaster]

Hi @dungnmaster, thanks for looking into Job scheduler. What is the status of the PR? Could you summarize the design which helps us understanding the implementation. Thanks!

• In the current implementation, the job scheduler process can be started using the start_jobs() method of the EvaDbConnection object and stopped using the stop_jobs() method.
• The start_jobs() method, starts a new Python process that keeps polling for valid executable tasks in the catalog and also executes them in the same process. This will continue until the job scheduler is stopped or the calling python parent process terminates.

[xzdandy]

We can update the result of job in the catalog. so it is queryable. We can do this in later PRs. Maybe add a TODO.

[dungnmaster]

Sure. I'll add this as a followup change.

[xzdandy]

Great work! Very good quality. Could you also update the documentation https://evadb.readthedocs.io/en/stable/source/reference/evaql.html for the CREATE JOB query? Thanks!