Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add time of the day option when refreshing queries once a week/month #1357

Closed
MarcoMene opened this issue Oct 21, 2016 · 8 comments
Closed

Comments

@MarcoMene
Copy link

It's nice to have top possibility to refresh queries once per week/month.
It would be nicer to have the possibilty to choose the day of the week/month and time of refresh.

@luke14free
Copy link
Contributor

Since with celery we can achieve almost cron-like scheduling (http://docs.celeryproject.org/en/2.0-archived/getting-started/periodic-tasks.html#crontab-like-schedules), it would be nice to expose this to the end user, adding the possibility of running the query on certain days of the week or of the month.

For the days of the week we could show a tick-list of all the days of the week and then the user can chose when to refresh the query (usecase -> I don't want to refresh my dashboard on weekends).

For the days of the month we can have a simple text input asking the user to type, separated with a comma, all the days of the month in which he wants the query to be run.

Another, less user friendly idea, would be to let the user input a crontab-like string to decide when the queries can be updated.

@luke14free
Copy link
Contributor

An idea of UI could be the following:

screen shot 2016-11-22 at 2 13 56 pm

screen shot 2016-11-22 at 2 15 02 pm

screen shot 2016-11-22 at 2 15 08 pm

wdyt @arikfr? there would need to be some changes in the structure of the queries models.. etc

@arikfr
Copy link
Member

arikfr commented Nov 22, 2016

Since with celery we can achieve almost cron-like scheduling (http://docs.celeryproject.org/en/2.0-archived/getting-started/periodic-tasks.html#crontab-like-schedules), it would be nice to expose this to the end user, adding the possibility of running the query on certain days of the week or of the month.

We don't use Celery's scheduler. The main reason being that our scheduler predates the use of Celery in Redash, but also because it's not clear how dynamic Celery's scheduler is. It seems to be improved to be more dynamic in Celery v4.0 (was recently released).

(I will comment about the UI next)

@arikfr
Copy link
Member

arikfr commented Nov 22, 2016

Is "daily" the daily/weekly/cron options and periodic is "Every X"? If so, I would have 3 options there: "No Refresh", "Simple Schedule" ("Every X"), "Advanced Schedule" (the new one).

And why change in the structure of queries model? Maybe we can change the schedule string to store a cron style string? We should be able to represent any of this with a cron syntax, no?

@luke14free
Copy link
Contributor

I agree with the labeling. Yes it would be a cool idea, so basically query.schedule becomes always a CRON-like string. One question tho is whether we are able to execute queries easily in the backend using celery + a cron string without using the scheduler (I still haven't studied well the way we run stuff)

@arikfr
Copy link
Member

arikfr commented Nov 22, 2016

The way scheduling works is:

  • Every 30 seconds, Celery's scheduler invokes the refresh_queries task.
  • This in turn calls Query.outdated_queries method which returns the queries that need to be executed. Then refresh_queries enqueues a task for each of them.
  • outdated_queries calls the should_schedule_next function with the schedule string and the retrieved_at time of the last query result of each query to determine if a query needs to be updated.

So basically you need to alter the should_schedule_next method to support CRON strings. There are several Python libs that parse the cron string.

@luke14free
Copy link
Contributor

👍 thanks

@ranbena
Copy link
Contributor

ranbena commented Apr 18, 2019

I believe this has been addressed in #2396.

@ranbena ranbena closed this as completed Apr 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants