-
Notifications
You must be signed in to change notification settings - Fork 14.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add "queuedEvent" endpoint to get/delete DatasetDagRunQueue #37176
Conversation
a66bc6e
to
3abd4df
Compare
3abd4df
to
c904030
Compare
I came up a few questions during implementation
|
c904030
to
f85319b
Compare
Discussed with @jedcunningham earlier today and make the following changes according to Jed's suggestion (Thanks Jed!)
For point 5, @vincbeck , we're wondering whether you have any thought on it. Thanks! |
I would use |
Thanks for your suggestions! Just updated it. |
f85319b
to
a9ae4c2
Compare
1a2230b
to
32642c5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
might be good for @bbovenzi to have a look. i suspect there wants to be a UI component here for management of the queue if it gets to undesirable state. @cmarteepants is that part of the plan?
365ede9
to
340866e
Compare
For the content in |
7aa1615
to
60861a8
Compare
I think it probably does not matter either way. The provider release only includes provider code, so only the provider changes of this PR will be included anyway. |
Got it. If that's the case, should/could we merge this PR? or do we want to wait for more response on this? Thanks! |
… and get DatasetDagRunQueue
…return NotFound for queueEvent batch delete
…ueue_event, delete_dag_dataset_queue_event, get_dag_dataset_queue_events and delete_dag_dataset_queue_events
…et_queue_events and delete_dataset_queue_events
60861a8
to
5617d60
Compare
Why
In #36618, a user asked for a way to delete DatasetDagRunQueue and devised a workaround that directly manipulated the airflow database. Thus, in this PR, I'm trying to create API endpoints to provide a way to do so without the user needing to directly manipulate the DB
What
Add the following API endpoints to get/delete
DatasetDagRunQueue
with an argumentbefore
, which allows users to filter only theDatasetDagRunQueue
created before this time.DELETE /dags/{dag_id}/datasets/queuedEvent/{uri}
DELETE /dags/{dag_id}/datasets/queuedEvent
DELETE /datasets/queuedEvent/{uri}
GET /dags/{dag_id}/datasets/queuedEvent/{uri}
GET /dags/{dag_id}/datasets/queuedEvent
GET /datasets/queuedEvent/{uri}
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.