-
Notifications
You must be signed in to change notification settings - Fork 16.4k
AIP-67 - Introduce team id to executor names and loader #44710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AIP-67 - Introduce team id to executor names and loader #44710
Conversation
When multi team is eventually in place, the scheduler will need to initialize a set of executors for each team. This change is a first step towards that which updates the ExecutorName data model and the executor loader to be "team aware".
250f26c to
30396d8
Compare
|
CC @potiuk |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume that it is done as part of AIP-67 - I think that it's worth mentioning it in the title and contents, as well as creating a project for that so it will become easier to track.
Before merging, it would be nice to outline a roadmap (general guidelines) so it would be clearer to understand the broader context of this task :)
Yupp this is AIP-67. That's a fair call out, I'll update the description, title and add a tag.
As I said above, I'll work on this, but it has never before been a requirement of merging PRs. And I don't think it's wise to add that barrier to our contribution process. So I don't think it should block this PR's merge. |
That import was removed in apache#44839, but apache#44710 wasn't up-to-date with main so static checks there didn't fail. This simply adds it back.
I apologize if it was interpreted as a barrier/blocker - it wasn't my intention at all. The doc that you created is exactly what I was looking for, just to understand how it's going to work. |
When multi team (AIP-67) is eventually in place, the scheduler will need to initialize a set of executors for each team. This change is a first step towards that which updates the ExecutorName data model and the executor loader to be "team aware"
That import was removed in apache#44839, but apache#44710 wasn't up-to-date with main so static checks there didn't fail. This simply adds it back.
When multi team (AIP-67) is eventually in place, the scheduler will need to initialize a set of executors for each team. This change is a first step towards that which updates the ExecutorName data model and the executor loader to be "team aware".
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.