Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add try_number to log table #40739

Merged
merged 4 commits into from
Jul 17, 2024

Conversation

dstandish
Copy link
Contributor

@dstandish dstandish commented Jul 11, 2024

Needed in order to be able to associate log records with specific TI try.

NOTE I'm actually somewhat tempted to just create a separate log table for this kind of thing, e.g. task_instance_event_log. Being smaller and having a narrower focus has advantages. WDYT?

After talking with @bbovenzi I'm going back to this approach. I.e. just enhance the existing log table to make it work better for this use case. There will be a couple followup PRs for the UI parts and for getting rid of task context logger.

@hussein-awala hussein-awala added type:new-feature Changelog: New Features and removed kind:documentation labels Jul 11, 2024
@hussein-awala hussein-awala added this to the Airflow 2.10.0 milestone Jul 11, 2024
Needed in order to be able to associate log records with specific TI try.
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once comment I had before - this will likely get people some new/unexpected records if they are using the API, but the likeloihood of it is small, so that would not be too breaking (and it could be easily workarounded by try_number not set. If that does not bother others - I am fine with it too

@dstandish
Copy link
Contributor Author

dstandish commented Jul 17, 2024

Once comment I had before - this will likely get people some new/unexpected records if they are using the API, but the likeloihood of it is small, so that would not be too breaking (and it could be easily workarounded by try_number not set. If that does not bother others - I am fine with it too

Yeah I think getting more information has to be expected on the part of users, and the other way around would probably be more breaking! (i.e. not emitting certain events). but still maybe not really breaking even in that case.

@dstandish
Copy link
Contributor Author

the one static check failure is in main; merging

@dstandish dstandish merged commit 103726c into apache:main Jul 17, 2024
47 of 48 checks passed
@dstandish dstandish deleted the add-try-number-to-log-table branch July 17, 2024 18:13
dstandish added a commit that referenced this pull request Jul 17, 2024
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
This is needed in order to be able to associate log records with specific TI try, and to connect the events with specific TIs in the UI.
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:db-migrations PRs with DB migration type:new-feature Changelog: New Features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants