Fix Execution API incompatibilty with addition of consumed_asset_events
#48888
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…nts` Unfortunately since we have forbid extra in the generated client this resulted in old clients failing to work with new servers, one of the things we are deliberately using Cadwyn to avoid. And it does help, when we remember/know to use it. This fixes this specific case, and I've created apache#48887 to catch it in CI long-term.
kaxil
approved these changes
Apr 7, 2025
amoghrajesh
approved these changes
Apr 7, 2025
ashb
added a commit
to astronomer/airflow
that referenced
this pull request
Apr 7, 2025
… change This is a similar sort of change to apache#48888, but in that case the fix is API versioning, here we can't make use of that approach as the the TI isn't sent over the API -- it's sent via the Executor to the worker. So for now the best approach we have here is to allow extra, and be careful when adding new fields to make sure it is version compatible. This will need special handling in apache#48887 too.
ashb
added a commit
that referenced
this pull request
Apr 7, 2025
… change (#48894) This is a similar sort of change to #48888, but in that case the fix is API versioning, here we can't make use of that approach as the the TI isn't sent over the API -- it's sent via the Executor to the worker. So for now the best approach we have here is to allow extra, and be careful when adding new fields to make sure it is version compatible. This will need special handling in #48887 too.
simonprydden
pushed a commit
to simonprydden/airflow
that referenced
this pull request
Apr 8, 2025
simonprydden
pushed a commit
to simonprydden/airflow
that referenced
this pull request
Apr 8, 2025
… change (apache#48894) This is a similar sort of change to apache#48888, but in that case the fix is API versioning, here we can't make use of that approach as the the TI isn't sent over the API -- it's sent via the Executor to the worker. So for now the best approach we have here is to allow extra, and be careful when adding new fields to make sure it is version compatible. This will need special handling in apache#48887 too.
ashb
added a commit
to astronomer/airflow
that referenced
this pull request
Apr 8, 2025
And, like I should have done in apache#48888, add tests this time. The issue was caused by me misunderstanding how Cadwyn works, the `instructions_to_migrate_to_previous_version` only affect the OpenAPI schema, but since the change in this case results in an "invalid" model being returned, we have to write our own migration for the data. For those curious, the error without this migration function is: > fastapi.exceptions.ResponseValidationError: 1 validation errors: > {'type': 'extra_forbidden', 'loc': ('response', 'dag_run', 'consumed_asset_events'), 'msg': 'Extra inputs are not permitted', 'input': []}
ashb
added a commit
that referenced
this pull request
Apr 8, 2025
And, like I should have done in #48888, add tests this time. The issue was caused by me misunderstanding how Cadwyn works, the `instructions_to_migrate_to_previous_version` only affect the OpenAPI schema, but since the change in this case results in an "invalid" model being returned, we have to write our own migration for the data. For those curious, the error without this migration function is: > fastapi.exceptions.ResponseValidationError: 1 validation errors: > {'type': 'extra_forbidden', 'loc': ('response', 'dag_run', 'consumed_asset_events'), 'msg': 'Extra inputs are not permitted', 'input': []}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Unfortunately since we have forbid extra in the generated client this resulted
in old clients failing to work with new servers, one of the things we are
deliberately using Cadwyn to avoid.
And it does help, when we remember/know to use it.
This fixes this specific case, and I've created #48887 to catch it in CI
long-term.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.