Skip to content

Conversation

@arvindp25
Copy link
Contributor

@arvindp25 arvindp25 commented Jul 9, 2025


closes: #53030
RCA is written on actual issue.
Debug step:
I logged key and value which are being passed to persist() method and noticed task_instance obj is getting passed. So when serialization happen. It get failed.
image

Solution: Removed code.

Proof of working
image

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@boring-cyborg boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels Jul 9, 2025
@arvindp25 arvindp25 changed the title Fix GCSToBigQueryOperator: avoid pushing non-serializable task_instancece to XCom Fix GCSToBigQueryOperator: avoid pushing non-serializable task_instance to XCom Jul 9, 2025
@arvindp25 arvindp25 marked this pull request as ready for review July 9, 2025 16:00
@arvindp25
Copy link
Contributor Author

hi @kyungjunleeme
I don't have permission to add you as reviewer. Mentioning here so you can review

Copy link
Member

@gopidesupavan gopidesupavan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice :)

@gopidesupavan gopidesupavan merged commit 35a6b2c into apache:main Jul 10, 2025
123 of 141 checks passed
@kyungjunleeme
Copy link
Contributor

@arvindp25 ah ha its okay. Nice work!!👍👍👍

@jimmykodes
Copy link

I'm curious what the turn around time to getting this tagged usually is. I have a broken deployment from this exact error right now and I'm wondering if I should go through the downgrade to 15 or hang in there.

@potiuk
Copy link
Member

potiuk commented Jul 12, 2025

I'm curious what the turn around time to getting this tagged usually is. I have a broken deployment from this exact error right now and I'm wondering if I should go through the downgrade to 15 or hang in there.

We release providers every two weeks roughly. Ad-hoc release of amazon was just done today after "all providers" release last week. so I guess you can expect ~ 2 weeks release time.

But you can either migrate down or build and install your own provider from the source code and install it if you are in a hurry - this is a regular python project in our repo that you can easily build yourself and install - either using standard python build tools or breeze command line - you can look at the dev RELEASE_PROVIDERS instruction on how release manager does it. So you are absolutely not blocked with any of that - it's up to you how much your time you want to invest in it, the docs and processes are all there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GCSToBigQueryOperator not working after updating to 16.1.0

5 participants