GoogleCalendarToGCSOperator: add unwrap_single flag and return full GCS URIs by Ajay9704 · Pull Request #61284 · apache/airflow

Ajay9704 · 2026-01-31T10:09:03Z

I improved the return values of two Google ToGCS operators to provide full gs:// URIs instead of partial paths, making them more consistent and directly usable.
Changes made:
GoogleCalendarToGCSOperator - Now returns complete GCS URIs (e.g. gs://bucket/calendar_id.json) instead of just object names
GoogleDisplayVideo360SDFtoGCSOperator - Now returns proper GCS URIs (e.g., gs://bucket/object_name) instead of bucket/object format
These changes address part of issue #11323 by standardizing the return format to provide more useful destination URIs for users

Used AI for resolving conflicts

shahar1

Please read my edit to the original issue regarding handling of single entities - we should have a flag for returning a list of strings (unwrap_single).
Please add unit tests.
Before creating new PRs related to this issue, please ask for assignment in the original issue (I've assigned you for these two operators - but please consider it for the next times).

Ajay9704 · 2026-01-31T10:34:49Z

Please read my edit to the original issue regarding handling of single entities - we should have a flag for returning a list of strings (unwrap_single).

Please add unit tests.

Before creating new PRs related to this issue, please ask for assignment in the original issue (I've assigned you for these two operators - but please consider it for the next times).

I've made changes accordingly , please review and let me know if anything else is required

…_unwrap_single_false test

shahar1

Looks better, a couple of comments to resolve before we're ready to go

providers/google/tests/unit/google/cloud/transfers/test_calendar_to_gcs.py

providers/google/tests/unit/google/marketing_platform/operators/test_display_video.py

providers/google/src/airflow/providers/google/marketing_platform/operators/display_video.py

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py

providers/google/src/airflow/providers/google/marketing_platform/operators/display_video.py

providers/google/tests/unit/google/marketing_platform/operators/test_display_video.py

shahar1

Please handle these last changes and I'll aprove

airflow-core/docs/administration-and-deployment/plugins.rst

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py

providers/google/src/airflow/providers/google/marketing_platform/operators/display_video.py

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py

shahar1

Great work Ajay!

cc: @VladaZakharova @MaksYermak

shahar1 · 2026-01-31T15:05:17Z

@Ajay9704
Could you please run prek run --all-files, and fix manually where needed? (it seems that you just did)

One last thing (half related) - it seems that some of the tests are written using Python's native unittest module, instead of pytest. I'll carefully assume that it is due to the usage of AI, which is not forbidden - but it should be supervised and declared explicitly in the PR's description. Please refer to the related docs and note for your next PRs.

Ajay9704 · 2026-01-31T15:23:11Z

@Ajay9704 Could you please run prek run --all-files, and fix manually where needed? (it seems that you just did)

One last thing (half related) - it seems that some of the tests are written using Python's native unittest module, instead of pytest. I'll carefully assume that it is due to the usage of AI, which is not forbidden - but it should be supervised and declared explicitly in the PR's description. Please refer to the related docs and note for your next PRs.

Sorry sir, while resolving I got stuck at one point, so I took help from AI. I will remember this and won’t do it again, and even if I take a small help next time, I will mention it in the PR

I have updated PR description accordingly

Ajay9704 · 2026-01-31T18:01:01Z

@shahar1 i am unable to understand what went wrong can you please help me with this

[Tests / Integration and System Tests / Integration: providers celery (pull_request)] (https://github.com/apache/airflow/actions/runs/21547359138/job/62091006890?pr=61284)Failing after 8m

shahar1 · 2026-01-31T18:13:09Z

@shahar1 i am unable to understand what went wrong can you please help me with this

[Tests / Integration and System Tests / Integration: providers celery (pull_request)] (https://github.com/apache/airflow/actions/runs/21547359138/job/62091006890?pr=61284)Failing after 8m

Sure! I tried to rerun it - it doesn't seem related to this PR specifically, so it's either a flaky integration test or something else that should be solved by merging from main.
Please leave this PR as-is for now and feel free to work on other issues, I'll take care of merging it once it's solved.

Ajay9704 · 2026-01-31T18:23:46Z

@shahar1 i am unable to understand what went wrong can you please help me with this

[Tests / Integration and System Tests / Integration: providers celery (pull_request)] (https://github.com/apache/airflow/actions/runs/21547359138/job/62091006890?pr=61284)Failing after 8m

Sure! I tried to rerun it - it doesn't seem related to this PR specifically, so it's either a flaky integration test or something else that should be solved by merging from main. Please leave this PR as-is for now and feel free to work on other issues, I'll take care of merging it once it's solved.

Sure sir, and thank you for helping throughout

shahar1 · 2026-01-31T18:34:01Z

@shahar1 i am unable to understand what went wrong can you please help me with this

[Tests / Integration and System Tests / Integration: providers celery (pull_request)] (https://github.com/apache/airflow/actions/runs/21547359138/job/62091006890?pr=61284)Failing after 8m

Sure! I tried to rerun it - it doesn't seem related to this PR specifically, so it's either a flaky integration test or something else that should be solved by merging from main. Please leave this PR as-is for now and feel free to work on other issues, I'll take care of merging it once it's solved.

Sure sir, and thank you for helping throughout

Seems to pass succesfully :)
Before merging, I want to give a couple of days for the Google provider team to review it if they have additional comments.

…erator - Add unwrap_single keyword-only parameter to __init__ (default=False) - Update execute() return type to str | list[str] - Implement conditional return logic: - Returns str when unwrap_single=True - Returns list[str] by default (backward compatible) - Add comprehensive docstring updates - Add test_execute_with_unwrap_single() to validate new behavior - Maintain backward compatibility with existing code Follows the pattern used in PR apache#61284

shahar1 · 2026-02-06T08:17:21Z

Are you able to run the system tests for the changes operators? Google team asks to do that before merging to ensure that they still pass after making changes.
If you're not able to do so, we'll have to wait until Google team or someone else does it (if it takes too long I'll handle it on my own).

…erator - Add unwrap_single keyword-only parameter to __init__ (default=False) - Update execute() return type to str | list[str] - Implement conditional return logic: - Returns str when unwrap_single=True - Returns list[str] by default (backward compatible) - Add comprehensive docstring updates - Add test_execute_with_unwrap_single() to validate new behavior - Maintain backward compatibility with existing code Follows the pattern used in PR apache#61284

yuseok89 · 2026-02-09T13:01:41Z

@shahar1
I ran the system tests for GoogleCalendarToGCSOperator.

Test Screenshots

unrwap_single = True

unrwap_single = False

As for GoogleDisplayVideo360SDFtoGCSOperator, I'm not sure an individual account can run the system tests, as they likely require DV360 enterprise access.

shahar1 · 2026-02-10T07:21:02Z

@shahar1 I ran the system tests for GoogleCalendarToGCSOperator.

Test Screenshots

unrwap_single = True
### unrwap_single = False As for `GoogleDisplayVideo360SDFtoGCSOperator`, I'm not sure an individual account can run the system tests, as they likely require DV360 enterprise access.

Thanks a lot, and good point!
@VladaZakharova @MaksYermak What should we do regarding the GoogleDisplayVideo360SDFtoGCSOperator? Are you able to test it on your premises?

@Ajay9704 Until we figure out the above, could you please split the GoogleDisplayVideo360SDFtoGCSOperator from this PR into a separate one?

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py

shahar1

Good job!

I've accidentally misled you in this comment - I meant that return_gcs_uri cannot be set to False altogether with unwrap_single=False (I flipped it the other way around when writing it).
Please add a deprecation warning to return_gcs_uri parameter as well. Be careful - when checking the condition of both flags, they should be bool and not None (otherwise you might compare with None that has a falsy beahvior).
Adjusts tests according to the above.

Afterwards I think that we're good to go.

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py

shahar1

Very last small fix and we're good to go :)

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py

Ajay9704 · 2026-02-11T08:33:34Z

Very last small fix and we're good to go :)

The requested fix has been applied, and all checks are now successful. Please review when convenient.

…CS URIs (apache#61284)

changes address part of issue

9c6f2e9

Ajay9704 requested a review from shahar1 as a code owner January 31, 2026 10:09

boring-cyborg bot added area:providers provider:google Google (including GCP) related issues labels Jan 31, 2026

shahar1 mentioned this pull request Jan 31, 2026

Return files destination uris in all ToGCS operators #11323

Open

14 tasks

shahar1 requested changes Jan 31, 2026

View reviewed changes

added unit tests

7c240b1

Ajay9704 added 3 commits January 31, 2026 16:51

Fix test_calendar_to_gcs.py: Remove problematic test_upload_data_with…

a340898

…_unwrap_single_false test

addressed checks

ca328a3

return types fixed

bf659a8

shahar1 reviewed Jan 31, 2026

View reviewed changes

feedback addressed

f36b1e1

shahar1 reviewed Jan 31, 2026

View reviewed changes

shahar1 self-requested a review January 31, 2026 13:07

shahar1 approved these changes Jan 31, 2026

View reviewed changes

Ajay9704 added 2 commits January 31, 2026 20:36

static

ec9fa3f

static

dc37d27

Ajay9704 force-pushed the GCS_URIs_#11323 branch from 966a548 to dc37d27 Compare January 31, 2026 15:19

Ajay9704 and others added 2 commits January 31, 2026 21:29

addressed issues

d958c34

Update plugins.rst

9c4fe04

Ajay9704 force-pushed the GCS_URIs_#11323 branch from 4041732 to 9c4fe04 Compare February 2, 2026 13:12

Ajay9704 changed the title ~~Return full GCS URIs from Google Calendar and Display Video operators~~ Return full GCS URIs from Google Calendar Feb 10, 2026

Ajay9704 mentioned this pull request Feb 10, 2026

GoogleDisplayVideo360SDFtoGCSOperator: add unwrap_single flag #61709

Merged

Ajay9704 added 6 commits February 10, 2026 13:36

Update display_video.py

89acaac

Update test_display_video.py

cf9b3ae

Update test_display_video.py

3b98ef9

Update display_video.py

7c32887

Update test_display_video.py

7a70086

Update display_video.py

aee021a

shahar1 approved these changes Feb 10, 2026

View reviewed changes

shahar1 requested changes Feb 10, 2026

View reviewed changes

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py Outdated Show resolved Hide resolved

added return gcs uri flag

e398e00

shahar1 reviewed Feb 10, 2026

View reviewed changes

added suggestions

345f220

Ajay9704 force-pushed the GCS_URIs_#11323 branch from 0cf8da2 to 345f220 Compare February 10, 2026 19:39

Update test_calendar_to_gcs.py

1904910

shahar1 approved these changes Feb 11, 2026

View reviewed changes

providers/google/src/airflow/providers/google/cloud/transfers/calendar_to_gcs.py Outdated Show resolved Hide resolved

Update calendar_to_gcs.py

bd9663b

shahar1 changed the title ~~Return full GCS URIs from Google Calendar~~ GoogleCalendarToGCSOperator: add unwrap_single flag and return full GCS URIs Feb 11, 2026

shahar1 merged commit 4ca1da7 into apache:main Feb 11, 2026
90 checks passed

Ratasa143 pushed a commit to Ratasa143/airflow that referenced this pull request Feb 15, 2026

GoogleCalendarToGCSOperator: add unwrap_single flag and return full G…

b1b94a9

…CS URIs (apache#61284)

choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026

GoogleCalendarToGCSOperator: add unwrap_single flag and return full G…

d012619

…CS URIs (apache#61284)

Comments

Conversation

Ajay9704 commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shahar1 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ajay9704 commented Jan 31, 2026

Uh oh!

shahar1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shahar1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shahar1 left a comment

Choose a reason for hiding this comment

Uh oh!

shahar1 commented Jan 31, 2026

Uh oh!

Ajay9704 commented Jan 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Ajay9704 commented Jan 31, 2026

Uh oh!

shahar1 commented Jan 31, 2026

Uh oh!

Ajay9704 commented Jan 31, 2026

Uh oh!

shahar1 commented Jan 31, 2026

Uh oh!

shahar1 commented Feb 6, 2026

Uh oh!

yuseok89 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Screenshots

unrwap_single = True

unrwap_single = False

Uh oh!

shahar1 commented Feb 10, 2026

Test Screenshots

unrwap_single = True

Uh oh!

Uh oh!

shahar1 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shahar1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Ajay9704 commented Feb 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Ajay9704 commented Jan 31, 2026 •

edited

Loading

shahar1 left a comment •

edited

Loading

Ajay9704 commented Jan 31, 2026 •

edited

Loading

yuseok89 commented Feb 9, 2026 •

edited

Loading

shahar1 left a comment •

edited

Loading