Skip to content

fix(prefect-gcp): prevent double-nesting in GcsBucket._resolve_path#20177

Merged
zzstoatzz merged 5 commits intomainfrom
fix-gcs-bucket-double-nesting-20174
Jan 12, 2026
Merged

fix(prefect-gcp): prevent double-nesting in GcsBucket._resolve_path#20177
zzstoatzz merged 5 commits intomainfrom
fix-gcs-bucket-double-nesting-20174

Conversation

@zzstoatzz
Copy link
Collaborator

@zzstoatzz zzstoatzz commented Jan 12, 2026

closes #20174

this pr fixes the GcsBucket._resolve_path double-nesting bug that occurs when storage_block_id is null (e.g., when context is serialized to remote workers like Ray).

changes

  • add duplicate-prefix check in _resolve_path() to prevent double-nesting when path already starts with bucket_folder
  • add regression test for the fix
  • fix test_push_to_gcs fixture that had broken .prefectignore patterns due to leading whitespace
  • add filterwarnings for upstream deprecation warnings (httplib2, pathspec, google.api_core, coolname)
  • increase test harness timeout to 60s to fix CI flakiness on Python 3.11+ (see Unit tests randomly fail to connect to ephemeral Prefect API server #16397)

prior art

this is the same fix that was applied to _join_bucket_folder in #18237 - the _resolve_path method had the same bug but wasn't fixed at that time.

reproduction script
from prefect_gcp.cloud_storage import GcsBucket

bucket = GcsBucket(bucket="test-bucket", bucket_folder="results/")

# simulate what happens when storage_block_id is null:
# create_result_record() adds bucket_folder to storage_key
already_prefixed_path = "results/abc123"

# before fix: returns "results/results/abc123" (double-nested)
# after fix: returns "results/abc123" (correct)
result = bucket._resolve_path(already_prefixed_path)
print(f"Result: {result}")
assert not result.startswith("results/results/"), f"Double nesting detected: {result}"

🤖 Generated with Claude Code

When storage_block_id is null (e.g., context serialized to Ray workers),
create_result_record() adds bucket_folder to storage_key via _resolve_path.
Then write_path() calls _resolve_path again, causing double-nested paths
like "results/results/abc123" instead of "results/abc123".

Add duplicate-prefix check to _resolve_path, matching the existing check
in _join_bucket_folder.

Fixes #20174

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@github-actions github-actions bot added bug Something isn't working integrations Related to integrations with other services labels Jan 12, 2026
@zzstoatzz zzstoatzz marked this pull request as draft January 12, 2026 16:08
zzstoatzz and others added 2 commits January 12, 2026 10:22
- fix .prefectignore content in test fixture (remove leading whitespace)
- add filterwarnings to pyproject.toml to suppress upstream deprecation warnings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
remove extra disable_logging fixture and temporary_settings wrapper
to match pattern used by prefect-aws and other integrations

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@zzstoatzz zzstoatzz force-pushed the fix-gcs-bucket-double-nesting-20174 branch from caa3a20 to 0d23491 Compare January 12, 2026 16:48
fixes ephemeral server startup timeouts on Python 3.11+ with xdist workers
see #16397

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@zzstoatzz zzstoatzz marked this pull request as ready for review January 12, 2026 17:25
@zzstoatzz zzstoatzz added the development Tech debt, refactors, CI, tests, and other related work. label Jan 12, 2026
Comment on lines 105 to 112
"ignore:'setName' deprecated:DeprecationWarning:httplib2",
"ignore:'setParseAction' deprecated:DeprecationWarning:httplib2",
"ignore:'addParseAction' deprecated:DeprecationWarning:httplib2",
"ignore:'leaveWhitespace' deprecated:DeprecationWarning:httplib2",
"ignore:'delimitedList' deprecated:DeprecationWarning:httplib2",
"ignore:GitWildMatchPattern .* is deprecated:DeprecationWarning:pathspec",
"ignore:You are using a Python version .* which Google will stop supporting:FutureWarning:google.api_core",
"ignore:codecs.open\\(\\) is deprecated:DeprecationWarning:coolname",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would updating the versions of libraries these warnings are originating get rid of the warnings?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defining pathspec specifically as a dep would fix the GitWildMatchPattern but for httplib2 and coolname no, it would not

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant updating the uv.lock for prefect-gcp specific libraries since I don't know the last time that was updated.

Copy link
Collaborator Author

@zzstoatzz zzstoatzz Jan 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, that's what i did locally. im saying that even with updating prefect-gcp to latest versions for the problematic packages, i still see the warnings. the only mediation i saw was upgrading pathspec fixes the GitWildMatchPattern warning, but the rest still warn on their newest versions

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i think pathspec is still warning even on the latest

@zzstoatzz zzstoatzz requested a review from desertaxle January 12, 2026 19:01
@zzstoatzz zzstoatzz merged commit 3cebd24 into main Jan 12, 2026
14 checks passed
@zzstoatzz zzstoatzz deleted the fix-gcs-bucket-double-nesting-20174 branch January 12, 2026 20:03
@vladgrish
Copy link

when can we expect a release with the fix applied?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working development Tech debt, refactors, CI, tests, and other related work. integrations Related to integrations with other services

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[prefect-gcp] GcsBucket._resolve_path causes double-nesting when storage_block_id is null

3 participants