feat(experimental): flush the last chunk in append method #1699

Pulkit0110 · 2026-01-08T09:26:15Z

Earlier the last chunk was being flushed while calling the close() method. Now it will be done inside the append method itself.

gemini-code-assist · 2026-01-08T09:26:36Z

Summary of Changes

Hello @Pulkit0110, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refactors the asynchronous object writing mechanism to improve data persistence guarantees. By shifting the responsibility of flushing the final data chunk from the close() method to the append() method, data is now persisted more promptly. This change streamlines the write process, ensures that all data sent via append() is immediately accounted for, and simplifies the overall state management of the object writer.

Highlights

Flushing Logic Relocation: The mechanism for flushing the last data chunk has been moved from the close() method directly into the append() method, ensuring immediate persistence of the final segment of data.
Enhanced Append Method: The append() method now explicitly sets flush=True and state_lookup=True on the gRPC request for the last chunk of data being sent, providing more granular control over data persistence and state updates.
Simplified Close Method: The close() method has been simplified by removing the explicit flush call when finalize_on_close is false, as this responsibility is now handled by the append() method.
Stream Management in Finalize: The finalize() method now includes the logic for closing the underlying write stream and resetting the writer's internal state, centralizing stream cleanup when an object is finalized.
Comprehensive Test Coverage: New system tests and updated unit tests have been added to thoroughly validate the revised flushing behavior and stream management within the AsyncAppendableObjectWriter.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request refactors the AsyncAppendableObjectWriter to flush the last chunk of data within the append method itself, rather than waiting for the close method. This change improves the writer's behavior by ensuring data is persisted more promptly. The implementation correctly handles flushing based on both the flush interval and whether a chunk is the last in an append call. The close and finalize methods have been updated to reflect this new behavior, and the stream is now correctly closed within finalize. The accompanying test changes, including a new system test, effectively validate the new logic. I have one suggestion to slightly refactor the flushing logic for improved clarity.

google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py

chandra-siri · 2026-01-08T11:44:03Z

google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py

+
+            if is_last_chunk:
+                request.state_lookup = True
+                request.flush = True


also set self.bytes_appended_since_last_flush to 0

chandra-siri · 2026-01-08T11:46:51Z

google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py

    async def finalize(self) -> _storage_v2.Object:
        """Finalizes the Appendable Object.

        Note: Once finalized no more data can be appended.


Since during our discussion there was confusion between close and finalize. Can you add one more sentence in Note, to avoid confusion b/w us and users or anyone.

Please note this method is different from `close`, if `.close()` is called data may still be appended to object at a later point in time by opening with generation number. (i.e. `open(..., generation=<object_generation_number>)` . However if `.finalize()` is called no more data can be appended to the object.

chandra-siri · 2026-01-08T11:51:25Z

tests/system/test_zonal.py

+    assert final_object.size == total_size
+
+    # Verify the full content of the object.
+    full_data = small_data + large_data


instead of downloading , should we just match checksum ? it'll be faster (& our system tests will be faster)

chandra-siri · 2026-01-08T11:57:46Z

tests/unit/asyncio/test_async_appendable_object_writer.py

        _storage_v2.BidiWriteObjectRequest(finish_write=True)
    )
    mock_stream.recv.assert_awaited_once()
+    mock_stream.close.assert_awaited_once()


add two more asserts

assert writer._is_stream_open is False assert writer.offset is None

chandra-siri · 2026-01-08T12:04:35Z

google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py

        This method sends the provided `data` to the GCS server in chunks.
        and persists data in GCS at every `_MAX_BUFFER_SIZE_BYTES` bytes by
-        calling `self.simple_flush`.
+        sending `flush=true`.


This method sends the provided `data` to the GCS server in chunks and persists data in GCS at every `_MAX_BUFFER_SIZE_BYTES` bytes or at the last chunk whichever is earlier. Persisting is done by setting `flush=True` on request.

chandra-siri · 2026-01-08T12:27:38Z

google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py

+                request.state_lookup = True
+                request.flush = True
+
+            await self.write_obj_stream.send(request)


we need to add one writer_obj_stream.recv() call because we're sending state_lookup=True in last request.

otherwise gRPC client side buffer will gets filled up in scenario where user just continuously calls appends.

feat(experimental): flush the last chunk in append method

2c3cd2f

Pulkit0110 requested review from a team as code owners January 8, 2026 09:26

blunderbuss-gcf bot assigned chandra-siri Jan 8, 2026

product-auto-label bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/python-storage API. labels Jan 8, 2026

gemini-code-assist bot reviewed Jan 8, 2026

View reviewed changes

google/cloud/storage/_experimental/asyncio/async_appendable_object_writer.py Show resolved Hide resolved

chandra-siri requested changes Jan 8, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(experimental): flush the last chunk in append method #1699

feat(experimental): flush the last chunk in append method #1699

Pulkit0110 commented Jan 8, 2026

Uh oh!

gemini-code-assist bot commented Jan 8, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

chandra-siri Jan 8, 2026

Uh oh!

chandra-siri Jan 8, 2026

Uh oh!

chandra-siri Jan 8, 2026

Uh oh!

chandra-siri Jan 8, 2026

Uh oh!

chandra-siri Jan 8, 2026

Uh oh!

chandra-siri Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat(experimental): flush the last chunk in append method #1699

Are you sure you want to change the base?

feat(experimental): flush the last chunk in append method #1699

Conversation

Pulkit0110 commented Jan 8, 2026

Uh oh!

gemini-code-assist bot commented Jan 8, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

chandra-siri Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

chandra-siri Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

chandra-siri Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

chandra-siri Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

chandra-siri Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

chandra-siri Jan 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants