feat: pipelines public preview

[daniel-sanche]

This PR adds support for Pipeline Queries

[gemini-code-assist]

Summary of Changes

Hello @daniel-sanche, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the capabilities of the Firestore client library by introducing a powerful new pipeline query system. This system allows users to construct and execute multi-stage data processing workflows directly within Firestore, offering greater flexibility and efficiency for complex data operations. Alongside this major feature, the library has been updated to support the latest Python version, ensuring continued compatibility and leveraging modern language features.

Highlights

• New Feature: Firestore Pipeline Queries: This pull request introduces comprehensive support for Firestore Pipeline Queries, enabling complex data transformations and queries directly within the database. This includes new classes for defining pipeline stages, expressions, and handling results, available for both synchronous and asynchronous clients.
• Python 3.14 Compatibility: The client library now officially supports Python 3.14, with updated noxfile.py configurations and setup.py classifiers to reflect this compatibility across unit and system tests.
• Firestore Admin API Enhancements: Added a new CloneDatabase RPC method to the Firestore Admin API, allowing users to clone existing databases. This includes new request/metadata types and integration into the async and synchronous client services and transports.
• Query to Pipeline Conversion: Existing Firestore queries can now be converted into pipeline objects using the new _build_pipeline method, facilitating a smooth transition and integration of existing query logic with the new pipeline functionality.
• Improved mTLS Client Certificate Handling: Refined the logic for determining whether mTLS client certificates should be used, leveraging google-auth's should_use_client_cert for automatic enablement and falling back to environment variables if necessary.
• Enhanced Type Hinting and Code Clarity: Various files across the library have received updated and more explicit type hints, including for WriteResult, document_data, field_updates, and get_nested_value, improving code readability and maintainability.
• Updated Changelog and Documentation: The CHANGELOG.md has been updated to version 2.22.0, detailing new features like mTLS support, Python 3.14, and the CloneDatabase functionality. Documentation files like README.rst and CONTRIBUTING.rst have also been updated to reflect Python 3.14 support.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Ignored Files

• Ignored by pattern: .github/workflows/** (4)
  • .github/workflows/lint.yml
  • .github/workflows/mypy.yml
  • .github/workflows/system_emulated.yml
  • .github/workflows/unittest.yml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a significant new feature, Pipeline Queries, which allows for more complex data transformations and queries. The implementation includes a new fluent API for building pipelines, along with extensive unit and system tests. The PR also migrates the code generation process from owlbot to librarian and adds support for Python 3.14.

Overall, the changes are well-structured. However, I've identified a few issues that need attention. There's a potential issue in the librarian.py script that could affect the release process by overwriting version files. The changelog is also missing an entry for the new pipeline feature, and there are some minor formatting inconsistencies in the new configuration files.

I am having trouble creating individual review comments. Click here to see my feedback.

.librarian/generator-input/librarian.py (46)

Removing **/gapic_version.py from the excludes list is concerning. The gapic_version.py file is typically managed by release-please and should not be overwritten by the generator. This change could potentially break the release process. Please restore the exclusion for **/gapic_version.py.

    s.move(library / f"google/cloud/firestore_{library.name}", excludes=[f"__init__.py", "**/gapic_version.py", "noxfile.py"])

.librarian/generator-input/librarian.py (53)

Removing **/gapic_version.py from the excludes list is concerning. The gapic_version.py file is typically managed by release-please and should not be overwritten by the generator. This change could potentially break the release process. Please restore the exclusion for **/gapic_version.py.

    s.move(library / f"google/cloud/firestore_admin_{library.name}", excludes=[f"__init__.py", "**/gapic_version.py", "noxfile.py"])

.librarian/generator-input/librarian.py (92)

Removing **/gapic_version.py from the excludes list is concerning. The gapic_version.py file is typically managed by release-please and should not be overwritten by the generator. This change could potentially break the release process. Please restore the exclusion for **/gapic_version.py.

        excludes=["**/gapic_version.py", "noxfile.py"],

.kokoro/samples/python3.14/continuous.cfg (6)

This file is missing a final newline. For consistency with other configuration files in this repository, please add one. The same applies to .kokoro/samples/python3.14/presubmit.cfg.

}

.kokoro/samples/python3.14/presubmit.cfg (6)

This file is missing a final newline. For consistency with other configuration files in this repository, please add one.

}

CHANGELOG.md (10)

The changelog is missing an entry for the main feature of this PR, "Pipeline Queries". This is a significant feature and should be highlighted in the release notes. Please add a feature entry for it.

### Features

* feat: Add support for Pipeline Queries