[AI] Unit tests for urlContext APIs #15343

paulb777 · 2025-09-19T21:48:38Z

Add unit tests for the seven mocks added in Add URL Context mock responses FirebaseExtended/vertexai-sdk-test-data#54
Add two additional tests and supporting infra to test empty urlMetadata arrays.

Summary of Changes

This pull request introduces a robust suite of unit tests for the urlContext APIs within both the GoogleAI and VertexAI generative models. The tests cover various response scenarios, ensuring the correct parsing and handling of URL context metadata by the SDK. To support these new tests, the underlying MockURLProtocol was enhanced to handle direct Data responses, complemented by a new utility function for generating mock JSON HTTP handlers.

Highlights

New Unit Tests for URL Context APIs (GoogleAI): Added comprehensive unit tests to GenerativeModelGoogleAITests.swift to validate the handling of urlContext metadata. These tests cover successful retrieval, mixed validity scenarios (success and error), and cases where URL metadata is empty. A new streaming test for urlContext was also introduced.
New Unit Tests for URL Context APIs (VertexAI): Similar to the GoogleAI tests, new unit tests were added to GenerativeModelVertexAITests.swift for urlContext APIs. These tests verify successful retrieval, mixed validity, handling of nonexistent retrieved URLs, and empty URL metadata. A streaming test for urlContext was also included.
From [Firebase AI] Add Candidate decoding tests for urlMetadata #15348
- Added a similar decoding test in GenerateContentResponseTests to verify the same parsing logic when urlMetadata is [] in the Candidate.
- Added another decoding test in GenerateContentResponseTests to verify that the same behaviour occurs when urlMetadata is omitted (rather than an empty array).
  This behaviour is more likely to be observed in practice since [] is typically omitted during JSON serialization in the backend (since it is a default value).

Changelog

FirebaseAI/Tests/Unit/GenerativeModelGoogleAITests.swift
- Added testGenerateContent_success_urlContext to verify successful URL context retrieval.
- Added testGenerateContent_success_urlContext_mixedValidity to test scenarios with both successful and erroneous URL metadata.
- Added testGenerateContent_success_urlContext_emptyURLMetadata to ensure correct handling when URL metadata is empty.
- Added testGenerateContentStream_success_urlContext to test streaming responses with URL context.
FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift
- Added testGenerateContent_success_urlContext to verify successful URL context retrieval.
- Added testGenerateContent_success_urlContext_mixedValidity to test scenarios with both successful and erroneous URL metadata.
- Added testGenerateContent_success_urlContext_nonexistentRetrievedURL to test cases where retrievedURL is missing.
- Added testGenerateContent_success_urlContext_emptyURLMetadata to ensure correct handling when URL metadata is empty.
- Added testGenerateContentStream_success_urlContext to test streaming responses with URL context.
FirebaseAI/Tests/Unit/MockURLProtocol.swift
- Introduced dataRequestHandler static variable to allow mocking HTTP responses with raw Data.
- Modified startLoading to conditionally use requestHandler (for streaming) or dataRequestHandler (for static data).
FirebaseAI/Tests/Unit/TestUtilities/GenerativeModelTestUtil.swift
- Added httpRequestHandler(json:statusCode:) utility function to create URLRequest handlers that return a specified JSON string as Data.

gemini-code-assist · 2025-09-19T21:48:41Z

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

paulb777 · 2025-09-19T21:48:51Z

/gemini review

google-oss-bot · 2025-09-19T21:49:28Z

	1 Warning
⚠️	Did you forget to add a changelog entry? (Add #no-changelog to the PR description to silence this warning.)

Generated by 🚫 Danger

gemini-code-assist

Code Review

This pull request adds unit tests for the urlContext APIs, covering both unary and streaming responses for Google AI and Vertex AI backends. The tests are well-structured and cover important scenarios like success cases and mixed validity responses. I've provided a few suggestions to improve code clarity and correctness. Specifically, I've pointed out some unnecessary uses of XCTUnwrap that can be simplified. I also found a test case where the name and assertion seem to contradict each other, and I've suggested a correction to align them. Overall, these are solid additions to the test suite.

gemini-code-assist · 2025-09-19T21:50:06Z

FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift

+    let candidate = try XCTUnwrap(response.candidates.first)
+    let urlContextMetadata = try XCTUnwrap(candidate.urlContextMetadata)
+    let urlMetadata = try XCTUnwrap(urlContextMetadata.urlMetadata.first)
+    XCTAssertEqual(urlMetadata.retrievedURL?.absoluteString, "https://example.com/8")


There seems to be a contradiction in this test. The test name testGenerateContent_success_urlContext_missingRetrievedURL and the mock resource name unary-success-url-context-missing-retrievedurl suggest that the retrievedURL should be missing or nil. However, the assertion checks if retrievedURL?.absoluteString is equal to a specific string, which implies retrievedURL is not nil.

If the intention is to test for a missing URL, the assertion should probably be XCTAssertNil(urlMetadata.retrievedURL). If the current assertion is correct, consider renaming the test to avoid confusion.

Suggested change

XCTAssertEqual(urlMetadata.retrievedURL?.absoluteString, "https://example.com/8")

XCTAssertNil(urlMetadata.retrievedURL)

Renamed function to testGenerateContent_success_urlContext_retrievedURLPresentOnErrorStatus cc: @dlarocque @andrewheard

FirebaseAI/Tests/Unit/GenerativeModelGoogleAITests.swift

FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

paulb777 · 2025-09-19T22:06:58Z

/gemini summarize
/gemini review

gemini-code-assist · 2025-09-19T22:07:11Z

I can only process one command at a time. Please let me know which command you'd like me to execute first: /gemini summarize or /gemini review.

paulb777 · 2025-09-19T22:07:41Z

/gemini summarize

paulb777 · 2025-09-19T22:10:47Z

/gemini review

gemini-code-assist

Code Review

This pull request adds a robust suite of unit tests for the urlContext APIs, significantly improving test coverage. The changes also include enhancements to the MockURLProtocol to better support testing with static JSON data. The new tests are well-written and cover various scenarios. I have a few minor suggestions to improve the clarity and consistency of some tests.

FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift

FirebaseAI/Tests/Unit/GenerativeModelGoogleAITests.swift

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

FirebaseAI/Tests/Unit/MockURLProtocol.swift

paulb777 · 2025-09-22T18:20:08Z

/gemini summary

paulb777 · 2025-09-22T18:26:12Z

/gemini summary

paulb777 · 2025-09-22T18:26:31Z

/gemini review

gemini-code-assist

Code Review

This pull request adds a comprehensive set of unit tests for the urlContext APIs for both GoogleAI and VertexAI models, covering unary and streaming responses. The tests validate various scenarios like successful retrieval, mixed validity, and empty metadata. The changes are well-structured and significantly improve test coverage for the URL context feature. I've provided a few minor suggestions to enhance test code clarity, consistency, and maintainability by refactoring duplicated code and improving assertions.

FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift

FirebaseAI/Tests/Unit/Types/GenerateContentResponseTests.swift

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

andrewheard

Thanks, Paul!

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Andrew Heard <andrewheard@google.com>

paulb777 added 3 commits September 19, 2025 14:11

checkpoint

538b890

Tests for seven new mock responses now pass

774ce1c

style

48eda93

google-oss-bot added the api: firebaseai label Sep 19, 2025

gemini-code-assist bot reviewed Sep 19, 2025

View reviewed changes

paulb777 and others added 3 commits September 19, 2025 14:53

Apply suggestions from code review

71210ea

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

improve function name

1f410ae

empty urlMetadata array tests

ebd9444

gemini-code-assist bot reviewed Sep 19, 2025

View reviewed changes

FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift Outdated Show resolved Hide resolved

FirebaseAI/Tests/Unit/GenerativeModelGoogleAITests.swift Outdated Show resolved Hide resolved

FirebaseAI/Tests/Unit/GenerativeModelGoogleAITests.swift Show resolved Hide resolved

Apply suggestions from code review

7df2377

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

paulb777 requested review from andrewheard and dlarocque September 19, 2025 22:22

andrewheard reviewed Sep 22, 2025

View reviewed changes

FirebaseAI/Tests/Unit/MockURLProtocol.swift Show resolved Hide resolved

andrewheard and others added 2 commits September 22, 2025 11:14

[Firebase AI] Add Candidate decoding tests for urlMetadata (#15348)

83ee78a

review

ada7b65

cleanup

729edba

firebase deleted a comment from gemini-code-assist bot Sep 22, 2025

gemini-code-assist bot reviewed Sep 22, 2025

View reviewed changes

FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift Show resolved Hide resolved

FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift Outdated Show resolved Hide resolved

FirebaseAI/Tests/Unit/Types/GenerateContentResponseTests.swift Show resolved Hide resolved

Update FirebaseAI/Tests/Unit/GenerativeModelVertexAITests.swift

8382596

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

paulb777 requested a review from andrewheard September 22, 2025 18:31

andrewheard approved these changes Sep 22, 2025

View reviewed changes

dlarocque approved these changes Sep 22, 2025

View reviewed changes

paulb777 merged commit 0e9df30 into ah/firebaseai-urlcontext Sep 22, 2025
43 of 44 checks passed

paulb777 deleted the pb-url-context-unit-tests branch September 22, 2025 21:03

paulb777 added a commit that referenced this pull request Sep 22, 2025

[AI] Unit tests for urlContext APIs (#15343)

e02fd29

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Andrew Heard <andrewheard@google.com>

firebase locked and limited conversation to collaborators Oct 23, 2025

	XCTAssertEqual(urlMetadata.retrievedURL?.absoluteString, "https://example.com/8")
	XCTAssertNil(urlMetadata.retrievedURL)

[AI] Unit tests for urlContext APIs #15343

[AI] Unit tests for urlContext APIs #15343

Uh oh!

Conversation

paulb777 commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of Changes

Highlights

Uh oh!

gemini-code-assist bot commented Sep 19, 2025

Uh oh!

paulb777 commented Sep 19, 2025

Uh oh!

google-oss-bot commented Sep 19, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Sep 19, 2025

Choose a reason for hiding this comment

Uh oh!

paulb777 Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

paulb777 commented Sep 19, 2025

Uh oh!

gemini-code-assist bot commented Sep 19, 2025

Uh oh!

paulb777 commented Sep 19, 2025

Uh oh!

paulb777 commented Sep 19, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paulb777 commented Sep 22, 2025

Uh oh!

paulb777 commented Sep 22, 2025

Uh oh!

paulb777 commented Sep 22, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andrewheard left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

paulb777 commented Sep 19, 2025 •

edited

Loading

paulb777 Sep 19, 2025 •

edited

Loading