Skip to content

Conversation

@shuoweil
Copy link
Contributor

@shuoweil shuoweil commented Oct 4, 2025

When displaying a DataFrame containing JSON columns (including nested JSON in lists or structs), the anywidget table would fail to render and fall back to the "Computation deferred" message. This was caused by a limitation in PyArrow (apache/arrow#45262).

This branch only updates the relevant notebook. The underlying code fix for handling JSON subfields in STRUCT and ARRAY columns was already merged via PR #2216 (fix: support results with STRUCT and ARRAY columns containing JSON subfields in to_pandas_batches()).

This change simply aligns the notebook with the updated code.

Fixes #<448126500 and 453561268> 🦕

@shuoweil shuoweil self-assigned this Oct 4, 2025
@shuoweil shuoweil requested review from a team as code owners October 4, 2025 07:39
@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. labels Oct 4, 2025
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Oct 4, 2025
Copy link
Collaborator

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In b/448126500 I suggest investigating the errors that happen when visualizing STRUCT columns, but this PR doesn't test such cases.

@shuoweil shuoweil force-pushed the shuowei-anywidget-col branch 2 times, most recently from 43a938c to 4a33ccf Compare October 9, 2025 06:43
@shuoweil shuoweil requested a review from tswast October 9, 2025 06:45
@shuoweil shuoweil force-pushed the shuowei-anywidget-col branch from 30dfa7d to 237c134 Compare October 15, 2025 19:45
@shuoweil shuoweil requested a review from tswast October 15, 2025 20:51
Comment on lines 786 to 788
# anywdiget mode uses the same display logic as the "deferred" mode
# for faster execution
if opts.repr_mode in ("deferred", "anywidget"):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's revert this.

@shuoweil shuoweil force-pushed the shuowei-anywidget-col branch from f18fd9e to 8601e52 Compare October 20, 2025 08:20
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Oct 21, 2025
@shuoweil shuoweil force-pushed the shuowei-anywidget-col branch from 533541a to 860db3e Compare October 21, 2025 01:57
@product-auto-label product-auto-label bot added size: m Pull request size is medium. size: l Pull request size is large. and removed size: l Pull request size is large. size: m Pull request size is medium. labels Oct 21, 2025
@shuoweil shuoweil changed the title perf: Default to interactive display for SQL in anywidget mode feat: Display JSON columns in anywidget mode Oct 21, 2025
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Oct 30, 2025
@shuoweil shuoweil changed the title fix: Correctly display DataFrames with JSON columns in anywidget fix: Handle empty DataFrames with nested JSON columns in to_pandas_batches() Oct 30, 2025
@shuoweil shuoweil changed the title fix: Handle empty DataFrames with nested JSON columns in to_pandas_batches() feat: Display JSON columns in anywidget mode fix: Correctly display DataFrames with JSON columns in anywidget Oct 30, 2025
@shuoweil shuoweil changed the title feat: Display JSON columns in anywidget mode fix: Correctly display DataFrames with JSON columns in anywidget feat: Display JSON columns in anywidget mode Oct 30, 2025
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Oct 31, 2025
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels Nov 4, 2025
@shuoweil shuoweil changed the title feat: Display JSON columns in anywidget mode docs: sync notebook with JSON subfield fix from #2216 Nov 4, 2025
@shuoweil shuoweil marked this pull request as ready for review November 4, 2025 20:22
@shuoweil shuoweil changed the title docs: sync notebook with JSON subfield fix from #2216 docs: update notebook for JSON subfields support in to_pandas_batches() Nov 4, 2025
@tswast tswast added the owlbot:run Add this label to trigger the Owlbot post processor. label Nov 4, 2025
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Nov 4, 2025
Copy link
Collaborator

@tswast tswast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tswast tswast enabled auto-merge (squash) November 4, 2025 22:34
@tswast tswast merged commit 5663d2a into main Nov 4, 2025
25 checks passed
@tswast tswast deleted the shuowei-anywidget-col branch November 4, 2025 22:34
sycai pushed a commit that referenced this pull request Nov 10, 2025
🤖 I have created a release *beep* *boop*
---


##
[2.29.0](v2.28.0...v2.29.0)
(2025-11-10)


### Features

* Add bigframes.bigquery.st_regionstats to join raster data from Earth
Engine
([#2228](#2228))
([10ec52f](10ec52f))
* Add DataFrame.resample and Series.resample
([#2213](#2213))
([c9ca02c](c9ca02c))
* SQL Cell no longer escapes formatted string values
([#2245](#2245))
([d2d38f9](d2d38f9))
* Support left_index and right_index for merge
([#2220](#2220))
([da9ba26](da9ba26))


### Bug Fixes

* Correctly iterate over null struct values in ManagedArrowTable
([#2209](#2209))
([12e04d5](12e04d5))
* Simplify UnsupportedTypeError message
([#2212](#2212))
([6c9a18d](6c9a18d))
* Support results with STRUCT and ARRAY columns containing JSON
subfields in `to_pandas_batches()`
([#2216](#2216))
([3d8b17f](3d8b17f))


### Documentation

* Switch API reference docs to pydata theme
([#2237](#2237))
([9b86dcf](9b86dcf))
* Update notebook for JSON subfields support in to_pandas_batches()
([#2138](#2138))
([5663d2a](5663d2a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants