CHANGELOG.md

Changelog

PyPI History

3.27.0 (2024-11-01)

Features

• Updates to allow users to set max_stream_count (#2039) (7372ad6)

3.26.0 (2024-09-25)

Features

• Include LegacyPandasError in init imports (#2014) (3ab5e95)
• Use bigquery-magics package for the %%bigquery magic (#1965) (60128a5)

Bug Fixes

• Add docfx to the presubmit configuration and delete docs-presubmit (#1995) (bd83cfd)
• Add warning when encountering unknown field types (#1989) (8f5a41d)
• Allow protobuf 5.x; require protobuf >=3.20.2; proto-plus >=1.22.3 (#1976) (57bf873)
• Do not set job timeout extra property if None (#1987) (edcb79c)
• Set pyarrow field nullable to False for a BigQuery field in REPEATED mode (#1999) (5352870)

Dependencies

• Bump min version of google-api-core and google-cloud-core to 2.x (#1972) (a958732)

Documentation

• Add short mode query sample & test (#1978) (ba61a8a)
• Improve QueryJobConfig.destination docstring (#2016) (1b4cca0)

3.25.0 (2024-06-17)

Features

• Add prefer_bqstorage_client option for Connection (#1945) (bfdeb3f)
• Support load job option ColumnNameCharacterMap (#1952) (7e522ee)

Bug Fixes

• Do not overwrite page_size with max_results when start_index is set (#1956) (7d0fcee)

3.24.0 (2024-06-04)

Features

• Add default timeout for Client.get_job() (#1935) (9fbad76)
• Add support for map target type in Parquet options (#1919) (c3f7b23)

Bug Fixes

• Create query job in job.result() if doesn't exist (#1944) (8f5b4b7)
• Retry is_job_done on ConnectionError (#1930) (4f72723)

Performance Improvements

• If page_size or max_results is set on QueryJob.result(), use to download first page of results (#1942) (3e7a48d)

3.23.1 (2024-05-21)

Performance Improvements

• Decrease the threshold in which we use the BQ Storage Read API (#1925) (eaa1a52)

3.23.0 (2024-05-16)

Features

• Adds timer decorator to facilitate debugging (#1917) (ea750e0)
• Support insertAll for range (#1909) (74e75e8)

Bug Fixes

• Add pyarrow version check for range support (#1914) (a86d7b9)
• Edit presubmit for to simplify configuration (#1915) (b739596)

3.22.0 (2024-04-19)

Features

• Support RANGE in queries Part 2: Arrow (#1868) (5251b5d)

3.21.0 (2024-04-18)

Features

• Add compression option ZSTD. (#1890) (5ed9cce)
• Adds billing to opentel (#1889) (38697fb)
• Support RANGE in queries Part 1: JSON (#1884) (3634405)

Bug Fixes

• Add types to DatasetReference constructor (#1601) (bf8861c)
• Creates linting-typing.cfg in presubmit (#1881) (c852c15)
• Remove duplicate key time_partitioning from Table._PROPERTY_TO_A… (#1898) (82ae908)
• Retry query jobs that fail even with ambiguous jobs.getQueryResults REST errors (#1903, #1900) (1367b58)

Performance Improvements

• Avoid unnecessary API call in QueryJob.result() when job is already finished (#1900) (1367b58)

3.20.1 (2024-04-01)

Bug Fixes

• Make pyarrow an optional dependency post-3.20.0 yanked release (#1879) (21714e1)

3.20.0 (2024-03-27)

Features

• Add fields parameter to set_iam_policy for consistency with update methods (#1872) (08b1e6f)

Bug Fixes

• Correct type checking (#1848) (2660dbd)
• Update error logging when converting to pyarrow column fails (#1836) (0ac6e9b)
• Updates a number of optional dependencies (#1864) (c2496a1)
• Use an allowlist instead of denylist to determine when query_and_wait uses jobs.query API (#1869) (e265db6)

3.19.0 (2024-03-11)

Features

• Support RANGE query parameters (#1827) (b359a9a)
• Support range sql (#1807) (86a45c9)

Bug Fixes

• Add google-auth as a direct dependency (713ce2c)
• Augment universe_domain handling (#1837) (53c2cbf)
• deps: Require google-api-core>=1.34.1, >=2.11.0 (713ce2c)
• Supplementary fix to env-based universe resolution (#1844) (b818992)
• Supplementary fix to env-based universe resolution (#1847) (6dff50f)

3.18.0 (2024-02-29)

Features

• Support nullable boolean and Int64 dtypes in insert_rows_from_dataframe (#1816) (ab0cf4c)
• Support slot_ms in QueryPlanEntry (#1831) (d62cabb)

Bug Fixes

• Keyword rendering and docstring improvements (#1829) (4dfb920)

Documentation

• samples: Updates to urllib3 constraint for Python 3.7 (#1834) (b099c32)
• Update client_query_w_named_params.py to use query_and_wait API (#1782) (89dfcb6)

3.17.2 (2024-01-30)

Bug Fixes

• Change load_table_from_json autodetect logic (#1804) (6249032)

Documentation

• Update to use API (#1781) (81563b0)
• Update client_query_destination_table.py sample to use query_and_wait (#1783) (68ebbe1)
• Update query_external_sheets_permanent_table.py to use query_and_wait API (#1778) (a7be88a)
• Update sample for query_to_arrow to use query_and_wait API (#1776) (dbf10de)
• Update the query destination table legacy file to use query_and_wait API (#1775) (ef89f9e)
• Update to use query_and_wait in client_query_w_positional_params.py (#1786) (410f71e)
• Update to use query_and_wait in samples/client_query_w_timestamp_params.py (#1785) (ba36948)
• Update to_geodataframe to use query_and_wait functionality (#1800) (1298594)

3.17.1 (2024-01-24)

Bug Fixes

• Add pyarrow.large_strign to the _ARROW_SCALAR_IDS_TO_BQ map (#1796) (b402a6d)
• Retry 'job exceeded rate limits' for DDL queries (#1794) (39f33b2)

3.17.0 (2024-01-24)

Features

• Support universe resolution (#1774) (0b5c1d5)

Bug Fixes

• query_and_wait now retains unknown query configuration _properties (#1793) (4ba4342)
• Raise ValueError in query_and_wait with wrong job_config type (4ba4342)

Documentation

• Remove unused query code sample (#1769) (1f96439)
• Update snippets.py to use query_and_wait (#1773) (d90602d)
• Update multiple samples to change query to query_and_wait (#1784) (d1161dd)
• Update the query with no cache sample to use query_and_wait API (#1770) (955a4cd)
• Updates query to query and wait in samples/desktopapp/user_credentials.py (#1787) (89f1299)

3.16.0 (2024-01-12)

Features

• Add table_constraints field to Table model (#1755) (a167f9a)
• Support jsonExtension in LoadJobConfig (#1751) (0fd7347)

Bug Fixes

• Add detailed message in job error (#1762) (08483fb)

3.15.0 (2024-01-09)

Features

• Support JSON type in insert_rows and as a scalar query parameter (#1757) (02a7d12)
• Support RANGE in schema (#1746) (8585747)

Bug Fixes

• Deserializing JSON subfields within structs fails (#1742) (0d93073)
• Due to upstream change in dataset, updates expected results (#1761) (132c14b)
• Load_table_from_dataframe for higher scale decimal (#1703) (b9c8be0)
• Updates types-protobuf version for mypy-samples nox session (#1764) (c0de695)

Performance Improvements

• DB-API uses more efficient query_and_wait when no job ID is provided (#1747) (d225a94)

3.14.1 (2023-12-13)

Bug Fixes

• Add missing handler for deserializing json value (#1587) (09017a9)

3.14.0 (2023-12-08)

Features

• Add Client.query_and_wait which directly returns a RowIterator of results (#1722) (89a647e)
• Add job_id, location, project, and query_id properties on RowIterator (#1733) (494f275)
• Add job_timeout_ms to job configuration classes (#1675) (84d64cd)
• Add support dataset.max_time_travel_hours (#1683) (f22eff2)
• Add support for Dataset.isCaseInsensitive (#1671) (386fa86)
• Add support for Python 3.12 (#1736) (3c0976a)
• Removed pkg_resources from all test files and moved importlib into pandas extra (#1726) (1f4ebb1)
• Support data_governance_type (#1708) (eff365d)

Bug Fixes

• load_table_from_dataframe now assumes there may be local null values (#1735) (f05dc69)
• Ensure query job retry has longer deadline than API request deadline (#1734) (5573579)
• Keep RowIterator.total_rows populated after iteration (#1748) (8482f47)
• Move grpc, proto-plus and protobuf packages to extras (#1721) (5ce4d13)

Performance Improvements

• Use the first page a results when query(api_method="QUERY") (#1723) (6290517)

3.13.0 (2023-10-30)

Features

• Add Model.transform_columns property (#1661) (5ceed05)
• Add support for dataset.default_rounding_mode (#1688) (83bc768)

Bug Fixes

• AccessEntry API representation parsing (#1682) (a40d7ae)

Documentation

• Remove redundant bigquery_update_table_expiration code sample (#1673) (2dded33)
• Revised create_partitioned_table sample (#1447) (40ba859)
• Revised relax column mode sample (#1467) (b8c9276)

3.12.0 (2023-10-02)

Features

• Add Dataset.storage_billing_model setter, use client.update_dataset(ds, fields=["storage_billing_model"]) to update (#1643) (5deba50)
• Search statistics (#1616) (b930e46)
• Widen retry predicate to include ServiceUnavailable (#1641) (3e021a4)

Bug Fixes

• Allow storage_billing_model to be explicitly set to None to use project default value (#1665) (514d3e1)
• Relax timeout expectations (#1645) (1760e94)
• Use isinstance() per E721, unpin flake8 (#1659) (54a7769)

Documentation

• Revise update_table_expiration sample (#1457) (03194e0)

3.11.4 (2023-07-19)

Bug Fixes

• Updates typing in function definitions (#1613) (db755ce)

3.11.3 (2023-06-27)

Bug Fixes

• Type annotations include Optional when None is accepted (#1554) (6c1ab80)

3.11.2 (2023-06-21)

Bug Fixes

• Updates tests based on revised hacker_news tables (#1591) (d73cf49)

3.11.1 (2023-06-09)

Documentation

• Add/reformat return types for cloud RAD docs (#1582) (6efdce1)

3.11.0 (2023-06-01)

Features

• Add remote function options to routines (#1558) (84ad11d)

Bug Fixes

• Filter None values from OpenTelemetry attributes (#1567) (9ea2e21)
• Handle case when expirationMs is None (#1553) (fa6e13d)
• Raise most recent exception when not able to fetch query job after starting the job (#1362) (09cc1df)

3.10.0 (2023-04-18)

Features

• Add date, datetime, time, timestamp dtype to to_dataframe (#1547) (64e913d)

3.9.0 (2023-03-28)

Features

• Expose query job on dbapi cursor (#1520) (339eb0e)

Bug Fixes

• Keyerror when the load_table_from_dataframe accesses a unmapped dtype dataframe index (#1535) (a69348a)

3.8.0 (2023-03-24)

Features

• Add bool, int, float, string dtype to to_dataframe (#1529) (5e4465d)
• Add default LoadJobConfig to Client (#1526) (a2520ca)
• Expose configuration property on CopyJob, ExtractJob, LoadJob, QueryJob (#1521) (8270a10)

Bug Fixes

• Loosen ipywidgets restrictions further to address ipython compatibility issues (#1531) (50e5026)

3.7.0 (2023-03-06)

Features

• Add connection_properties and create_session to LoadJobConfig (#1509) (cd0aaa1)
• Add default_query_job_config property and property setter to BQ client (#1511) (a23092c)

Documentation

• Remove < 3.11 reference from README (#1502) (c7417f4)

3.6.0 (2023-02-22)

Features

• Adding preserveAsciiControlCharacter to CSVOptions (#1491) (f832e7a)

Bug Fixes

• Annotate optional integer parameters with optional type (#1487) (a190aaa)
• Loosen ipywidget dependency (#1504) (20d3276)
• Removes scope to avoid unnecessary duplication (#1503) (665d7ba)

Dependencies

• Update minimum google-cloud-core to 1.6.0 (a190aaa)

3.5.0 (2023-01-31)

Features

• Add str method to DatasetReference (#1477) (f32df1f)
• Add preserveAsciiControlCharacter to LoadJobConfig (#1484) (bd1da9a)

Documentation

• Adds snippet for creating table with external data config (#1420) (f0ace2a)
• Revise delete label table code sample, add TODO to clean up sni… (#1466) (0dab7d2)
• samples: Table variable fix (#1287) (a71888a)

3.4.2 (2023-01-13)

Bug Fixes

• Add support for python 3.11 (#1463) (730a1de)
• Require grpcio >= 1.49.1 for python 3.11 (72b25c5)

Dependencies

• Remove upper bound on packaging dependency (#1440) (6088129)

Documentation

• Create sample to write schema file from table (#1439) (093cc68)
• Created samples for load table and create table from schema file (#1436) (8ad2e5b)
• Revise create table cmek sample (#1452) (57740e4)
• Revise get table labels code sample, add TODO to clean up snipp… (#1464) (b5ccbfe)
• Revise label table code samples (#1451) (14ae1f2)
• Revise sample for nested schema (#1446) (a097631)

3.4.1 (2022-12-09)

Documentation

• Add info about streaming quota limits to insert_rows* methods (#1409) (0f08e9a)

Dependencies

• make pyarrow and BQ Storage optional dependencies (e1aa921)

3.4.0 (2022-11-17)

Features

• Add reference_file_schema_uri to LoadJobConfig, ExternalConfig (#1399) (931285f)
• Add default value expression (#1408) (207aa50)
• Add More Specific Type Annotations for Row Dictionaries (#1295) (eb49873)

3.3.6 (2022-11-02)

Features

• Reconfigure tqdm progress bar in %%bigquery magic (#1355) (506f781)

Bug Fixes

• Corrects test for non-existent attribute (#1395) (a80f436)
• deps: Allow protobuf 3.19.5 (#1379) (3e4a074)
• deps: Allow pyarrow < 11 (#1393) (c898546)
• deps: Require requests>=2.21.0 (#1388) (e398336)
• Refactor to adapt to changes to shapely dependency (#1376) (2afd278)

Documentation

• Fix typos (#1372) (21cc525)

Miscellaneous Chores

• release 3.3.6 (4fce1d9)

3.3.4 (2022-09-29)

Bug Fixes

• deps: Require protobuf >= 3.20.2 (#1369) (f13383a)

3.3.3 (2022-09-28)

Bug Fixes

• Refactors code to account for a tdqm code deprecation (#1357) (1369a9d)
• Validate opentelemetry span job attributes have values (#1327) (8287af1)

Documentation

• samples: uses function (create_job) more appropriate to the described sample intent (5aeedaa)

3.3.2 (2022-08-16)

Bug Fixes

• deps: require proto-plus >= 1.22.0 (1de7a52)
• deps: require protobuf >=3.19, < 5.0.0 (#1311) (1de7a52)

3.3.1 (2022-08-09)

Bug Fixes

• deps: allow pyarrow < 10 (#1304) (13616a9)

3.3.0 (2022-07-25)

Features

• add destination_expiration_time property to copy job (#1277) (728b07c)

Bug Fixes

• require python 3.7+ (#1284) (52d9f14)

Documentation

• samples: add table snapshot sample (#1274) (e760d1b)
• samples: explicitly add bq to samples reqs, upgrade grpc to fix bug on m1 (#1290) (9b7e3e4)

3.2.0 (2022-06-06)

Features

• add support for table clones (#1235) (176fb2a)

Bug Fixes

• deps: proto-plus >= 1.15.0, <2.0.0dev (ba58d3a)
• deps: require packaging >= 14.3, <22.0.0dev (ba58d3a)
• deps: require protobuf>= 3.12.0, <4.0.0dev (#1263) (ba58d3a)

Documentation

• fix changelog header to consistent size (#1268) (d03e2a2)

3.1.0 (2022-05-09)

Features

• add str method to table (#1199) (8da4fa9)
• refactor AccessEntry to use _properties pattern (#1125) (acd5612)
• support using BIGQUERY_EMULATOR_HOST environment variable (#1222) (39294b4)

Bug Fixes

• deps: allow pyarrow v8 (#1245) (d258690)
• export bigquery.HivePartitioningOptions (#1217) (8eb757b)
• Skip geography_as_object conversion for REPEATED fields (#1220) (4d3d6ec)

Documentation

• updated variable typo in comment in code sample (#1239) (e420112)

3.0.1 (2022-03-30)

Bug Fixes

• deps: raise exception when pandas is installed but db-dtypes is not (#1191) (4333910)
• deps: restore dependency on python-dateutil (#1187) (212d7ec)

3.0.0 (2022-03-29)

⚠ BREAKING CHANGES

• BigQuery Storage and pyarrow are required dependencies (#776)
• use nullable Int64 and boolean dtypes in to_dataframe (#786)
• destination tables are no-longer removed by create_job (#891)
• In to_dataframe, use dbdate and dbtime dtypes from db-dtypes package for BigQuery DATE and TIME columns (#972)
• automatically convert out-of-bounds dates in to_dataframe, remove date_as_object argument (#972)
• mark the package as type-checked (#1058)
• default to DATETIME type when loading timezone-naive datetimes from Pandas (#1061)
• remove out-of-date BigQuery ML protocol buffers (#1178)

Features

• add api_method parameter to Client.query to select INSERT or QUERY API (#967) (76d88fb)
• default to DATETIME type when loading timezone-naive datetimes from Pandas (#1061) (76d88fb)
• destination tables are no-longer removed by create_job (#891) (76d88fb)
• In to_dataframe, use dbdate and dbtime dtypes from db-dtypes package for BigQuery DATE and TIME columns (#972) (76d88fb)
• mark the package as type-checked (#1058) (76d88fb)
• use StandardSqlField class for Model.feature_columns and Model.label_columns (#1117) (76d88fb)

Bug Fixes

• automatically convert out-of-bounds dates in to_dataframe, remove date_as_object argument (#972) (76d88fb)
• improve type annotations for mypy validation (#1081) (76d88fb)
• remove out-of-date BigQuery ML protocol buffers (#1178) (76d88fb)
• use nullable Int64 and boolean dtypes in to_dataframe (#786) (76d88fb)

Documentation

• Add migration guide from version 2.x to 3.x (#1027) (76d88fb)

Dependencies

• BigQuery Storage and pyarrow are required dependencies (#776) (76d88fb)

2.34.3 (2022-03-29)

Bug Fixes

• update content-type header (#1171) (921b440)

2.34.2 (2022-03-05)

Bug Fixes

• deps: require google-api-core>=1.31.5, >=2.3.2 (#1157) (0c15790)
• deps: require proto-plus>=1.15.0 (0c15790)

2.34.1 (2022-03-02)

Dependencies

• add "extra" for IPython, exclude bad IPython release (#1151) (0fbe12d)
• allow pyarrow 7.0 (#1112) (57f8ea9)

2.34.0 (2022-02-18)

Features

• support BI Engine statistics in query job (#1144) (7482549)

2.33.0 (2022-02-16)

Features

• add --no_query_cache option to %%bigquery magics to disable query cache (#1141) (7dd30af)

Bug Fixes

• return 403 when VPC-SC violation happens (#1131) (f5daa9b)

Documentation

• reference BigQuery REST API defaults in LoadJobConfig descrip… (#1132) (18d9580)
• show common job properties in get_job and cancel_job samples (#1137) (8edc10d)

2.32.0 (2022-01-12)

Features

• support authorized dataset entity (#1075) (c098cd0)

Bug Fixes

• remove query text from exception message, use exception.debug_message instead (#1105) (e23114c)

2.31.0 (2021-11-24)

Features

• allow cell magic body to be a $variable (#1053) (3a681e0)
• promote RowIterator.to_arrow_iterable to public method (#1073) (21cd710)

Bug Fixes

• apply timeout to all resumable upload requests (#1070) (3314dfb)

Dependencies

• support OpenTelemetry >= 1.1.0 (#1050) (4616cd5)

2.30.1 (2021-11-04)

Bug Fixes

• error if eval()-ing repr(SchemaField) (#1046) (13ac860)

Documentation

• show gcloud command to authorize against sheets (#1045) (20c9024)
• use stable URL for pandas intersphinx links (#1048) (73312f8)

2.30.0 (2021-11-03)

Features

• accept TableListItem where TableReference is accepted (#1016) (fe16adc)
• support Python 3.10 (#1043) (5bbb832)

Documentation

• add code samples for Jupyter/IPython magics (#1013) (61141ee)
• samples: add create external table with hive partitioning (#1033) (d64f5b6)

2.29.0 (2021-10-27)

Features

• add QueryJob.schema property for dry run queries (#1014) (2937fa1)
• add session and connection properties to QueryJobConfig (#1024) (e4c94f4)
• add support for INTERVAL data type to list_rows (#840) (e37380a)
• allow queryJob.result() to be called on a dryRun (#1015) (685f06a)

Documentation

• document ScriptStatistics and other missing resource classes (#1023) (6679109)
• fix formatting of generated client docstrings (#1009) (f7b0ee4)

Dependencies

• allow pyarrow 6.x (#1031) (1c2de74)

2.28.1 (2021-10-07)

Bug Fixes

• support ARRAY data type when loading from DataFrame with Parquet (#980) (1e59083)

2.28.0 (2021-09-30)

Features

• add AvroOptions to configure AVRO external data (#994) (1a9431d)

Documentation

• link to stable pandas docs (#990) (ea50e80)

2.27.1 (2021-09-27)

Bug Fixes

• remove py.typed since package fails mypy check (#988) (39030f2)

2.27.0 (2021-09-24)

Features

• Add py.typed for PEP 561 compliance (#976) (96e6bee)
• include key metadata in Job representation (#964) (acca1cb)

Bug Fixes

• Arrow extension-type metadata was not set when calling the REST API or when there are no rows (#946) (864383b)
• disambiguate missing policy tags from explicitly unset policy tags (#983) (f83c00a)
• remove default timeout (#974) (1cef0d4)

Documentation

• simplify destination table sample with f-strings (#966) (ab6e76f)

2.26.0 (2021-09-01)

Features

• set the X-Server-Timeout header when timeout is set (#927) (ba02f24)

Bug Fixes

• guard imports against unsupported pyarrow versions (#934) (b289076)

2.25.2 (2021-08-31)

Bug Fixes

• error inserting DataFrame with REPEATED field (#925) (656d2fa)
• underscores weren't allowed in struct field names when passing parameters to the DB API (#930) (fcb0bc6)

Documentation

• update docstring for bigquery_create_routine sample (#883) (#917) (e2d12b7)

2.25.1 (2021-08-25)

Bug Fixes

• populate default timeout and retry after client-side timeout (#896) (b508809)
• use REST API in cell magic when requested (#892) (1cb3e55)

2.25.0 (2021-08-24)

Features

• Support using GeoPandas for GEOGRAPHY columns (#848) (16f65e6)

2.24.1 (2021-08-13)

Bug Fixes

• remove pytz dependency and require pyarrow>=3.0.0 (#875) (2cb3563)

2.24.0 (2021-08-11)

Features

• add support for transaction statistics (#849) (7f7b1a8)
• make the same Table* instances equal to each other (#867) (c1a3d44)
• retry failed query jobs in result() (#837) (519d99c)
• support ScalarQueryParameterType for type_ argument in ScalarQueryParameter constructor (#850) (93d15e2)

Bug Fixes

• make unicode characters working well in load_table_from_json (#865) (ad9c802)

2.23.3 (2021-08-06)

Bug Fixes

• increase default retry deadline to 10 minutes (#859) (30770fd)

2.23.2 (2021-07-29)

Dependencies

• expand pyarrow pins to support 5.x releases (#833) (80e3a61)

2.23.1 (2021-07-28)

Bug Fixes

• insert_rows() accepts float column values as strings again (#824) (d9378af)

2.23.0 (2021-07-27)

Features

• Update proto definitions for bigquery/v2 to support new proto fields for BQML. (#817) (fe7a902)

Bug Fixes

• no longer raise a warning in to_dataframe if max_results set (#815) (3c1be14)
• retry ChunkedEncodingError by default (#802) (419d36d)

Documentation

• correct docs for LoadJobConfig.destination_table_description (#810) (da87fd9)

2.22.1 (2021-07-22)

Bug Fixes

• issue a warning if buggy pyarrow is detected (#787) (e403721)
• use a larger chunk size when loading data (#799) (b804373)

Documentation

• add Samples section to CONTRIBUTING.rst (#785) (e587029)
• add sample to delete job metadata (#798) (be9b242)

2.22.0 (2021-07-19)

Features

• add LoadJobConfig.projection_fields to select DATASTORE_BACKUP fields (#736) (c45a738)
• add standard sql table type, update scalar type enums (#777) (b8b5433)
• add support for more detailed DML stats (#758) (36fe86f)
• add support for user defined Table View Functions (#724) (8c7b839)

Bug Fixes

• avoid possible job already exists error (#751) (45b9308)

Dependencies

• allow 2.x versions of google-api-core, google-cloud-core, google-resumable-media (#770) (87a09fa)

Documentation

• add loading data from Firestore backup sample (#737) (22fd848)

2.21.0 (2021-07-12)

Features

• Add max_results parameter to some of the QueryJob methods. (#698) (2a9618f)
• Add support for decimal target types. (#735) (7d2d3e9)
• Add support for table snapshots. (#740) (ba86b2a)
• Enable unsetting policy tags on schema fields. (#703) (18bb443)
• Make it easier to disable best-effort deduplication with streaming inserts. (#734) (1246da8)
• Support passing struct data to the DB API. (#718) (38b3ef9)

Bug Fixes

• Inserting non-finite floats with insert_rows(). (#728) (d047419)
• Use pandas function to check for NaN. (#750) (67bc5fb)

Documentation

• Add docs for all enums in module. (#745) (145944f)
• Omit mention of Python 2.7 in CONTRIBUTING.rst. (#706) (27d6839)

2.20.0 (2021-06-07)

Features

• support script options in query job config (#690) (1259e16)

2.19.0 (2021-06-06)

Features

• list_tables, list_projects, list_datasets, list_models, list_routines, and list_jobs now accept a page_size parameter to control page size (#686) (1f1c4b7)

2.18.0 (2021-06-02)

Features

• add support for Parquet options (#679) (d792ce0)

2.17.0 (2021-05-21)

Features

• detect obsolete BQ Storage extra at runtime (#666) (bd7dbda)
• Support parameterized NUMERIC, BIGNUMERIC, STRING, and BYTES types (#673) (45421e7)

Bug Fixes

• tests: invalid path to strptime() (#672) (591cdd8)

2.16.1 (2021-05-12)

Bug Fixes

• executemany rowcount only reflected the last execution (#660) (aeadc8c)

2.16.0 (2021-05-05)

Features

• add with_name() to ScalarQueryParameterType (#644) (6cc6876)

Dependencies

• expand supported pyarrow versions to v4 (#643) (9e1d386)

2.15.0 (2021-04-29)

Features

• Extended DB API parameter syntax to optionally provide parameter types (#626) (8bcf397)

Bug Fixes

• add DECIMAL and BIGDECIMAL as aliases for NUMERIC and BIGNUMERIC (#638) (aa59023)
• The DB API Binary function accepts bytes data (#630) (4396e70)

2.14.0 (2021-04-26)

Features

• accept DatasetListItem where DatasetReference is accepted (#597) (c8b5581)
• accept job object as argument to get_job and cancel_job (#617) (f75dcdf)
• add Client.delete_job_metadata method to remove job metadata (#610) (0abb566)
• add max_queue_size argument to RowIterator.to_dataframe_iterable (#575) (f95f415)
• add type hints for public methods (#613) (f8d4aaa)
• DB API cursors are now iterable (#618) (e0b373d)
• retry google.auth TransportError by default (#624) (34ecc3f)
• use pyarrow stream compression, if available (#593) (dde9dc5)

Bug Fixes

• consistent percents handling in DB API query (#619) (6502a60)
• missing license headers in new test files (#604) (df48cc5)
• unsetting clustering fields on Table is now possible (#622) (33a871f)

Documentation

• add sample to run DML query (#591) (ff2ec3a)
• update the description of the return value of _QueryResults.rows() (#594) (8f4c0b8)

2.13.1 (2021-03-23)

Bug Fixes

• add ConnectionError to default retry (#571) (a3edb8b)

2.13.0 (2021-03-22)

Features

• add ExternalConfig.connection_id property to connect to external sources (#560) (d93986e)

Bug Fixes

• avoid overly strict dependency on pyarrow 3.x (#564) (97ee6ec)
• avoid policy tags 403 error in load_table_from_dataframe (#557) (84e646e)

2.12.0 (2021-03-16)

Features

• make QueryJob.done() method more performant (#544) (a3ab9ef)

Bug Fixes

• remove DB-API dependency on pyarrow with decimal query parameters (#551) (1b946ba)

2.11.0 (2021-03-09)

Features

• add context manager support to client (#540) (d5c7e11)

2.10.0 (2021-02-25)

Features

• add BIGNUMERIC support (#527) (cc3394f)

Bug Fixes

• error using empty array of structs parameter (#474) (c1d15f4)
• QueryJob.exception() returns the errors, not raises them (#467) (d763279)

Documentation

• bigquery: Add alternative approach to setting credentials (#517) (60fbf28)
• explain retry behavior for DONE jobs (#532) (696c443)

2.9.0 (2021-02-18)

Features

• add determinism level for javascript UDFs (#522) (edd3328)
• expose reservation usage stats on jobs (#524) (4ffb4e0)

Documentation

• clarify `%%bigquery`` magics and fix broken link (#508) (eedf93b)
• update python contributing guide (#514) (01e851d)

2.8.0 (2021-02-08)

Features

• Add mTLS support to client. (#492) (1823cad)

Bug Fixes

• Don't try to close closed cursors. (#498) (bf44e7b)

2.7.0 (2021-01-27)

Bug Fixes

• invalid conversion of timezone-aware datetime values to JSON (#480) (61b4385)
• reading the labels attribute on Job instances (#471) (80944f0)
• use explicitly given project over the client's default project for load jobs (#482) (530e1e8)

Dependencies

• declare support for Python 3.9 (#488) (55daa7d)

2.6.2 (2021-01-11)

Bug Fixes

• add minimum timeout to getQueryResults API requests (#444) (015a73e)
• use debug logging level for OpenTelemetry message (#442) (7ea6b7c)

Documentation

• add GEOGRAPHY data type code samples (#428) (dbc68b3)
• fix Shapely import in GEOGRAPHY sample (#431) (96a1c5b)
• move and refresh view samples (#420) (079b6a1)

2.6.1 (2020-12-09)

Bug Fixes

• handle null values in array query parameters (#426) (78fde4a)

Documentation

• add examples of fields argument to update methods (#418) (8c7e02b)

2.6.0 (2020-12-07)

Features

• add support for materialized views (#408) (57ffc66), closes #407
• convert BIGNUMERIC values to decimal objects (#414) (d472d2d), closes #367
• support CSV format in load_table_from_dataframe pandas connector (#399) (0046742)

Bug Fixes

• preserve timestamp microsecond precision with rows from REST API (#402) (04510a7)

Documentation

• update intersphinx links (#404) (a9d8ae8)

2.5.0 (2020-12-02)

Features

• add TableReference.__str__ to get table ID in standard SQL (#405) (53dff2a), closes #354
• add progress bar for magics (#396) (04d0273)
• add support for unrecognized model types (#401) (168f035)

Bug Fixes

• avoid floating point for timestamp in insert_rows (#393) (a1949ae)

Performance Improvements

• don't fetch rows when waiting for query to finish (#400) (730df17), closes #374 #394

Documentation

• samples: add more clustering code snippets (#330) (809e4a2), closes #329

Dependencies

• update required version of opentelementry for opentelemetry-exporter-google-cloud (#398) (673a9cb)

2.4.0 (2020-11-16)

Features

• add progress bar to QueryJob.to_dataframe and to_arrow (#352) (dc78edd)
• allow routine references (#378) (f9480dc)

Bug Fixes

• dbapi: allow rows to be fetched from scripts (#387) (b899ad1), closes #377

Performance Improvements

• avoid extra API calls from to_dataframe if all rows are cached (#384) (c52b317)
• cache first page of jobs.getQueryResults rows (#374) (86f6a51)
• use getQueryResults from DB-API (#375) (30de15f)

Dependencies

• expand pyarrow dependencies to include version 2 (#368) (cd9febd)

2.3.1

11-05-2020 09:27 PST

Internal / Testing Changes

• update google.cloud.bigquery.__version__

2.3.0 (2020-11-04)

Features

• add reload argument to *Job.done() functions (#341) (e51fd45)
• pass retry from Job.result() to Job.done() (#41) (284e17a)

Bug Fixes

• add missing spaces in opentelemetry log message (#360) (4f326b1)
• dbapi: avoid running % format with no query parameters (#348) (5dd1a5e)
• create_job method accepts dictionary arguments (#300) (155bacc)

Performance Improvements

• use jobs.getQueryResults to download result sets (#363) (0c3476d)

Documentation

• add documents for QueryPlanEntry and QueryPlanEntryStep (#344) (dca2e4c)

2.2.0 (2020-10-19)

Features

• add method api_repr for table list item (#299) (07c70f0)
• add support for listing arima, automl, boosted tree, DNN, and matrix factorization models (#328) (502a092)
• add timeout paramter to load_table_from_file and it dependent methods (#327) (b0dd892)
• add to_api_repr method to Model (#326) (fb401bd)
• allow client options to be set in magics context (#322) (5178b55)

Bug Fixes

• make TimePartitioning repr evaluable (#110) (20f473b), closes #109
• use version.py instead of pkg_resources.get_distribution (#307) (b8f502b)

Performance Improvements

• add size parameter for load table from dataframe and json methods (#280) (3be78b7)

Documentation

• update clustering field docstrings (#286) (5ea1ece), closes #285
• update snippets samples to support version 2.0 (#309) (61634be)

Dependencies

• add protobuf dependency (#306) (cebb5e0), closes #305
• require pyarrow for pandas support (#314) (801e4c0), closes #265

2.1.0 (2020-10-08)

Features

• add constants for MONTH and YEAR time partitioning types (#283) (9090e1c)

Bug Fixes

• remove unnecessary dependency on libcst (#308) (c055930)

Performance Improvements

• remove redundant array deepcopy (#26) (b54f867)

Documentation

• samples: add create_table_clustered code snippet (#291) (d1eb8b3)

2.0.0

09-30-2020 14:51 PDT

Implementation Changes

• Transition the library to microgenerator. (#278) This is a breaking change that drops support for Python 2.7 and 3.5 and brings a few other changes. See migration guide for more info.

Internal / Testing Changes

• Update protoc-generated comments (via synth). (#270)
• Add CI secrets manager (via synth). (#271)

1.28.0 (2020-09-22)

Features

• add custom cell magic parser to handle complex --params values (#213) (dcfbac2)
• add instrumentation to list methods (#239) (fa9f9ca)
• add opentelemetry tracing (#215) (a04996c)
• expose require_partition_filter for hive_partition (#257) (aa1613c)

Bug Fixes

• fix dependency issue in fastavro (#241) (2874abf)
• update minimum dependency versions (#263) (1be66ce)
• validate job_config.source_format in load_table_from_dataframe (#262) (6160fee)

Documentation

• recommend insert_rows_json to avoid call to tables.get (#258) (ae647eb)

1.27.2 (2020-08-18)

Bug Fixes

• rationalize platform constraints for 'pyarrow' extra (#235) (c9a0567)

1.27.1 (2020-08-18)

Bug Fixes

• tweak pyarrow extra to soothe PyPI (#230) (c15efbd)

1.27.0 (2020-08-15)

Features

• add support and tests for struct fields (#146) (fee2ba8)
• add support for getting and setting table IAM policy (#144) (f59fc9a)
• bigquery: add client_options to base class (#216) (478597a)

Bug Fixes

• converting to dataframe with out of bounds timestamps (#209) (8209203), closes #168
• raise error if inserting rows with unknown fields (#163) (8fe7254)

1.26.1 (2020-07-25)

Documentation

• Migrated code samples from https://github.com/GoogleCloudPlatform/python-docs-samples

Bug Fixes

• RowIterator.to_arrow() error when BQ Storage client cannot be created (#181) (7afa3d7)

Dependencies

• Updated version constraints on grmp dependency in anticipation of 1.0.0 release (#189)

1.26.0 (2020-07-20)

Features

• use BigQuery Storage client by default (if dependencies available) (#55) (e75ff82), closes #91
• bigquery: add eq method for class PartitionRange and RangePartitioning (#162) (0d2a88d)
• bigquery: expose date_as_object parameter to users (#150) (a2d5ce9)
• bigquery: expose date_as_object parameter to users (#150) (cbd831e)

Bug Fixes

• dry run queries with DB API cursor (#128) (bc33a67)
• omit NaN values when uploading from insert_rows_from_dataframe (#170) (f9f2f45)

Documentation

• bigquery: add client thread-safety documentation (#132) (fce76b3)
• bigquery: add docstring for conflict exception (#171) (9c3409b)
• bigquery: consistent use of optional keyword (#153) (79d8c61)
• bigquery: fix the broken docs (#139) (3235255)

1.25.0 (2020-06-06)

Features

• add BigQuery storage client support to DB API (#36) (ba9b2f8)
• bigquery: add create job method (#32) (2abdef8)
• bigquery: add support of model for extract job (#71) (4a7a514)
• add HOUR support for time partitioning interval (#91) (0dd90b9)
• add support for policy tags (#77) (38a5c01)
• make AccessEntry objects hashable (#93) (23a173b)
• bigquery: expose start index parameter for query result (#121) (be86de3)
• bigquery: unit and system test for dataframe with int column with Nan values (#39) (5fd840e)

Bug Fixes

• allow partial streaming_buffer statistics (#37) (645f0fd)
• distinguish server timeouts from transport timeouts (#43) (a17be5f)
• improve cell magic error message on missing query (#58) (6182cf4)
• bigquery: fix repr of model reference (#66) (26c6204)
• bigquery: fix start index with page size for list rows (#27) (400673b)

1.24.0

02-03-2020 01:38 PST

Implementation Changes

• Fix inserting missing repeated fields. (#10196)
• Deprecate client.dataset() in favor of DatasetReference. (#7753)
• Use faster to_arrow + to_pandas in to_dataframe() when pyarrow is available. (#10027)
• Write pandas datetime[ns] columns to BigQuery TIMESTAMP columns. (#10028)

New Features

• Check rows argument type in insert_rows(). (#10174)
• Check json_rows arg type in insert_rows_json(). (#10162)
• Make RowIterator.to_dataframe_iterable() method public. (#10017)
• Add retry parameter to public methods where missing. (#10026)
• Add timeout parameter to Client and Job public methods. (#10002)
• Add timeout parameter to QueryJob.done() method. (#9875)
• Add create_bqstorage_client parameter to to_dataframe() and to_arrow() methods. (#9573)

Dependencies

• Fix minimum versions of google-cloud-core and google-resumable-media dependencies. (#10016)

Documentation

• Fix a comment typo in job.py. (#10209)
• Update code samples of load table file and load table URI. (#10175)
• Uncomment Client constructor and imports in samples. (#10058)
• Remove unused query code sample. (#10024)
• Update code samples to use strings for table and dataset IDs. (#9974)

Internal / Testing Changes

• Bump copyright year to 2020, tweak docstring formatting (via synth). #10225
• Add tests for concatenating categorical columns. (#10180)
• Adjust test assertions to the new default timeout. (#10222)
• Use Python 3.6 for the nox blacken session (via synth). (#10012)

1.23.1

12-16-2019 09:39 PST

Implementation Changes

• Add iamMember entity type to allowed access classes. (#9973)
• Fix typo in import error message (pandas -> pyarrow). (#9955)

Dependencies

• Add six as an explicit dependency. (#9979)

Documentation

• Add sample to read from query destination table. (#9964)

1.23.0

12-11-2019 13:31 PST

New Features

• Add close() method to client for releasing open sockets. (#9894)
• Add support of use_avro_logical_types for extract jobs. (#9642)
• Add support for hive partitioning options configuration. (#9626)
• Add description for routine entities. (#9785)

Documentation

• Update code samples to use strings for table and dataset IDs. (#9495)

Internal / Testing Changes

• Run unit tests with Python 3.8. (#9880)
• Import Mapping from collections.abc not from collections. (#9826)

1.22.0

11-13-2019 12:23 PST

Implementation Changes

• Preserve job config passed to Client methods. (#9735)
• Use pyarrow fallback for improved schema detection. (#9321)
• Add TypeError if wrong job_config type is passed to client job methods. (#9506)
• Fix arrow deprecation warning. (#9504)

New Features

• Add --destination_table parameter to IPython magic. (#9599)
• Allow passing schema as a sequence of dicts. (#9550)
• Implement defaultEncryptionConfiguration on datasets. (#9489)
• Add range partitioning to tables, load jobs, and query jobs. (#9477)

Dependencies

• Pin google-resumable-media to includ 0.5.x. (#9572)

Documentation

• Fix link anchors in external config docstrings. (#9627)
• Add python 2 sunset banner to documentation. (#9036)
• Add table create sample using integer range partitioning. (#9478)
• Document how to achieve higher write limit and add tests. (#9574)
• Add code sample for scripting. (#9537)
• Rewrite docs in Google style, part 2. (#9481)
• Use multi-regional key path for CMEK in snippets. (#9523)

Internal / Testing Changes

• Fix undelete table system test to use milliseconds in snapshot decorator. (#9649)
• Format code with latest version of black. (#9556)
• Remove duplicate test dependencies. (#9503)

1.21.0

10-16-2019 10:33 PDT

New Features

• add ability to pass in a table ID instead of a query to the %%bigquery magic (#9170)
• add support for custom QueryJobConfig in BigQuery.cursor.execute method (#9278)
• store QueryJob to destination var on error in %%bigquery magic (#9245)
• add script statistics to job resource (#9428)
• add support for sheets ranges (#9416)
• add support for listing jobs by parent job (#9225)
• expose customer managed encryption key for ML models (#9302)
• add Dataset.default_partition_expiration_ms and Table.require_partition_filter properties (#9464)

Dependencies

• restrict version range of google-resumable-media (#9243)

Documentation

• document how to load data as JSON string (#9231)
• standardize comments and formatting in existing code samples (#9212)
• rewrite docstrings in Google style (#9326)
• fix incorrect links to REST API in reference docs (#9436)

Internal / Testing Changes

• add code samples to lint check (#9277)
• update code samples to use strings for table and dataset IDs (#9136)
• simplify scripting system test to reduce flakiness (#9458)

1.20.0

09-13-2019 11:22 PDT

Implementation Changes

• Change default endpoint to bigquery.googleapis.com (#9213)
• Change the default value of Cursor instances' arraysize attribute to None (#9199)
• Deprecate automatic schema conversion. (#9176)
• Fix list_rows() max results with BQ storage client (#9178)

New Features

• Add Model.encryption_config. (via synth) (#9214)
• Add Client.insert_rows_from_dataframe() method (#9162)
• Add support for array parameters to Cursor.execute(). (#9189)
• Add support for project IDs with org prefix to Table.from_string() factory. (#9161)
• Add --max_results option to Jupyter magics (#9169)
• Autofetch table schema on load if not provided. (#9108)
• Add max_results parameter to QueryJob.result(). (#9167)

Documentation

• Fix doc link. (#9200)

Internal / Testing Changes

• Revert "Disable failing snippets test (#9156)." (#9220)

1.19.0

09-03-2019 14:33 PDT

Implementation Changes

• Raise when unexpected fields are present in the LoadJobConfig.schema when calling load_table_from_dataframe. (#9096)
• Determine the schema in load_table_from_dataframe based on dtypes. (#9049)
• Raise helpful error when loading table from dataframe with STRUCT columns. (#9053)
• Fix schema recognition of struct field types. (#9001)
• Fix deserializing None in QueryJob for queries with parameters. (#9029)

New Features

• Include indexes in table written by load_table_from_dataframe, only if fields corresponding to indexes are present in LoadJobConfig.schema. (#9084)
• Add client_options to constructor. (#8999)
• Add --dry_run option to %%bigquery magic. (#9067)
• Add load_table_from_json() method to create a table from a list of dictionaries. (#9076)
• Allow subset of schema to be passed into load_table_from_dataframe. (#9064)
• Add support for unsetting LoadJobConfig.schema. (#9077)
• Add support to Dataset for project IDs containing an org prefix. (#8877)
• Add enum with SQL type names allowed to be used in SchemaField. (#9040)

Documentation

• Fix the reference URL for Client.create_dataset(). (#9149)
• Update code samples to use strings for table names instead of client.dataset(). (#9032)
• Remove compatability badges from READMEs. (#9035)
• Fix Pandas DataFrame load example under Python 2.7. (#9022)

Internal / Testing Changes

• Disable failing snippets test for copying CMEK-protected tables. (#9156)
• Fix BigQuery client unit test assertions (#9112)
• Replace avro with arrow schemas in test_table.py (#9056)

1.18.0

08-08-2019 12:28 PDT

New Features

• Add bqstorage_client param to QueryJob.to_arrow() (#8693)
• Include SQL query and job ID in exception messages. (#8748)
• Allow using TableListItem to construct a Table object. (#8738)
• Add StandardSqlDataTypes enum to BigQuery (#8782)
• Add to_standard_sql() method to SchemaField (#8880)
• Add debug logging statements to track when BQ Storage API is used. (#8838)
• Hide error traceback in BigQuery cell magic (#8808)
• Allow choice of compression when loading from dataframe (#8938)
• Additional clustering metrics for BQML K-means models (via synth). (#8945)

Documentation

• Add compatibility check badges to READMEs. (#8288)
• Link to googleapis.dev documentation in READMEs. (#8705)
• Remove redundant service account key code sample. (#8891)

Internal / Testing Changes

• Fix several pytest "skip if" markers (#8694)
• Update tests to support conversion of NaN as NULL in pyarrow 0.14.*. (#8785)
• Mock external calls in one of BigQuery unit tests (#8727)
• Set IPython user agent when running queries with IPython cell magic (#8713)
• Use configurable bucket name for GCS samples data in systems tests. (#8783)
• Move maybe_fail_import() to top level test utils (#8840)
• Set BQ Storage client user-agent when in Jupyter cell (#8734)

1.17.0

07-12-2019 07:56 PDT

New Features

• Support faster Arrow data format in to_dataframe when using BigQuery Storage API. (#8551)
• Add to_arrow to get a pyarrow.Table from query results. (#8609)

Dependencies

• Exclude bad 0.14.0 pyarrow release. (#8551)

1.16.0

07-01-2019 10:22 PDT

New Features

• Add Routines API. (#8491)
• Add more stats to Models API, such as optimization_strategy (via synth). (#8344)

Documentation

• Add docs job to publish to googleapis.dev. (#8464)
• Add sample demonstrating how to create a job. (#8422)
• Use autodetected location in code samples. (#8340, #8341)

Internal / Testing Changes

• Refactor to_dataframe to deterministicly update progress bar. (#8303)

1.15.0

06-14-2019 10:10 PDT

Implementation Changes

• Fix bug where load_table_from_dataframe could not append to REQUIRED fields. (#8230)

New Features

• Add page_size parameter to QueryJob.result. (#8206)

1.14.0

06-04-2019 11:11 PDT

New Features

• Add maximum_bytes_billed argument and context.default_query_job_config property to magics. (#8179)

Dependencies

• Don't pin google-api-core in libs using google-cloud-core. (#8213)

1.13.0

05-31-2019 10:22 PDT

New Features

• Use job_config.schema for data type conversion if specified in load_table_from_dataframe. (#8105)

Internal / Testing Changes

• Adds private _connection object to magics context. (#8192)
• Fix coverage in 'types.py' (via synth). (#8146)

1.12.1

05-21-2019 11:16 PDT

Implementation Changes

• Don't raise error when encountering unknown fields in Models API. (#8083)

Documentation

• Use alabaster theme everwhere. (#8021)

Internal / Testing Changes

• Add empty lines (via synth). (#8049)

1.12.0

05-16-2019 11:25 PDT

Implementation Changes

• Remove duplicates from index on pandas DataFrames returned by to_dataframe(). (#7953)
• Prevent error when time partitioning is populated with empty dict (#7904)
• Preserve order in to_dataframe with BQ Storage from queries containing ORDER BY (#7793)
• Respect progress_bar_type in to_dataframe when used with BQ Storage API (#7697)
• Refactor QueryJob.query to read from resource dictionary (#7763)
• Close the to_dataframe progress bar when finished. (#7757)
• Ensure that KeyboardInterrupt during to_dataframeno longer hangs. (#7698)
• Raise ValueError when BQ Storage is required but missing (#7726)
• Make total_rows available on RowIterator before iteration (#7622)
• Avoid masking auth errors in to_dataframe with BQ Storage API (#7674)

New Features

• Add support for passing client_info. (#7849 and (#7806)
• Phase 1 for storing schemas for later use. (#7761)
• Add destination and related properties to LoadJob. (#7710)
• Add clustering_fields property to TableListItem (#7692)
• Add created and expires properties to TableListItem (#7684)

Dependencies

• Pin google-cloud-core >= 1.0.0, < 2.0dev. (#7993)
• Add [all] extras to install all extra dependencies (#7610)

Documentation

• Move table and dataset snippets to samples/ directory (#7683)

Internal / Testing Changes

• Blacken unit tests. (#7960)
• Cleanup client tests with method to create minimal table resource (#7802)

1.11.2

04-05-2019 08:16 PDT

Dependencies

• Add dependency on protobuf. (#7668)

1.11.1

04-04-2019 09:19 PDT

Internal / Testing Changes

• Increment version number in setup.py.

1.11.0

04-03-2019 19:33 PDT

Implementation Changes

• Remove classifier for Python 3.4 for end-of-life. (#7535)

New Features

• Enable fastparquet support by using temporary file in load_table_from_dataframe (#7545)
• Allow string for copy sources, query destination, and default dataset (#7560)
• Add progress_bar_type argument to to_dataframe to use tqdm to display a progress bar (#7552)
• Call get_table in list_rows if the schema is not available (#7621)
• Fallback to BQ API when there are problems reading from BQ Storage. (#7633)
• Add methods for Models API (#7562)
• Add option to use BigQuery Storage API from IPython magics (#7640)

Documentation

• Remove typo in Table.from_api_repr docstring. (#7509)
• Add docs session to nox configuration for BigQuery (#7541)

Internal / Testing Changes

• Refactor table() methods into shared implementation. (#7516)
• Blacken noxfile and setup file in nox session (#7619)
• Actually use the progress_bar_type argument in QueryJob.to_dataframe(). (#7616)

1.10.0

03-06-2019 15:20 PST

Implementation Changes

• Harden 'ArrayQueryParameter.from_api_repr' against missing 'parameterValue'. (#7311)
• Allow nested records w/ null values. (#7297)

New Features

• Add exists_ok and not_found_ok options to ignore errors when creating/deleting datasets/tables. (#7491)
• Accept a string in Table and Dataset constructors. (#7483)

Documentation

• Update docstring of RowIterator's to_dataframe (#7306)
• Updated client library documentation URLs. (#7307)

Internal / Testing Changes

• Fix lint. (#7383)

1.9.0

02-04-2019 13:28 PST

New Features

• Add arguments to select dtypes and use BQ Storage API to QueryJob.to_dataframe(). (#7241)

Documentation

• Add sample for fetching total_rows from query results. (#7217)

1.8.1

12-17-2018 17:53 PST

Documentation

• Document Python 2 deprecation (#6910)
• Normalize docs for 'page_size' / 'max_results' / 'page_token' (#6842)

1.8.0

12-10-2018 12:39 PST

Implementation Changes

• Add option to use BQ Storage API with to_dataframe (#6854)
• Fix exception type in comment (#6847)
• Add to_bqstorage to convert from Table[Reference] google-cloud-bigquery-storage reference (#6840)
• Import iam.policy from google.api_core. (#6741)
• Add avro logical type control for load jobs. (#6827)
• Allow setting partition expiration to 'None'. (#6823)
• Add retry argument to _AsyncJob.result. (#6302)

Dependencies

• Update dependency to google-cloud-core (#6835)

Documentation

• Add avro load samples (#6832)

Internal / Testing Changes

• Blacken libraries (#6794)
• Fix copy/paste typos in noxfile comments (#6831)

1.7.0

11-05-2018 16:41 PST

Implementation Changes

• Add destination table properties to LoadJobConfig. (#6202)
• Allow strings or references in create_dataset and create_table (#6199)
• Fix swallowed error message (#6168)

New Features

• Add --params option to %%bigquery magic (#6277)
• Expose to_api_repr method for jobs. (#6176)
• Allow string in addition to DatasetReference / TableReference in Client methods. (#6164)
• Add keyword arguments to job config constructors for setting properties (#6397)

Documentation

• Update README service links in quickstart guides. (#6322)
• Move usage guides to their own docs. (#6238)
• Normalize use of support level badges (#6159)

Internal / Testing Changes

• Deprecation cleanups (#6304)
• Use _get_sub_prop helper so missing load stats don't raise. (#6269)
• Use new Nox (#6175)
• Harden snippets against transient GCS errors. (#6184)

1.6.0

New Features

• Add support for GEOGRAPHY type (#6147)
• Add default QueryJobConfig to Client (#6088)

Documentation

• Remove unused "append" samples (#6100)

Internal / Testing Changes

• Address dataset leaks, conflicts in systests (#6099)
• Harden bucket teardown against 429 Too Many Requests. (#6101)

1.5.1

Implementation Changes

• Retry '502 Bad Gateway' errors by default. (#5930)
• Avoid pulling entire result set into memory when constructing dataframe. (#5870)
• Add support for retrying unstructured 429 / 500 / 502 responses. (#6011)
• Populate the jobReference from the API response. (#6044)

Documentation

• Prepare documentation for repo split (#5955)
• Fix leakage of bigquery/spanner sections into sidebar menu. (#5986)

Internal / Testing Changes

• Test pandas support under Python 3.7. (#5857)
• Nox: use inplace installs (#5865)
• Update system test to use test data in bigquery-public-data. (#5965)

1.5.0

Implementation Changes

• Make 'Table.location' read-only. (#5687)

New Features

• Add 'clustering_fields' properties. (#5630)
• Add support for job labels (#5654)
• Add 'QueryJob.estimated_bytes_processed' property (#5655)
• Add support/tests for loading tables from 'gzip.GzipFile'. (#5711)
• Add 'ExternalSourceFormat' enum. (#5674)
• Add default location to client (#5678)

Documentation

• Fix typo in CopyJob sources docstring (#5690)

Internal / Testing Changes

• Add/refactor snippets for managing BigQuery jobs (#5631)
• Reenable systests for 'dataset.update'/'table.update'. (#5732)

1.4.0

Implementation Changes

• Add 'internalError' to retryable error reasons. (#5599)
• Don't raise exception if viewing CREATE VIEW DDL results (#5602)

New Features

• Add Orc source format support and samples (#5500)
• Move 'DEFAULT_RETRY' (w/ its predicate) to a new public 'retry' module. (#5552)
• Allow listing rows on an empty table. (#5584)

Documentation

• Add load_table_from_dataframe() to usage docs and changelog and dedents snippets in usage page (#5501)
• Add samples for query external data sources (GCS & Sheets) (#5491)
• Add BigQuery authorized view samples (#5515)
• Update docs to show pyarrow as the only dependency of load_table_from_dataframe() (#5582)

Internal / Testing Changes

• Add missing explict coverage for '_helpers' (#5550)
• Skip update_table and update_dataset tests until etag issue is resolved. (#5590)

1.3.0

New Features

• NUMERIC type support (#5331)
• Add timeline and top-level slot-millis to query statistics. (#5312)
• Add additional statistics to query plan stages. (#5307)
• Add client.load_table_from_dataframe() (#5387)

Documentation

• Use autosummary to split up API reference docs (#5340)
• Fix typo in Client docstrings (#5342)

Internal / Testing Changes

• Prune systests identified as reduntant to snippets. (#5365)
• Modify system tests to use prerelease versions of grpcio (#5304)
• Improve system test performance (#5319)

1.2.0

Implementation Changes

• Switch list_partitions helper to a direct metatable read (#5273)
• Fix typo in Encoding.ISO_8859_1 enum value (#5211)

New Features

• Add UnknownJob type for redacted jobs. (#5281)
• Add project parameter to list_datasets and list_jobs (#5217)
• Add from_string factory methods to Dataset and Table (#5255)
• Add column based time partitioning (#5267)

Documentation

• Standardize docstrings for constants (#5289)
• Fix docstring / impl of ExtractJob.destination_uri_file_counts. (#5245)

Internal / Testing Changes

• Add testing support for Python 3.7; remove testing support for Python 3.4. (#5295)

1.1.0

New Features

• Add client.get_service_account_email (#5203)

Documentation

• Update samples and standardize region tags (#5195)

Internal / Testing Changes

• Fix trove classifier to be Production/Stable
• Don't suppress 'dots' output on test (#5202)

1.0.0

Implementation Changes

• Remove deprecated Client methods (#5182)

0.32.0

⚠️ Interface changes

• Use job.configuration resource for XXXJobConfig classes (#5036)

Interface additions

• Add page_size parameter for list_rows and use in DB-API for arraysize (#4931)
• Add IPython magics for running queries (#4983)

Documentation

• Add job string constant parameters in init and snippets documentation (#4987)

Internal / Testing changes

• Specify IPython version 5.5 when running Python 2.7 tests (#5145)
• Move all Dataset property conversion logic into properties (#5130)
• Remove unnecessary _Table class from test_job.py (#5126)
• Use explicit bytes to initialize 'BytesIO'. (#5116)
• Make SchemaField be able to include description via from_api_repr method (#5114)
• Remove _ApiResourceProperty class (#5107)
• Add dev version for 0.32.0 release (#5105)
• StringIO to BytesIO (#5101)
• Shorten snippets test name (#5091)
• Don't use selected_fields for listing query result rows (#5072)
• Add location property to job classes. (#5071)
• Use autospec for Connection in tests. (#5066)
• Add Parquet SourceFormat and samples (#5057)
• Remove test_load_table_from_uri_w_autodetect_schema_then_get_job because of duplicate test in snippets (#5004)
• Fix encoding variable and strings UTF-8 and ISO-8859-1 difference documentation (#4990)

0.31.0

Interface additions

• Add support for EncryptionConfiguration (#4845)

Implementation changes

• Allow listing/getting jobs even when there is an "invalid" job. (#4786)

Dependencies

• The minimum version for google-api-core has been updated to version 1.0.0. This may cause some incompatibility with older google-cloud libraries, you will need to update those libraries if you have a dependency conflict. (#4944, #4946)

Documentation

• Update format in Table.full_table_id and TableListItem.full_table_id docstrings. (#4906)

Testing and internal changes

• Install local dependencies when running lint (#4936)
• Re-enable lint for tests, remove usage of pylint (#4921)
• Normalize all setup.py files (#4909)
• Remove unnecessary debug print from tests (#4907)
• Use constant strings for job properties in tests (#4833)

0.30.0

This is the release candidate for v1.0.0.

Interface changes / additions

• Add delete_contents to delete_dataset. (#4724)

Bugfixes

• Add handling of missing properties in SchemaField.from_api_repr(). (#4754)
• Fix missing return value in LoadJobConfig.from_api_repr. (#4727)

Documentation

• Minor documentation and typo fixes. (#4782, #4718, #4784, #4835, #4836)

0.29.0

Interface changes / additions

• Add to_dataframe() method to row iterators. When Pandas is installed this method returns a DataFrame containing the query's or table's rows. (#4354)
• Iterate over a QueryJob to wait for and get the query results. (#4350)
• Add Table.reference and Dataset.reference properties to get the TableReference or DatasetReference corresponding to that Table or Dataset, respectively. (#4405)
• Add Row.keys(), Row.items(), and Row.get(). This makes Row act more like a built-in dictionary. (#4393, #4413)

Interface changes / breaking changes

• Add Client.insert_rows() and Client.insert_rows_json(), deprecate Client.create_rows() and Client.create_rows_json(). (#4657)
• Add Client.list_tables, deprecate Client.list_dataset_tables. (#4653)
• Client.list_tables returns an iterators of TableListItem. The API only returns a subset of properties of a table when listing. (#4427)
• Remove QueryJob.query_results(). Use QueryJob.result() instead. (#4652)
• Remove Client.query_rows(). Use Client.query() instead. (#4429)
• Client.list_datasets returns an iterator of DatasetListItem. The API only returns a subset of properties of a dataset when listing. (#4439)

0.28.0

0.28.0 significantly changes the interface for this package. For examples of the differences between 0.28.0 and previous versions, see Migrating to the BigQuery Python client library 0.28. These changes can be summarized as follows:

• Query and view operations default to the standard SQL dialect. (#4192)
• Client functions related to jobs, like running queries, immediately start the job.
• Functions to create, get, update, delete datasets and tables moved to the client class.

Fixes

• Populate timeout parameter correctly for queries (#4209)
• Automatically retry idempotent RPCs (#4148, #4178)
• Parse timestamps in query parameters using canonical format (#3945)
• Parse array parameters that contain a struct type. (#4040)
• Support Sub Second Datetimes in row data (#3901, #3915, #3926), h/t @page1

Interface changes / additions

• Support external table configuration (#4182) in query jobs (#4191) and tables (#4193).
• New Row class allows for access by integer index like a tuple, string index like a dictionary, or attribute access like an object. (#4149)
• Add option for job ID generation with user-supplied prefix (#4198)
• Add support for update of dataset access entries (#4197)
• Add support for atomic read-modify-write of a dataset using etag (#4052)
• Add support for labels to Dataset (#4026)
• Add support for labels to Table (#4207)
• Add Table.streaming_buffer property (#4161)
• Add TableReference class (#3942)
• Add DatasetReference class (#3938, #3942, #3993)
• Add ExtractJob.destination_uri_file_counts property. (#3803)
• Add client.create_rows_json() to bypass conversions on streaming writes. (#4189)
• Add client.get_job() to get arbitrary jobs. (#3804, #4213)
• Add filter to client.list_datasets() (#4205)
• Add QueryJob.undeclared_query_parameters property. (#3802)
• Add QueryJob.referenced_tables property. (#3801)
• Add new scalar statistics properties to QueryJob (#3800)
• Add QueryJob.query_plan property. (#3799)

Interface changes / breaking changes

• Remove client.run_async_query(), use client.query() instead. (#4130)
• Remove client.run_sync_query(), use client.query_rows() instead. (#4065, #4248)
• Make QueryResults read-only. (#4094, #4144)
• Make get_query_results private. Return rows for QueryJob.result() (#3883)
• Move *QueryParameter and UDFResource classes to query module (also exposed in bigquery module). (#4156)

Changes to tables

• Remove client from Table class (#4159)
• Remove table.exists() (#4145)
• Move table.list_parations to client.list_partitions (#4146)
• Move table.upload_from_file to client.load_table_from_file (#4136)
• Move table.update() and table.patch() to client.update_table() (#4076)
• Move table.insert_data() to client.create_rows(). Automatically generates row IDs if not supplied. (#4151, #4173)
• Move table.fetch_data() to client.list_rows() (#4119, #4143)
• Move table.delete() to client.delete_table() (#4066)
• Move table.create() to client.create_table() (#4038, #4043)
• Move table.reload() to client.get_table() (#4004)
• Rename Table.name attribute to Table.table_id (#3959)
• Table constructor takes a TableReference as parameter (#3997)

Changes to datasets

• Remove client from Dataset class (#4018)
• Remove dataset.exists() (#3996)
• Move dataset.list_tables() to client.list_dataset_tables() (#4013)
• Move dataset.delete() to client.delete_dataset() (#4012)
• Move dataset.patch() and dataset.update() to client.update_dataset() (#4003)
• Move dataset.create() to client.create_dataset() (#3982)
• Move dataset.reload() to client.get_dataset() (#3973)
• Rename Dataset.name attribute to Dataset.dataset_id (#3955)
• client.dataset() returns a DatasetReference instead of Dataset. (#3944)
• Rename class: dataset.AccessGrant -> dataset.AccessEntry. (#3798)
• dataset.table() returns a TableReference instead of a Table (#4014)
• Dataset constructor takes a DatasetReference (#4036)

Changes to jobs

• Make job.begin() method private. (#4242)
• Add LoadJobConfig class and modify LoadJob (#4103, #4137)
• Add CopyJobConfig class and modify CopyJob (#4051, #4059)
• Type of Job's and Query's default_dataset changed from Dataset to DatasetReference (#4037)
• Rename client.load_table_from_storage() to client.load_table_from_uri() (#4235)
• Rename client.extract_table_to_storage to client.extract_table(). Method starts the extract job immediately. (#3991, #4177)
• Rename XJob.name to XJob.job_id. (#3962)
• Rename job classes. LoadTableFromStorageJob -> LoadJob and ExtractTableToStorageJob -> jobs.ExtractJob (#3797)

Dependencies

• Updating to google-cloud-core ~= 0.28, in particular, the google-api-core package has been moved out of google-cloud-core. (#4221)

PyPI: https://pypi.org/project/google-cloud-bigquery/0.28.0/

0.27.0

• Remove client-side enum validation. (#3735)
• Add Table.row_from_mapping helper. (#3425)
• Move google.cloud.future to google.api.core (#3764)
• Fix __eq__ and __ne__. (#3765)
• Move google.cloud.iterator to google.api.core.page_iterator (#3770)
• nullMarker support for BigQuery Load Jobs (#3777), h/t @leondealmeida
• Allow job_id to be explicitly specified in DB-API. (#3779)
• Add support for a custom null marker. (#3776)
• Add SchemaField serialization and deserialization. (#3786)
• Add get_query_results method to the client. (#3838)
• Poll for query completion via getQueryResults method. (#3844)
• Allow fetching more than the first page when max_results is set. (#3845)

PyPI: https://pypi.org/project/google-cloud-bigquery/0.27.0/

0.26.0

Notable implementation changes

• Using the requests transport attached to a Client for for resumable media (i.e. downloads and uploads) (#3705) (this relates to the httplib2 to requests switch)

Interface changes / additions

• Adding autodetect property on LoadTableFromStorageJob to enable schema autodetection. (#3648)
• Implementing the Python Futures interface for Jobs. Call job.result() to wait for jobs to complete instead of polling manually on the job status. (#3626)
• Adding is_nullable property on SchemaField. Can be used to check if a column is nullable. (#3620)
• job_name argument added to Table.upload_from_file for setting the job ID. (#3605)
• Adding google.cloud.bigquery.dbapi package, which implements PEP-249 DB-API specification. (#2921)
• Adding Table.view_use_legacy_sql property. Can be used to create views with legacy or standard SQL. (#3514)

Interface changes / breaking changes

• Removing results() method from the QueryJob class. Use query_results() instead. (#3661)
• SchemaField is now immutable. It is also hashable so that it can be used in sets. (#3601)

Dependencies

• Updating to google-cloud-core ~= 0.26, in particular, the underlying HTTP transport switched from httplib2 to requests (#3654, #3674)
• Adding dependency on google-resumable-media for loading BigQuery tables from local files. (#3555)

Packaging

• Fix inclusion of tests (vs. unit_tests) in MANIFEST.in (#3552)
• Updating author_email in setup.py to googleapis-publisher@google.com. (#3598)

PyPI: https://pypi.org/project/google-cloud-bigquery/0.26.0/