Python 3.10 support

[rkashapov]

Hello!

It would be great to get python 3.10 supported by this excellent package.

Thanks,
Rustam

[tswast]

Thanks for the request! We're working on making sure our test infrastructure and dependencies support Python 3.10 first.

[tswast]

Now that pandas (https://pypi.org/project/pandas/1.3.4/#files) and pyarrow (https://pypi.org/project/pyarrow/6.0.0/#files) have made releases that support Python 3.10, I believe this is unblocked.

[tswast]

Looks like we're missing a geopandas dependency named fiona, but since that's very much an optional dependency, we should be able to support Python 3.10 with some adjustments to our tests to omit geopandas.

[tswast]

Locally, there's one unit test failure I need to investigate. TBD re: system tests

Edit: Unit test failure went away once I installed the google-cloud-bigquery-storage package.

``` (dev-3.10) ➜ python-bigquery git:(issue1006-python310) pytest tests/unit ====================================================== test session starts ====================================================== platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0 rootdir: /Users/swast/src/github.com/googleapis/python-bigquery collected 1889 items / 1 skipped / 1888 selected

tests/unit/test__helpers.py sssssss...................................................................................... [ 4%]
................................................................... [ 8%]
tests/unit/test__http.py ...... [ 8%]
tests/unit/test__pandas_helpers.py ........................................................................sss........... [ 13%]
...............s...sssss........ [ 15%]
tests/unit/test_client.py .........................................................................sss.s................. [ 20%]
......................................................................s.................................................. [ 26%]
.................... [ 27%]
tests/unit/test_create_dataset.py ............. [ 28%]
tests/unit/test_dataset.py ................................................................................ [ 32%]
tests/unit/test_dbapi__helpers.py ................................................................................ [ 36%]
tests/unit/test_dbapi_connection.py s..s.ss.s.... [ 37%]
tests/unit/test_dbapi_cursor.py ........................ssss............................... [ 40%]
tests/unit/test_dbapi_types.py ........... [ 41%]
tests/unit/test_delete_dataset.py ........... [ 41%]
tests/unit/test_encryption_configuration.py .............. [ 42%]
tests/unit/test_external_config.py ........................................... [ 44%]
tests/unit/test_format_options.py ...... [ 44%]
tests/unit/test_job_retry.py ......... [ 45%]
tests/unit/test_list_datasets.py .... [ 45%]
tests/unit/test_list_jobs.py ........ [ 46%]
tests/unit/test_list_models.py .......... [ 46%]
tests/unit/test_list_projects.py .... [ 46%]
tests/unit/test_list_routines.py .......... [ 47%]
tests/unit/test_list_tables.py .............. [ 48%]
tests/unit/test_opentelemetry_tracing.py sssssss [ 48%]
tests/unit/test_query.py ................................................................................................ [ 53%]
........................... [ 55%]
tests/unit/test_retry.py ................... [ 56%]
tests/unit/test_schema.py ............................................................................ [ 60%]
tests/unit/test_signature_compatibility.py .. [ 60%]
tests/unit/test_table.py ................................................................................................ [ 65%]
.....................s.......s.......s.Fssssss......s...s..ss......s..ssssssssssss..sss.ss............................... [ 71%]
............ssss [ 72%]
tests/unit/enums/test_standard_sql_data_types.py ..s [ 72%]
tests/unit/helpers/test_from_json.py ................................................ [ 75%]
tests/unit/job/test_base.py ............................................................................... [ 79%]
tests/unit/job/test_copy.py .................... [ 80%]
tests/unit/job/test_extract.py ................ [ 81%]
tests/unit/job/test_load.py ............................. [ 82%]
tests/unit/job/test_load_config.py ...................................................................................... [ 87%]
....... [ 87%]
tests/unit/job/test_query.py ........................................................... [ 90%]
tests/unit/job/test_query_config.py .......................... [ 92%]
tests/unit/job/test_query_pandas.py ...........sssssss..sss..ss..sssssss [ 94%]
tests/unit/job/test_query_stats.py ......................... [ 95%]
tests/unit/line_arg_parser/test_lexer.py . [ 95%]
tests/unit/line_arg_parser/test_parser.py ........... [ 96%]
tests/unit/line_arg_parser/test_visitors.py . [ 96%]
tests/unit/model/test_model.py .................... [ 97%]
tests/unit/model/test_model_reference.py ........... [ 97%]
tests/unit/routine/test_routine.py ......................... [ 99%]
tests/unit/routine/test_routine_argument.py ...... [ 99%]
tests/unit/routine/test_routine_reference.py ........... [100%]

=========================================================== FAILURES ============================================================
______________________________ TestRowIterator.test_to_arrow_ensure_bqstorage_client_wo_bqstorage _______________________________

self = <tests.unit.test_table.TestRowIterator testMethod=test_to_arrow_ensure_bqstorage_client_wo_bqstorage>

@unittest.skipIf(pyarrow is None, "Requires `pyarrow`")
def test_to_arrow_ensure_bqstorage_client_wo_bqstorage(self):
    from google.cloud.bigquery.schema import SchemaField

    schema = [
        SchemaField("name", "STRING", mode="REQUIRED"),
        SchemaField("age", "INTEGER", mode="REQUIRED"),
    ]
    rows = [
        {"f": [{"v": "Alice"}, {"v": "98"}]},
        {"f": [{"v": "Bob"}, {"v": "99"}]},
    ]
    path = "/foo"
    api_request = mock.Mock(return_value={"rows": rows})

    mock_client = _mock_client()
    mock_client._ensure_bqstorage_client.return_value = None
    row_iterator = self._make_one(mock_client, api_request, path, schema)

    tbl = row_iterator.to_arrow(create_bqstorage_client=True)

    # The client attempted to create a BQ Storage client, and even though
    # that was not possible, results were still returned without errors.

  mock_client._ensure_bqstorage_client.assert_called_once()

tests/unit/test_table.py:2570:

---

_mock_self =

def assert_called_once(_mock_self):
    """assert that the mock was called only once.
    """
    self = _mock_self
    if not self.call_count == 1:
        msg = ("Expected '%s' to have been called once. Called %s times.%s"
               % (self._mock_name or 'mock',
                  self.call_count,
                  self._calls_repr()))

      raise AssertionError(msg)

E AssertionError: Expected '_ensure_bqstorage_client' to have been called once. Called 0 times.

/usr/local/Caskroom/miniconda/base/envs/dev-3.10/lib/python3.10/site-packages/mock/mock.py:891: AssertionError
==================================================== short test summary info ====================================================
FAILED tests/unit/test_table.py::TestRowIterator::test_to_arrow_ensure_bqstorage_client_wo_bqstorage - AssertionError: Expecte...
========================================== 1 failed, 1796 passed, 93 skipped in 30.84s ==========================================

</details>

[tswast]

System tests pass locally, as well.

(dev-3.10) ➜  python-bigquery git:(issue1006-python310) pytest tests/system                 
====================================================== test session starts ======================================================
platform darwin -- Python 3.10.0, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /Users/swast/src/github.com/googleapis/python-bigquery
collected 106 items                                                                                                             

tests/system/test_arrow.py ......                                                                                         [  5%]
tests/system/test_client.py .....................................................................                         [ 70%]
tests/system/test_job_retry.py ..                                                                                         [ 72%]
tests/system/test_list_rows.py ....                                                                                       [ 76%]
tests/system/test_magics.py .                                                                                             [ 77%]
tests/system/test_pandas.py ...............sssss                                                                          [ 96%]
tests/system/test_query.py ..                                                                                             [ 98%]
tests/system/test_structs.py ..                                                                                           [100%]

======================================================= warnings summary ========================================================
../python-bigquery-storage/google/cloud/bigquery_storage_v1/services/big_query_read/client.py:17
  /Users/swast/src/github.com/googleapis/python-bigquery-storage/google/cloud/bigquery_storage_v1/services/big_query_read/client.py:17: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    from distutils import util

tests/system/test_magics.py::test_bigquery_magic
  /Users/swast/src/github.com/googleapis/python-bigquery/google/cloud/bigquery/job/query.py:1643: UserWarning: A progress bar was requested, but there was an error loading the tqdm library. Please install tqdm to use the progress bar functionality.
    query_result = wait_for_query(self, progress_bar_type, max_results=max_results)

tests/system/test_magics.py::test_bigquery_magic
  /Users/swast/src/github.com/googleapis/python-bigquery/google/cloud/bigquery/table.py:1938: UserWarning: A progress bar was requested, but there was an error loading the tqdm library. Please install tqdm to use the progress bar functionality.
    record_batch = self.to_arrow(

-- Docs: https://docs.pytest.org/en/stable/warnings.html
==================================== 101 passed, 5 skipped, 3 warnings in 702.45s (0:11:42) =====================================

[tswast]

Re-opening because I missed the need to exclude geopandas from the Python 3.10 snippets session. See failure here: https://source.cloud.google.com/results/invocations/f49f91e3-f6a0-48e9-b2d5-87410cc213b4/targets