Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Enable support and testing for python 3.11 #512

Merged
merged 11 commits into from
Jan 5, 2023
40 changes: 40 additions & 0 deletions .kokoro/samples/python3.11/common.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Format: //devtools/kokoro/config/proto/build.proto
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tswast

are these files typically autogenerated OR do we add the manually?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect that these are autogenerated. However, I tried adding them manually as we had a test failure relating to Kokoro: https://source.cloud.google.com/results/invocations/b41e3d1e-1e73-454d-a24a-d1a790c7b129/targets/cloud-devrel%2Fclient-libraries%2Fpython%2Fgoogleapis%2Fpython-bigquery-sqlalchemy%2Fpresubmit%2Fpresubmit/log
I believe that the failure was the result of Session unit-3.11 failed: Python interpreter 3.11 not found. and I was experimenting whether adding these files resolved that issue. If it doesn't I'll go ahead and remove those files.

Copy link
Contributor Author

@mr-mcox mr-mcox Jan 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh shoot - it appears that I can't find that out the results of that experiment without adding the kokoro:run label, which I'm unable to do. I'll revert those additions as it was a slim chance that this would fix the build.


# Build logs will be here
action {
define_artifacts {
regex: "**/*sponge_log.xml"
}
}

# Specify which tests to run
env_vars: {
key: "RUN_TESTS_SESSION"
value: "py-3.11"
}

# Declare build specific Cloud project.
env_vars: {
key: "BUILD_SPECIFIC_GCLOUD_PROJECT"
value: "python-docs-samples-tests-311"
}

env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-bigquery-sqlalchemy/.kokoro/test-samples.sh"
}

# Configure the docker image for kokoro-trampoline.
env_vars: {
key: "TRAMPOLINE_IMAGE"
value: "gcr.io/cloud-devrel-kokoro-resources/python-samples-testing-docker"
}

# Download secrets for samples
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"

# Download trampoline resources.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-bigquery-sqlalchemy/.kokoro/trampoline_v2.sh"
6 changes: 6 additions & 0 deletions .kokoro/samples/python3.11/continuous.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Format: //devtools/kokoro/config/proto/build.proto

env_vars: {
key: "INSTALL_LIBRARY_FROM_SOURCE"
value: "True"
}
11 changes: 11 additions & 0 deletions .kokoro/samples/python3.11/periodic-head.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Format: //devtools/kokoro/config/proto/build.proto

env_vars: {
key: "INSTALL_LIBRARY_FROM_SOURCE"
value: "True"
}

env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-bigquery-sqlalchemy/.kokoro/test-samples-against-head.sh"
}
6 changes: 6 additions & 0 deletions .kokoro/samples/python3.11/periodic.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Format: //devtools/kokoro/config/proto/build.proto

env_vars: {
key: "INSTALL_LIBRARY_FROM_SOURCE"
value: "False"
}
6 changes: 6 additions & 0 deletions .kokoro/samples/python3.11/presubmit.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Format: //devtools/kokoro/config/proto/build.proto

env_vars: {
key: "INSTALL_LIBRARY_FROM_SOURCE"
value: "True"
}
10 changes: 6 additions & 4 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ In order to add a feature:
documentation.

- The feature must work fully on the following CPython versions:
3.7, 3.8, 3.9 and 3.10 on both UNIX and Windows.
3.7, 3.8, 3.9, 3.10 and 3.11 on both UNIX and Windows.

- The feature must not add unnecessary dependencies (where
"unnecessary" is of course subjective, but new dependencies should
Expand Down Expand Up @@ -72,7 +72,7 @@ We use `nox <https://nox.readthedocs.io/en/latest/>`__ to instrument our tests.

- To run a single unit test::

$ nox -s unit-3.10 -- -k <name of test>
$ nox -s unit-3.11 -- -k <name of test>


.. note::
Expand Down Expand Up @@ -143,12 +143,12 @@ Running System Tests
$ nox -s system

# Run a single system test
$ nox -s system-3.10 -- -k <name of test>
$ nox -s system-3.11 -- -k <name of test>


.. note::

System tests are only configured to run under Python 3.8 and 3.10.
System tests are only configured to run under Python 3.8 and 3.11.
For expediency, we do not run them in older versions of Python 3.

This alone will not run the tests. You'll need to change some local
Expand Down Expand Up @@ -225,11 +225,13 @@ We support:
- `Python 3.8`_
- `Python 3.9`_
- `Python 3.10`_
- `Python 3.11`_

.. _Python 3.7: https://docs.python.org/3.7/
.. _Python 3.8: https://docs.python.org/3.8/
.. _Python 3.9: https://docs.python.org/3.9/
.. _Python 3.10: https://docs.python.org/3.10/
.. _Python 3.11: https://docs.python.org/3.11/


Supported versions can be found in our ``noxfile.py`` `config`_.
Expand Down
9 changes: 4 additions & 5 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@

DEFAULT_PYTHON_VERSION = "3.8"

UNIT_TEST_PYTHON_VERSIONS = ["3.7", "3.8", "3.9", "3.10"]
UNIT_TEST_PYTHON_VERSIONS = ["3.7", "3.8", "3.9", "3.10", "3.11"]
UNIT_TEST_STANDARD_DEPENDENCIES = [
"mock",
"asyncmock",
Expand All @@ -51,13 +51,12 @@
"tests",
"alembic",
],
"3.10": [
"3.11": [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tswast

What are our guidelines on what goes into the Unit Test Extras?
Are those always used solely in conjunction with the earliest and latest Python Versions?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely lowest so that we make sure the lower dependency bounds we have in setup.py and our contraints files are correct. Latest SGTM so long as the extra dependencies have been released. In the past we'd update before pyarrow had cut a release, but this doesn't appear to be the case right now.

"tests",
"geography",
],
}

SYSTEM_TEST_PYTHON_VERSIONS = ["3.8", "3.10"]
SYSTEM_TEST_PYTHON_VERSIONS = ["3.8", "3.11"]
SYSTEM_TEST_STANDARD_DEPENDENCIES = [
"mock",
"pytest",
Expand All @@ -74,7 +73,7 @@
"tests",
"alembic",
],
"3.10": [
"3.11": [
"tests",
"geography",
],
Expand Down
6 changes: 3 additions & 3 deletions owlbot.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@
extras = ["tests"]
extras_by_python = {
"3.8": ["tests", "alembic"],
"3.10": ["tests", "geography"],
"3.11": ["tests", "geography"],
}
templated_files = common.py_library(
system_test_python_versions=["3.8", "3.10"],
system_test_python_versions=["3.8", "3.11"],
cov_level=100,
unit_test_extras=extras,
unit_test_extras_by_python=extras_by_python,
Expand Down Expand Up @@ -201,7 +201,7 @@ def compliance(session):
)
if session.python == "3.8":
extras = "[tests,alembic]"
elif session.python == "3.10":
elif session.python == "3.11":
extras = "[tests,geography]"
else:
extras = "[tests]"
Expand Down
3 changes: 2 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ def readme():
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Operating System :: OS Independent",
"Topic :: Database :: Front-Ends",
],
Expand All @@ -91,7 +92,7 @@ def readme():
"future",
],
extras_require=extras,
python_requires=">=3.7, <3.11",
python_requires=">=3.7, <3.12",
tests_require=["packaging", "pytz"],
entry_points={
"sqlalchemy.dialects": ["bigquery = sqlalchemy_bigquery:BigQueryDialect"]
Expand Down
1 change: 1 addition & 0 deletions testing/constraints-3.11.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
numpy>=1.23