-
-
Notifications
You must be signed in to change notification settings - Fork 636
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[internal] Replace pkgutil.get_data
with new read_resource
API
#16379
Changes from 2 commits
913e894
ac24a35
353f05d
6582d44
a914acc
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,13 @@ | ||
# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md). | ||
# Licensed under the Apache License, Version 2.0 (see LICENSE). | ||
|
||
resource(name="dependency_parser", source="dependency_parser.py") | ||
resource(name="dependency_parser", source="dependency_parser_py", dependencies=[":init_py",]) | ||
resource(name="init_py", source="__init__.py",) | ||
|
||
# Also expose scripts as python sources so they get formatted/linted/checked. | ||
python_source( | ||
name="dependency_parser_source", | ||
source="dependency_parser.py", | ||
source="dependency_parser_py", | ||
# This is run with Python 2.7 and 3.5+, so we shouldn't be running pyupgrade. | ||
# skip_pyupgrade=True, | ||
) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -78,3 +78,5 @@ pex_binary( | |
) | ||
|
||
python_tests(name="tests") | ||
|
||
resources(name="resources", sources=["VERSION"]) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../VERSION |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# Copyright 2022 Pants project contributors (see CONTRIBUTORS.md). | ||
# Licensed under the Apache License, Version 2.0 (see LICENSE). | ||
|
||
|
||
import importlib | ||
from importlib import resources | ||
from itertools import chain | ||
|
||
|
||
def read_resource(package_or_module: str, resource: str) -> bytes: | ||
"""Reads a resource file from within the Pants package itself. | ||
|
||
This helper function is designed for compatibility with `pkgutil.get_data()` wherever possible, | ||
but also allows compability with PEP302 pluggable importers such as included with PyOxidizer. | ||
This requires that resources are loaded from a valid Python package (i.e. must have an | ||
`__init__.py` file in the directory). | ||
""" | ||
|
||
a = importlib.import_module(package_or_module) | ||
package_ = a.__package__ | ||
|
||
if package_ is None: | ||
raise ValueError( | ||
"`read_resource` can only help find resources for packages or modules that live in " | ||
"a package." | ||
) | ||
|
||
resource_parts = resource.split("/") | ||
|
||
if len(resource_parts) == 1: | ||
package = package_ | ||
else: | ||
package = ".".join(chain((package_,), resource_parts[:-1])) | ||
resource = resource_parts[-1] | ||
|
||
return resources.read_binary(package, resource) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Deprecated in 3.11: https://docs.python.org/3.11/library/importlib.resources.html#importlib.resources.read_binary Now it's the much more awkward:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Understood. Note that a deprecation won't go away until Python 4.x, so we're safe to use it for a while, and the new API will not be available to us without backporting a new version of The alternative here is vendoring in a new version of |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,10 +2,11 @@ | |
# Licensed under the Apache License, Version 2.0 (see LICENSE). | ||
|
||
import os | ||
import pkgutil | ||
|
||
from packaging.version import Version | ||
|
||
from pants.util.resources import read_resource | ||
|
||
# Set this env var to override the version pants reports. Useful for testing. | ||
_PANTS_VERSION_OVERRIDE = "_PANTS_VERSION_OVERRIDE" | ||
|
||
|
@@ -14,7 +15,7 @@ | |
os.environ.get(_PANTS_VERSION_OVERRIDE) | ||
or | ||
# NB: We expect VERSION to always have an entry and want a runtime failure if this is false. | ||
pkgutil.get_data(__name__, "VERSION").decode().strip() # type: ignore[union-attr] | ||
read_resource("pants.bin", "VERSION").decode().strip() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Adding the relevant bit of the description as a comment here would be helpful. Re: Benjy's command, could probably also reference #16359. But I don't think that you should change that here... should be tackled independently. |
||
) | ||
|
||
PANTS_SEMVER = Version(VERSION) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bad change