refactor: pkg_resources -> importlib.resources #24578
Conversation
Codecov Report
@@ Coverage Diff @@
## master #24578 +/- ##
=======================================
Coverage 69.08% 69.08%
=======================================
Files 1906 1906
Lines 74168 74169 +1
Branches 8164 8164
=======================================
+ Hits 51239 51240 +1
Misses 20807 20807
Partials 2122 2122
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
(Part 1 of 2) setuptools' pkg_resources is in the process of being deprecated in favor of importlib.resources and importlib.metadata (see also PR apache#24514) importlib.resources has been available in the standard library since Python 3.7 This change follows the importlib_resources migration guide and does not perform any additional refactoring. The last component that is using pkg_resources is 'superset.config.BASE_DIR' 'superset.config.BASE_DIR' is mainly used as the default file upload directory in Flask and therefore requires special attention as this directory needs to be writable and needs to exist for the full lifetime of the Flask application.
cwegener
force-pushed
the
importlib-resources
branch
from
2c473c1 to
f864d01
Compare
cwegener
changed the title
|
cc @john-bodley this is the next part in the (The next and final part of the refactor will need to deal with the |
| queue.extend( | ||
| path_name / child_name | ||
| for child_name in resource_listdir("superset", str(path_name)) | ||
| path_name / str(child_name) |
There was a problem hiding this comment.
Why is str(...) needed here?
There was a problem hiding this comment.
Good question. I'll double check. I might have been blindly following mypy errors. Now I'm not sure if the mypy errors were even valid errors.
There was a problem hiding this comment.
Ok. I did some digging on joinpath().
mypy was correct. The argument to joinpath() is defined to be of StrPath type which is defined as Union[str, PathLike[str]]
So, whilst Traversable.join_path(Traversable) will execute in CPython (likely due to coercion), it breaks the API protocol of joinpath()
| ) | ||
| elif path_name.suffix.lower() in YAML_EXTENSIONS: | ||
| elif Path(str(path_name)).suffix.lower() in YAML_EXTENSIONS: |
There was a problem hiding this comment.
This is just a straight forward explicit coercion from Traversable -> str -> Path in order then call the Path.suffix() method.
| contents[path_name] = ( | ||
| resource_stream("superset", str(path_name)).read().decode("utf-8") | ||
| ) | ||
| contents[Path(str(path_name))] = ( |
There was a problem hiding this comment.
This is just a straight forward explicit coercion from Traversable -> str -> Path in order to create the key for the contents typed dict which has a contents: dict[Path, str] type.
mistercrunch
added
🏷️ bot
(Part 1 of 2)
setuptools' pkg_resources is in the process of being deprecated in favor
of importlib.resources and importlib.metadata (see also PR #24514)
importlib.resources has been available in the standard library since
Python 3.7
This change follows the importlib_resources migration guide and does not
perform any additional refactoring.
The last component that is using pkg_resources is
'superset.config.BASE_DIR'
'superset.config.BASE_DIR' is mainly used as the default file upload
directory in Flask and therefore requires special attention as this
directory needs to be writable and needs to exist for the full lifetime
of the Flask application.