Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Meta issue for cleaning up import system cruft #97850

Open
25 of 31 tasks
warsaw opened this issue Oct 4, 2022 · 11 comments
Open
25 of 31 tasks

Meta issue for cleaning up import system cruft #97850

warsaw opened this issue Oct 4, 2022 · 11 comments
Assignees

Comments

@warsaw
Copy link
Member

warsaw commented Oct 4, 2022

This is a meta issue tracking all the things that need to be cleaned up in the import system. From removing long deprecated APIs to migrating internal access to __spec__, we'll have several task lists for each related set of clean ups.

@brettcannon @ericsnowcurrently for visibility.

Removing deprecated APIs

Over in What's New for Python 3.11 there's a long list of things that have been deprecated long enough that they can be removed in Python 3.12. Here's that task list for tracking purposes. Watch for linked PRs (maybe one big one or many small ones -- we'll see!).

@brettcannon @ericsnowcurrently for visibility.

Update docs

  • Make sure examples and details from PEP 302 are covered in the importlib docs (/cc @Yhg1s )
  • Remove https://docs.python.org/3/reference/import.html#open-issues (PR)
  • It would be really nice to have a diagram.
  • (import_machinery.rst) how about a section devoted just to the attributes of modules and packages, perhaps expanding upon or supplanting the related entries in the data model reference page?
  • runpy, pkgutil, et al in the library manual should all get “See Also” links at the top pointing to the new import system section.
  • Add more explanation regarding the different ways in which __main__ is initialized?
  • Add more info on __main__ quirks/pitfalls (i.e. copy from PEP 395).

Missing deprecations

Here's a list of related APIs that have not yet been deprecated, but should be. We'll deprecate them in 3.12 and remove them in 3.14.

  • pkgutil.find_loader()
  • pkgutil.get_loader() (or it needs to be reimplemented to not use find_loader())

Other:

@warsaw warsaw changed the title Task list for cleaning up import system cruft for Python 3.12 Meta issue for cleaning up import system cruft Oct 4, 2022
@warsaw warsaw self-assigned this Oct 5, 2022
sobolevn added a commit to sobolevn/cpython that referenced this issue Oct 5, 2022
@sobolevn
Copy link
Member

sobolevn commented Oct 5, 2022

I've removed importlib.utils.set_loader, importlib.utils.set_package, and
importlib.utils.module_for_loader from importlib.utils in #97898 🎉

warsaw added a commit that referenced this issue Oct 5, 2022
Remove all known instances of module_repr()
@brettcannon brettcannon self-assigned this Oct 5, 2022
brettcannon added a commit to brettcannon/cpython that referenced this issue Oct 5, 2022
sobolevn added a commit to sobolevn/cpython that referenced this issue Oct 5, 2022
carljm added a commit to carljm/cpython that referenced this issue Oct 6, 2022
* main: (66 commits)
  pythongh-65961: Raise `DeprecationWarning` when `__package__` differs from `__spec__.parent` (python#97879)
  docs(typing): add "see PEP 675" to LiteralString (python#97926)
  pythongh-97850: Remove all known instances of module_repr() (python#97876)
  I changed my surname early this year (python#96671)
  pythongh-93738: Documentation C syntax (:c:type:<C type> -> :c:expr:<C type>) (python#97768)
  pythongh-91539: improve performance of get_proxies_environment  (python#91566)
  build(deps): bump actions/stale from 5 to 6 (python#97701)
  pythonGH-95172 Make the same version `versionadded` oneline (python#95172)
  pythongh-88050: Fix asyncio subprocess to kill process cleanly when process is blocked (python#32073)
  pythongh-93738: Documentation C syntax (Function glob patterns -> literal markup) (python#97774)
  pythongh-93357: Port test cases to IsolatedAsyncioTestCase, part 2 (python#97896)
  pythongh-95196: Disable incorrect pickling of the C implemented classmethod descriptors (pythonGH-96383)
  pythongh-97758: Fix a crash in getpath_joinpath() called without arguments (pythonGH-97759)
  pythongh-74696: Pass root_dir to custom archivers which support it (pythonGH-94251)
  pythongh-97661: Improve accuracy of sqlite3.Cursor.fetchone docs (python#97662)
  pythongh-87092: bring compiler code closer to a preprocessing-opt-assembler organisation (pythonGH-97644)
  pythonGH-96704: Add {Task,Handle}.get_context(), use it in call_exception_handler() (python#96756)
  pythongh-93738: Documentation C syntax (:c:type:`PyTypeObject*` -> :c:expr:`PyTypeObject*`) (python#97778)
  pythongh-97825: fix AttributeError when calling subprocess.check_output(input=None) with encoding or errors args (python#97826)
  Add re.VERBOSE flag documentation example (python#97678)
  ...
brettcannon added a commit that referenced this issue Oct 6, 2022
…97935)

Remove the open issues section from the import reference

Tracking in #97850 instead.
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 6, 2022
…nce (pythonGH-97935)

Remove the open issues section from the import reference

Tracking in python#97850 instead.
(cherry picked from commit f8edc6f)

Co-authored-by: Brett Cannon <brett@python.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Oct 6, 2022
…nce (pythonGH-97935)

Remove the open issues section from the import reference

Tracking in python#97850 instead.
(cherry picked from commit f8edc6f)

Co-authored-by: Brett Cannon <brett@python.org>
miss-islington added a commit that referenced this issue Oct 6, 2022
…H-97935)

Remove the open issues section from the import reference

Tracking in #97850 instead.
(cherry picked from commit f8edc6f)

Co-authored-by: Brett Cannon <brett@python.org>
miss-islington added a commit that referenced this issue Oct 6, 2022
…ence (GH-97935) (GH-97994)

Remove the open issues section from the import reference

Tracking in #97850 instead.
(cherry picked from commit f8edc6f)


Co-authored-by: Brett Cannon <brett@python.org>

Automerge-Triggered-By: GH:brettcannon
carljm added a commit to carljm/cpython that referenced this issue Oct 6, 2022
* main:
  fixes pythongh-96078: os.sched_yield release the GIL while calling sched_yield(2). (pythongh-97965)
  pythongh-65961: Do not rely solely on `__cached__` (pythonGH-97990)
  pythongh-97850: Remove the open issues section from the import reference (python#97935)
  Docs: pin sphinx-lint (pythonGH-97992)
  pythongh-94590: add signatures to operator itemgetter, attrgetter, methodcaller (python#94591)
  Add Pynche's move to the What's new in 3.11 (python#97974)
  pythongh-97781: Apply changes from importlib_metadata 5. (pythonGH-97785)
  pythongh-86482: Document assignment expression need for ()s (python#23291)
  pythongh-97943: PyFunction_GetAnnotations should return a borrowed reference. (python#97949)
  pythongh-94808: Coverage: Test that maximum indentation level is handled (python#95926)
warsaw pushed a commit that referenced this issue Oct 7, 2022
* gh-97850: Remove deprecated functions from `importlib.utils`

* Rebase and remove `set_package` from diff
carljm added a commit to carljm/cpython that referenced this issue Oct 8, 2022
* main: (53 commits)
  pythongh-94808: Coverage: Test that maximum indentation level is handled (python#95926)
  pythonGH-88050: fix race in closing subprocess pipe in asyncio  (python#97951)
  pythongh-93738: Disallow pre-v3 syntax in the C domain (python#97962)
  pythongh-95986: Fix the example using match keyword (python#95989)
  pythongh-97897: Prevent os.mkfifo and os.mknod segfaults with macOS 13 SDK (pythonGH-97944)
  pythongh-94808: Cover `PyUnicode_Count` in CAPI (python#96929)
  pythongh-94808: Cover `PyObject_PyBytes` case with custom `__bytes__` method (python#96610)
  pythongh-95691: Doc BufferedWriter and BufferedReader (python#95703)
  pythonGH-88968: Add notes about socket ownership transfers (python#97936)
  pythongh-96865: [Enum] fix Flag to use CONFORM boundary (pythonGH-97528)
  pythongh-65961: Raise `DeprecationWarning` when `__package__` differs from `__spec__.parent` (python#97879)
  docs(typing): add "see PEP 675" to LiteralString (python#97926)
  pythongh-97850: Remove all known instances of module_repr() (python#97876)
  I changed my surname early this year (python#96671)
  pythongh-93738: Documentation C syntax (:c:type:<C type> -> :c:expr:<C type>) (python#97768)
  pythongh-91539: improve performance of get_proxies_environment  (python#91566)
  build(deps): bump actions/stale from 5 to 6 (python#97701)
  pythonGH-95172 Make the same version `versionadded` oneline (python#95172)
  pythongh-88050: Fix asyncio subprocess to kill process cleanly when process is blocked (python#32073)
  pythongh-93738: Documentation C syntax (Function glob patterns -> literal markup) (python#97774)
  ...
carljm added a commit to carljm/cpython that referenced this issue Oct 8, 2022
* main:
  pythonGH-97002: Prevent `_PyInterpreterFrame`s from backing more than one `PyFrameObject` (pythonGH-97996)
  pythongh-97973: Return all necessary information from the tokenizer (pythonGH-97984)
  fixes pythongh-96078: os.sched_yield release the GIL while calling sched_yield(2). (pythongh-97965)
  pythongh-65961: Do not rely solely on `__cached__` (pythonGH-97990)
  pythongh-97850: Remove the open issues section from the import reference (python#97935)
  Docs: pin sphinx-lint (pythonGH-97992)
  pythongh-94590: add signatures to operator itemgetter, attrgetter, methodcaller (python#94591)
  Add Pynche's move to the What's new in 3.11 (python#97974)
  pythongh-97781: Apply changes from importlib_metadata 5. (pythonGH-97785)
  pythongh-86482: Document assignment expression need for ()s (python#23291)
  pythongh-97943: PyFunction_GetAnnotations should return a borrowed reference. (python#97949)
carljm added a commit to carljm/cpython that referenced this issue Oct 8, 2022
* main:
  pythongh-86298: Ensure that __loader__ and __spec__.loader agree in warnings.warn_explicit() (pythonGH-97803)
  pythongh-82874: Convert remaining importlib format uses to f-str. (python#98005)
  Docs: Fix backtick errors found by sphinx-lint (python#97998)
  pythongh-97850: Remove deprecated functions from `importlib.utils` (python#97898)
  Remove extra spaces in custom openSSL documentation. (python#93568)
  pythonGH-90985: Revert  "Deprecate passing a message into cancel()" (python#97999)
mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
…nce (python#97935)

Remove the open issues section from the import reference

Tracking in python#97850 instead.
mpage pushed a commit to mpage/cpython that referenced this issue Oct 11, 2022
…ython#97898)

* pythongh-97850: Remove deprecated functions from `importlib.utils`

* Rebase and remove `set_package` from diff
sobolevn added a commit to sobolevn/cpython that referenced this issue Oct 21, 2022
@brettcannon
Copy link
Member

FYI #64138 is a tracking issue for pkgutil all up.

pablogsal pushed a commit that referenced this issue Oct 22, 2022
…ence (GH-97935) (GH-97994)

Remove the open issues section from the import reference

Tracking in #97850 instead.
(cherry picked from commit f8edc6f)


Co-authored-by: Brett Cannon <brett@python.org>

Automerge-Triggered-By: GH:brettcannon
@warsaw
Copy link
Member Author

warsaw commented Apr 29, 2023

From the meta checklist:

Deprecate pkgutil.find_loader()

I kind of think we're okay here, at least for 3.12. This uses importlib.util.find_spec() so I feel like it's not so urgent to deprecate and actually probably has some utility.

Deprecate pkgutil.get_loader()

Similarly, I'm not sure this even needs to be deprecated. It is implemented in terms of the above, and thus find_spec(), and it has some potentially useful wrapper semantics.

I'm going to mark both issues as complete and not deprecate it in 3.12.

@warsaw
Copy link
Member Author

warsaw commented Apr 29, 2023

@brettcannon @ericsnowcurrently - I think this PR is ready for review. I feel good that we've implemented all the removals we need for 3.12. Like the other issues, we can address the doc updates after beta 1.

@warsaw
Copy link
Member Author

warsaw commented May 3, 2023

The removal branch has been merged, but I'm leaving this meta issue open for now so we can address the documentation checklist during the beta cycle.

AlexWaygood pushed a commit that referenced this issue May 3, 2023
…thod (#104133)

Suppress cross-references to ``module_repr``
AlexWaygood added a commit that referenced this issue May 3, 2023
…ctions (#104134)

`importlib.utils` -> `importlib.util` in a few places

---------

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
@ericsnowcurrently
Copy link
Member

Thanks again for tackling this, @warsaw!

@hugovk
Copy link
Member

hugovk commented May 3, 2023

[For the imp module, I created issues for most of the 71 projects in the PyPI top 5k (listed at python/cpython#98573 (comment)), where I could find their tracker. And some already had issues.

I've checked off projects which had already fixed but not yet released, or have fixed since I created the issue, or where the project is unmaintained or otherwise abandoned and a fix is unlikely in any case.

Currently 24/71 (34%) are checked off. In the issue I pointed to the 3.11 imp docs which have suggestions of how to upgrade each functions/constants, and most seemed fine with that so far, except for one noting imp.load_source has no suggestions.


Edit 2024-10-17: 57/71 (80%)

brettcannon added a commit that referenced this issue May 3, 2023
…8520)

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
@mdmintz
Copy link

mdmintz commented May 4, 2023

Due to nose being unmaintained (number 41 in your list), that's why there's pynose now (number 55 in your list).
It's a complete drop-in replacement for all occurrences. (Let me know if that isn't the case.)

@Irtiza90 Irtiza90 mentioned this issue Jun 8, 2023
AlexWaygood added a commit that referenced this issue Aug 31, 2023
… `importlib.util.module_for_loader` have been removed (#108719)

Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Aug 31, 2023
…r` and `importlib.util.module_for_loader` have been removed (pythonGH-108719)

Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed
(cherry picked from commit 013a99a)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Yhg1s pushed a commit that referenced this issue Aug 31, 2023
…er` and `importlib.util.module_for_loader` have been removed (GH-108719) (#108723)

gh-97850: Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed (GH-108719)

Note in py312 whatsnew that `importlib.util.set_loader` and `importlib.util.module_for_loader` have been removed
(cherry picked from commit 013a99a)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
@hugovk
Copy link
Member

hugovk commented Nov 5, 2023

Edit 2023-11-05: 36/71 (52%)

We're now over 50% of those in the checklist.

@hugovk
Copy link
Member

hugovk commented Jun 7, 2024

Edit 2024-06-07: 55/71 (78%)

We're now at 78% (fixed or unmaintained/abandoned).

hugovk added a commit that referenced this issue Nov 1, 2024
…119656)

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants