Deprecate portions or all of pkgutil module.

[ericsnowcurrently]

BPO	19939
Nosy	@brettcannon, @pjeby, @ncoghlan, @ericsnowcurrently

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2013-12-10.05:54:41.796>
labels = ['type-feature', 'library']
title = 'Deprecate portions or all of pkgutil module.'
updated_at = <Date 2013-12-10.15:11:15.442>
user = 'https://github.com/ericsnowcurrently'

bugs.python.org fields:

activity = <Date 2013-12-10.15:11:15.442>
actor = 'brett.cannon'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2013-12-10.05:54:41.796>
creator = 'eric.snow'
dependencies = []
files = []
hgrepos = []
issue_num = 19939
keywords = []
message_count = 3.0
messages = ['205771', '205778', '205823']
nosy_count = 5.0
nosy_names = ['brett.cannon', 'pje', 'ncoghlan', 'Arfrever', 'eric.snow']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue19939'
versions = ['Python 3.5']

[ericsnowcurrently]

In the last Python releases, particularly 3.3 and 3.4, we've made improvements to the import machinery that render (at least) portions of pkgutil obsolete. Here's a breakdown of the public API of pkgutil:

get_importer() - duplicate of PathFinder._path_importer_cache()
iter_importers() - yields the path entry finder for each path entry
find_loader() - a parent-importing wrapper around (deprecated) importlib.find_loader()
get_loader() - looks at module.__loader__ or calls find_loader()
walk_packages() - loader-focused
iter_modules() - loader-focused
get_data() - a wrapper around the InspectLoader.get_data() API
read_code() - duplicates importlib functionality
extend_path() - no longer needed (namespace packages)
ImpImporter - already deprecated in favor of importlib
ImpLoader - already deprecated in favor of importlib

I would expect that nearly all of the module could be deprecated and any gaps in functionality ported to importlib.util.

Is it worth it to go to the effort? To me the biggest thing would be identifying what functionality (e.g. locating all packages within a directory) in pkgutil is still relevant and should be distilled into public APIs in importlib. The job of actually deprecating and porting code would mostly be mechanical and not even a large amount of work.

[ncoghlan]

Programmatic deprecation definitely isn't worth it - setuptools et al need this for cross-version compatibility with 2.x, and packaging tools are hard enough to write without us programatically deprecating things in 3.x releases.

Explicit documented deprecations where appropriate would be good, though.

[brettcannon]

I think programmatic deprecation is actually fine since that only comes up when running under -W which would be a bit odd for any tool to be run under except when testing. E.g. I had no personal issue deprecating imp for Python 3.4 even though that's the only way to do 2/3 programmatic import craziness as its use should be discouraged as much as possible since it's now fundamentally the wrong paradigm. And the tools can simply silence the deprecation if they actually find it noisy.

I do agree it should just be a PendingDeprecationWarning and not expect to remove it until either Python 4 or when the community has heavily shifted to Python 3. But when there are semantic replacements I think not doing a programmatic deprecation with warnings off by default is a disservice.

[brettcannon]

I think we are down to iter_modules, walk_packages, and ModuleInfo that need replacing or a decision what to do about them.

[brettcannon]

Some notes on how to replace a bunch of things in pkgutil:

1. find_loader: importlib.util.find_spec().__loader__
2. get_importer: sys.path_importer_cache.get(path) or search sys.path_hooks (also no uses found in the first two pages of a code search on GitHub)
3. get_loader: importlib.util.find_spec().__loader__
4. iter_importers: Yield meta path finders or yield finders for __path__ (also no uses found in the first two pages of a code search on GitHub)

[adorilson]

From 3.3 until 3.11, if you see the pkgutil.ImpImporter and pkgutil.ImpLoader classes' docs, you will find a deprecation note without any mention of when it will be removed.

However, suddenly, they were removed in 3.12.

It is missing a will be removed note, like aifc.

[merwok]

Not correct: older docs show a deprecation message since 3.3, now the doc is cleaned up in 3.12 and the class is still there (this is a doc deprecation to avoid causing trouble to people).

[adorilson]

the class is still there

Which class is still where?

pkgutil.ImpImporter and pkgutil.ImpLoader has been removed in 3.12.

[merwok]

I meant in the code, not doc (that’s that we call a doc deprecation).

But I was looking at the wrong code locally, I was on 3.9 to build CPython for an old project.

@warsaw removed ImpImporter in 3.12 indeed