add support for path-like objects in sys.path

[cjerdonek]

BPO	32642
Nosy	@malemburg, @brettcannon, @jaraco, @ncoghlan, @merwok, @cjerdonek, @ericsnowcurrently, @serhiy-storchaka, @eryksun, @jayyyin, @noamcohen97
PRs	bpo-32642: adding compatibility for pathlike objects in sys.path #5691 bpo-32642: Allow for PathLike objects in sys.path #22000 bpo-32642: add support for path-like objects in sys.path #32118

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-01-24.05:27:12.840>
labels = ['type-feature', 'library', '3.11']
title = 'add support for path-like objects in sys.path'
updated_at = <Date 2022-03-26.18:25:50.267>
user = 'https://github.com/cjerdonek'

bugs.python.org fields:

activity = <Date 2022-03-26.18:25:50.267>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-01-24.05:27:12.840>
creator = 'chris.jerdonek'
dependencies = []
files = []
hgrepos = []
issue_num = 32642
keywords = ['patch']
message_count = 28.0
messages = ['310560', '311693', '311864', '311895', '311986', '312070', '312072', '313283', '313337', '313388', '313460', '313883', '313885', '313887', '313923', '314065', '340950', '409732', '415012', '416037', '416042', '416048', '416050', '416052', '416053', '416057', '416065', '416078']
nosy_count = 12.0
nosy_names = ['lemburg', 'brett.cannon', 'jaraco', 'ncoghlan', 'eric.araujo', 'chris.jerdonek', 'python-dev', 'eric.snow', 'serhiy.storchaka', 'eryksun', 'jayyin11043', 'ncohen']
pr_nums = ['5691', '22000', '32118']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue32642'
versions = ['Python 3.11']

[cjerdonek]

This issue is to suggest enhancing sys.path to recognize path-like objects, per PEP-519.

I recently ran into an issue where a path was getting added to sys.path, but the corresponding imports weren't working as they should, even though sys.path showed the path as being present. Eventually I tracked this down to the path being a pathlib.Path object rather than a string. This wasn't obvious because Path objects appear as strings in normal debug output, etc.

The sys.path docs currently say this:

Only strings and bytes should be added to sys.path; all other data types are ignored during import.

[jayyyin]

This looks a lot like https://bugs.python.org/issue32446, I'd like to tackle this, if we are going through with it.

[jayyyin]

what file(s) is/are the sys.path code located in?

[yan12125]

what file(s) is/are the sys.path code located in?

If I understand it correctly, sys.path is handled in importlib._bootstrap_external.PathFinder.find_spec(). I can patch PathFinder for handling path-like objects: https://github.com/yan12125/cpython/commit/e3fd473b54cbb368533e651fd896bbc813a43924

Here's an example usage:

# t.py
import pathlib
import sys

sys.path.append(pathlib.Path('foo'))

import s

# foo/s.py
print(123)

[jayyyin]

Thanks for the reply

[jayyyin]

ok, so I found the PathFinder class you referenced, just making sure, this issue pertains to changing "self.path"'s usage and declaration to be a path-like object instead of the hard coded 'sys', 'path' returns? or is that part of it already?, also since this uses a tuple to keep track of the parent path, is there functionality in the path-like objects to find parent paths?

[jayyyin]

cpython/Lib/importlib/_bootstrap_external.py

Lines 1069 to 1090 in 3c34aad

	class _NamespacePath:
	"""Represents a namespace package's path. It uses the module name
	to find its parent module, and from there it looks up the parent's
	__path__. When this changes, the module's own path is recomputed,
	using path_finder. For top-level modules, the parent module's path
	is sys.path."""
	def __init__(self, name, path, path_finder):
	self._name = name
	self._path = path
	self._last_parent_path = tuple(self._get_parent_path())
	self._path_finder = path_finder
	def _find_parent_path_names(self):
	"""Returns a tuple of (parent-module-name, parent-path-attr-name)"""
	parent, dot, me = self._name.rpartition('.')
	if dot == '':
	# This is a top-level module. sys.path contains the parent path.
	return 'sys', 'path'
	# Not a top-level module. parent-module.__path__ contains the
	# parent path.
	return parent, '__path__'

those are the particular class and lines I'm referring to

[jayyyin]

I'm unsure how to regenerate the files that interact with the code for sys.path as travisCI states
"
Generated files not up to date
M Python/importlib_external.h
"
since my code changes some of how the importing is handled

[brettcannon]

'make regen-all' should do it (but so will just running 'make -s -j' like
any old build of CPython).

On Mon, Mar 5, 2018, 12:43 Jay Yin, <report@bugs.python.org> wrote:

Jay Yin <jayyin11043@gmail.com> added the comment:

I'm unsure how to regenerate the files that interact with the code for
sys.path as travisCI states
"
Generated files not up to date
M Python/importlib_external.h
"
since my code changes some of how the importing is handled

----------

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32642\>

---

[jayyyin]

I've been stuck on "test_poplib" for over 149661 sec now, that's about 41 hours... I don't think this is working correctly, although I'm unsure what test_poplib is doing that has been affected by what I've changed here.

[jayyyin]

The issue was resolved by updating my version of the rest of the package apparently and remaking the whole thing, must have been some outdated stuff on my end causing the issue

[jayyyin]

I'm having issues with my local changes for my PR, I'm unsure why my local machine takes a seemingly infinite amount of time on test_poplib, so again I think something to do with my local environment is causing issues again, any help would be appreciated...