gh-117201: Handle leading // for posixpath.commonpath

[nineteendo]

NOTE: this is NOT a duplicate of #117202, the branch has been renamed, causing the old pull request to be closed.

Benchmark:

# test.sh
echo short && python -m timeit -s "import before.posixpath" "before.posixpath.commonpath(['/foo/bar', '/foo/baz'])" && python -m timeit -s "import after_alt.posixpath" "after_alt.posixpath.commonpath(['/foo/bar', '/foo/baz'])" && python -m timeit -s "import after.posixpath" "after.posixpath.commonpath(['/foo/bar', '/foo/baz'])"
echo long && python -m timeit -s "import before.posixpath" "before.posixpath.commonpath(['/usr/lib/python2.6/site-packages/gnome_sudoku/game_selector.py', '/usr/lib/python2.6/site-packages/gnome_sudoku/dialog_swallower.py'])" && python -m timeit -s "import after_alt.posixpath" "after_alt.posixpath.commonpath(['/usr/lib/python2.6/site-packages/gnome_sudoku/game_selector.py', '/usr/lib/python2.6/site-packages/gnome_sudoku/dialog_swallower.py'])" && python -m timeit -s "import after.posixpath" "after.posixpath.commonpath(['/usr/lib/python2.6/site-packages/gnome_sudoku/game_selector.py', '/usr/lib/python2.6/site-packages/gnome_sudoku/dialog_swallower.py'])"
echo many && python -m timeit -s "import before.posixpath" "before.posixpath.commonpath(['/foo/bar', '/foo/baz'] * 10)" && python -m timeit -s "import after_alt.posixpath" "after_alt.posixpath.commonpath(['/foo/bar', '/foo/baz'] * 10)" && python -m timeit -s "import after.posixpath" "after.posixpath.commonpath(['/foo/bar', '/foo/baz'] * 10)"

short
100000 loops, best of 5: 2.52 usec per loop # before
100000 loops, best of 5: 2.92 usec per loop # after without zip
100000 loops, best of 5: 2.91 usec per loop # after
# -> 1.15x slower
long
100000 loops, best of 5: 3.35 usec per loop # before
100000 loops, best of 5: 3.96 usec per loop # after without zip
100000 loops, best of 5: 3.73 usec per loop # after
# -> 1.11x slower
many
20000 loops, best of 5: 11.4 usec per loop # before
20000 loops, best of 5: 16.6 usec per loop # after without zip
20000 loops, best of 5: 16.5 usec per loop # after
# -> 1.45x slower

• Issue: Handle leading // for posixpath.commonpath #117201

[barneygale]

Could you please add test cases covering:

commonpath(['//foo', '/foo', 'foo'])
commonpath(['//foo', '/foo'])
commonpath(['//foo', 'foo'])

I think I'd expect all three to raise ValueError with an appropriate message.

[nineteendo]

Could you please add test cases covering:

commonpath(['//foo', '/foo'])

Do I have permission to change this test then to 3 slashes? In the current implementation I made sure it wouldn't break.

cpython/Lib/test/test_posixpath.py

Line 716 in 54f7e14

check(['/usr//local', '//usr/local'], '/usr/local')

Achieving the desired behaviour would be easy:

if min(roots) != max(roots):  # -> not all roots are the same
    raise ValueError

But I'm not sure what the error message would have to be. //foo & /foo are both absolute paths.

[barneygale]

Ah interesting, OK! Preserving existing behaviour for commonpath(['//foo', '/foo']) sounds fine.

[barneygale]

The patch seems larger than necessary to fix the bug. Could something like this work instead?

diff --git a/Lib/posixpath.py b/Lib/posixpath.py
index 4fc02be69b..4cbe1b1367 100644
--- a/Lib/posixpath.py
+++ b/Lib/posixpath.py
@@ -541,10 +541,9 @@ def commonpath(paths):
     try:
         split_paths = [path.split(sep) for path in paths]
 
-        try:
-            isabs, = set(p[:1] == sep for p in paths)
-        except ValueError:
-            raise ValueError("Can't mix absolute and relative paths") from None
+        prefixes = [splitroot(path)[1] for path in paths]
+        if any(prefixes) != all(prefixes):
+            raise ValueError("Can't mix absolute and relative paths")
 
         split_paths = [[c for c in s if c and c != curdir] for s in split_paths]
         s1 = min(split_paths)
@@ -555,7 +554,7 @@ def commonpath(paths):
                 common = s1[:i]
                 break
 
-        prefix = sep if isabs else sep[:0]
+        prefix = min(prefixes)
         return prefix + sep.join(common)
     except (TypeError, AttributeError):
         genericpath._check_arg_types('commonpath', *paths)

[nineteendo]

I've tried to keep as much of the old structure as possible. Your code appears to do more work.

[nineteendo]

The patch seems larger than necessary to fix the bug. Could something like this work instead?

python -m timeit -s "from test import commonpath1" "commonpath1(('/foo/bar', '/foo/baz'))"; python -m timeit -s "from test import commonpath2" "commonpath2(('/foo/bar', '/foo/baz'))"
100000 loops, best of 5: 3.02 usec per loop # current fix
100000 loops, best of 5: 3.34 usec per loop # your fix
# -> 1.11x slower

[nineteendo]

I've made some changes to further reduce the diff. Now the fix is way more readable.

[barneygale]

Might be faster to sort the roots up front and then check the first and last elements? (Untested).

[nineteendo]

@barneygale, do you think this can be merged now? There don't seem to be any optimisations remaining.

[serhiy-storchaka]

This change can break user code which expects the result always has a single slash prefix. It is unsafe to backport it. So it should be documented as a new feature.

How does this change affect performance? Please test for short (2 items) and long (e.g. /usr/lib/**) sequences.

[nineteendo]

How does this change affect performance? Please test for short (2 items) and long (e.g. /usr/lib/**) sequences.

The speed difference is smaller for long paths, but bigger for many paths (20). See, the updated benchmark.
I don't think the latter can be easily fixed.

[erlend-aasland]

This change can break user code which expects the result always has a single slash prefix.

We should be careful about subtle breaking changes like this.

@nineteendo: Ideally, a discussion would happen on the issue first, and then if a consensus on how to proceed has been reached, a PR can be created. Creating a lot of PRs before a decision has been reached tie up a reviewer resources, creating unneeded review churn and CI churn.