os.path.realpath uppercases Windows drive letter on Python 3.8

[saschanaz]

BPO	40368
Nosy	@eryksun, @saschanaz

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-04-22.21:25:43.654>
labels = ['type-bug', '3.8']
title = 'os.path.realpath uppercases Windows drive letter on Python 3.8'
updated_at = <Date 2020-04-23.20:10:43.760>
user = 'https://github.com/saschanaz'

bugs.python.org fields:

activity = <Date 2020-04-23.20:10:43.760>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = []
creation = <Date 2020-04-22.21:25:43.654>
creator = 'saschanaz'
dependencies = []
files = []
hgrepos = []
issue_num = 40368
keywords = []
message_count = 5.0
messages = ['367053', '367054', '367118', '367123', '367143']
nosy_count = 2.0
nosy_names = ['eryksun', 'saschanaz']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue40368'
versions = ['Python 3.8']

[saschanaz]

$ python3.7
Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os;os.path.realpath('c:/')
'c:\\'

$ python3.8
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os;os.path.realpath('c:/')
'C:\\'

This behavior is inconsistent with os.path.abspath where it always returns lowercased drive letter, and also causes a failure in Gecko build script: https://bugzilla.mozilla.org/show_bug.cgi?id=1628726

[eryksun]

This behavior is inconsistent with os.path.abspath where it always
returns lowercased drive letter,

ntpath.abspath calls GetFullPathNameW, which generally preserves the input case of the device/drive component. But it doesn't always preserve drive-letter case. In particular, a drive-relative path gets resolved with the upper-case drive letter:

    >>> os.path.abspath('c:spam')
    'C:\\Temp\\spam'

That's a corner case that maybe needs to be addressed for ntpath.abspath. However, regarding ntpath.realpath, I see no reason for it to preserve the input case. To the contrary, it should always make a best effort to normalize the input path to use the real device and component names and replace 8.3 short names with long names.

and also causes a failure in Gecko build script:
https://bugzilla.mozilla.org/show_bug.cgi?id=1628726

IMO, the bug there is using a case-insensitive path as a key without first case folding the string.

[saschanaz]

I mentioned os.path.abspath because os.path.abspath(".") on console returned 'c:\\Users\\Kagami\\Documents\\GitHub\\gecko-dev'. It seems this incompatibility is partially because MSYS shell prefers lowercase letter for Windows path while Windows prefers otherwise.

[saschanaz]

Should ntpath.normpath make the drive letter uppercase?

[eryksun]

os.path.abspath(".") returned
'c:\\Users\\Kagami\\Documents\\GitHub\\gecko-dev'.
Should ntpath.normpaoh make the drive letter uppercase?

ntpath.abspath and ntpath.normpath should preserve the input case for all components of a path because they don't query the system for the real path. On the other hand, ntpath.realpath in 3.8+ opens the path and queries the final path from the system.

With drive-relative paths, ntpath.abspath does upper-case the drive letter. That's due to the underlying GutFullPathNameW call on Windows (an API function that normalizes a path as a string-only operation). We should just leave that up to Windows instead of trying to impose consistency. The behavior could be documented, however, along with other Windows behaviors such as per-drive working directories with drive-relative paths, DOS devices (e.g. "C:/Temp/con.txt" -> r"\\.\con") and stripping of trailing dots and spaces (e.g. "C:/Temp/spam. . ." -> r"C:\Temp\spam"). These cases make ntpath.abspath(path) more complicated than just the documented equivalent of ntpath.normpath(ntpath.join(os.getcwd(), path))` on "most platforms" (i.e. most POSIX platforms -- not Windows).