os.path.realpath(symlink to DOS devices path that starts with '\\?\Some without ":"\') returns without prefix. #102440
Comments
wintermaples
changed the title
|
I don't see how A symbolic link reparse point contains a substitute path that the system uses, plus an optional print path for display1. What you see in the shell is the print path, but What is the result of
It's a known issue (#89760) that Footnotes |
I read a code of ntpath._getfinalpathname_nonstrict(), perhaps the bug is caused by os.readlink behaviors. The code of ntpath.realpath() processes L702-L706(Python3.11.0 amd64) because _getfinalpathname() causes FileNotFoundError (The specified path is symlink to DOS devices path that starts with '\?\Some without ":"'). except OSError as ex:
if strict:
raise
initial_winerror = ex.winerror
path = _getfinalpathname_nonstrict(path)That calls _getfinalpathname_nonstrict() and _getfinalpathname_nonstrict() calls os.readlink().
I know and read #89760 but the bug is remained even if patch #89760 (comment). |
|
This appears to be a shortcoming in If the reparse point is a symlink with If the reparse point is a mountpoint or if it's a symlink without Assuming the target of "media" really exists, then the following os.stat(r'\\?\GLOBALROOT\ContainerMappedDirectories\93AA221C-4061-460F-8C4E-A2EAC1BF3324') |
This problem happens on volume mount of windows container(docker), so that is not uncommon in practice...
I show reparsepoint of media symbolic link below. (Flags is 0x00000000) |
|
I'm trying to write a fix code to implement prepending "\?GLOBALROOT" to the result of os.readlink(). Modules/posixmodule.c -> os_readlink_impl wchar_t *name = NULL;
Py_ssize_t nameLen = 0;
if (rdb->ReparseTag == IO_REPARSE_TAG_SYMLINK)
{
name = (wchar_t *)((char*)rdb->SymbolicLinkReparseBuffer.PathBuffer +
rdb->SymbolicLinkReparseBuffer.SubstituteNameOffset);
nameLen = rdb->SymbolicLinkReparseBuffer.SubstituteNameLength / sizeof(wchar_t);
}
else if (rdb->ReparseTag == IO_REPARSE_TAG_MOUNT_POINT)
{
name = (wchar_t *)((char*)rdb->MountPointReparseBuffer.PathBuffer +
rdb->MountPointReparseBuffer.SubstituteNameOffset);
nameLen = rdb->MountPointReparseBuffer.SubstituteNameLength / sizeof(wchar_t);
}
else
{
PyErr_SetString(PyExc_ValueError, "not a symbolic link");
}
if (name == NULL) {
return NULL;
}
if (nameLen > 4 && wcsncmp(name, L"\\??\\", 4) == 0) {
/* Our buffer is mutable, so this is okay */
name[1] = L'\\';
}
/* If the substitute name is a rooted path then */
/* the reparse point is MountPoint || ( Symlink && Flags == 0 ) then */
/* should prepend \\?\GLOBALROOT prefix. */
wchar_t prefix[] = L"\\\\?\\GLOBALROOT";
if (!(nameLen > 4 && wcsncmp(name, L"\\\\?\\", 4) == 0) && (nameLen > 1 && wcsncmp(name, L"\\", 1) == 0)) {
bool shouldPrependPrefix =
(rdb->ReparseTag == IO_REPARSE_TAG_MOUNT_POINT) ||
(rdb->ReparseTag == IO_REPARSE_TAG_SYMLINK && rdb->SymbolicLinkReparseBuffer.Flags == 0);
if (shouldPrependPrefix) {
wchar_t *oldName = name;
nameLen = wcslen(prefix) + wcslen(oldName);
name = (wchar_t *) PyMem_Malloc(sizeof(wchar_t) * (nameLen + 1));
wcscpy_s(name, nameLen + 1, prefix);
wcscat_s(name, nameLen + 1, oldName);
PyMem_Free(oldName);
}
}
result = PyUnicode_FromWideChar(name, nameLen);
if (result && path->narrow) {
Py_SETREF(result, PyUnicode_EncodeFSDefault(result));
}
return result;If without PyMem_Free(oldName), the compiled python runs expectedly(but memory leaking). Could you give me some advice and review it? |
Using containers on Windows is still a niche usage that's relatively uncommon compared to the thousands of Python installations on Windows that will never see such a symlink that cannot be created by the normal Note that "\\?\ContainerMappedDirectories\3284C586-8F0D-4325-99EB-76DFF9D5D419" is the "print path" in the reparse point data buffer. We can't rely on the print path because the specification only says what it "SHOULD" be.
You can't call Here's what I wrote to address this problem before I saw your code. The #elif defined(MS_WINDOWS)
#ifndef SYMLINK_FLAG_RELATIVE
#define SYMLINK_FLAG_RELATIVE 0x00000001
#endif
PyObject *result = NULL;
char buf[_Py_MAXIMUM_REPARSE_DATA_BUFFER_SIZE];
_Py_REPARSE_DATA_BUFFER *rdb = (_Py_REPARSE_DATA_BUFFER *)buf;
BOOL fs_result = FALSE;
Py_BEGIN_ALLOW_THREADS
HANDLE hfile = CreateFileW(path->wide, 0, 0, NULL, OPEN_EXISTING,
FILE_FLAG_OPEN_REPARSE_POINT |
FILE_FLAG_BACKUP_SEMANTICS, NULL);
if (hfile != INVALID_HANDLE_VALUE) {
fs_result = DeviceIoControl(hfile, FSCTL_GET_REPARSE_POINT,
NULL, 0, buf, sizeof(buf),
NULL, NULL);
CloseHandle(hfile);
}
Py_END_ALLOW_THREADS
if (!fs_result) {
return path_error(path);
}
wchar_t *name = NULL;
Py_ssize_t length;
BOOL is_absolute;
if (rdb->ReparseTag == IO_REPARSE_TAG_SYMLINK) {
name = (wchar_t *)((char *)rdb->SymbolicLinkReparseBuffer.PathBuffer +
rdb->SymbolicLinkReparseBuffer.SubstituteNameOffset);
length = rdb->SymbolicLinkReparseBuffer.SubstituteNameLength /
sizeof(wchar_t);
is_absolute = rdb->SymbolicLinkReparseBuffer.Flags &
SYMLINK_FLAG_RELATIVE ? FALSE : TRUE;
}
else if (rdb->ReparseTag == IO_REPARSE_TAG_MOUNT_POINT) {
name = (wchar_t *)((char *)rdb->MountPointReparseBuffer.PathBuffer +
rdb->MountPointReparseBuffer.SubstituteNameOffset);
length = rdb->MountPointReparseBuffer.SubstituteNameLength /
sizeof(wchar_t);
is_absolute = TRUE;
}
if (name == NULL) {
PyErr_SetString(PyExc_ValueError, "not a symbolic link");
}
else {
wchar_t *temp_name = NULL;
if (is_absolute) {
assert(name[0] == L'\\');
if (length >= 4 && wcsncmp(name, L"\\??\\", 4) == 0) {
name[1] = L'\\';
}
else {
temp_name = PyMem_Malloc((length + 14 + 1) * sizeof(wchar_t));
if (temp_name == NULL) {
return PyErr_NoMemory();
}
wcscpy(temp_name, L"\\\\?\\GLOBALROOT");
wcsncat(temp_name, name, length);
name = temp_name;
length += 14;
}
}
result = PyUnicode_FromWideChar(name, length);
if (result && path->narrow) {
Py_SETREF(result, PyUnicode_EncodeFSDefault(result));
}
if (temp_name) {
PyMem_Free(temp_name);
}
}
return result;
#endif |
|
Thankyou your advice!
I agree your notion. We must use substitute name. But if os.readlink use substitute name without prepending suffix, we cannot resolve these symbolic paths on Python(except read and analyze reparse point on Python directly but it may be bad).
|
os.path.realpath(symblic link to DOS Device Paths)returns a path without\\?\prefix even if patch that fix.For example, there is below symlinks on C:\test.
Then exexute python and call os.path.realpath and call some functions to inspect.
As the log indicates, os.path.realpath returns without a prefix.
I think it is wrong that _getfinalpathname_nonstrict removes DOS devices paths prefix.
So to fix the problem, I think we need to fix _getfinalpathname function on "Modules/posixmodule.c".
Please some opinions.
The text was updated successfully, but these errors were encountered: