noexec /tmp causes pip install to miss chmod +x on scripts included in wheel file #6364
Labels
C: error messages
Improving error messages
C: wheel
The wheel format and 'pip wheel' command
UX
User experience related
Comments
cjerdonek
added
C: wheel
|
I was able to reproduce.
|
|
Thanks for filing this issue! I don't see any good way to work around this from pip's end. We need to use a temporary build directory to help ensure some amount of reproduciblilty in builds. I'm okay to add a check and a warning for this situation though. |
|
Thanks @chrahunt for the easy reproduction example! :D |
|
I'm interested in adding a better error message for this case, at the very least. My understanding of the original post is that the If So it seems like there are two possible routes here:
Reference for |
This would be my suggestion. |
|
Outline of thought process: My initial thinking was "just test the flags." That would look something like: def has_any_x_flag(path) -> bool:
mode = os.stat(path).st_mode
return bool((mode & 1) or ((mode >> 3) & 1) or ((mode >> 6) & 1))
def has_owner_x_flag(path) -> bool:
return bool((os.stat(path).st_mode >> 6) & 1)However, that will fall far short of what In effect what should be tested for is if this directory were not noexec, what would To that end, I think this implementation most closely mimics import os
def real_x_flag(path) -> bool:
st = os.stat(path)
mode = st.st_mode
user_x = (mode >> 6) & 1
group_x = (mode >> 3) & 1
any_x = mode & 1
any_x = any((user_x, group_x, any_x))
if not any_x:
# Case 1: 0 of 3 -x permissions set, never executable
return False
uid = os.getuid()
if uid == 0:
# Case 2: user is root; True if any of 3 -x permissions set
return any_x
elif (uid == st.st_uid) and user_x:
# Case 3: user is owner of file and user -x permission is set
return True
elif st.st_gid in os.getgroups() and group_x and (uid != st.st_uid):
# Case 4: group matches, owner does not match
return True
elif any_x and (uid != st.st_uid) and (st.st_gid not in os.getgroups()):
# Edge case: the 'any other' -x is effective if both
# user and group do not match
return True
return FalseTo compare, something like this could be used: import itertools as it
import subprocess
def test_x_flag(path):
perms = 'rwxrwxrwx'
print(os.stat(path))
all_modes = map(''.join, it.product('01', repeat=9))
for m in all_modes:
octmode = ''.join(str(int(i, 2)) for i in (m[0:3], m[3:6], m[6:9]))
subprocess.check_output(["sudo", "chmod", octmode, path])
print(
m,
oct(int(m, 2)),
''.join(p if i == '1' else '-' for p, i in zip(perms, m)),
real_x_flag(path),
os.access(path, os.X_OK),
sep="\t"
)(Tested on paths of differing owner/group combinations.) Sample output assuming I am $ ls -la foo.sh
---------- 1 testuser staff 28 Mar 2 19:46 foo.shThen: $ ./tflag.py foo.sh
os.stat_result(st_mode=32768, st_ino=8627062717, st_dev=16777220, st_nlink=1, st_uid=502, st_gid=20, st_size=28, st_atime=1583196374, st_mtime=1583196372, st_ctime=1583199149)
000000000 0o0 --------- False False
000000001 0o1 --------x False False
000000010 0o2 -------w- False False
000000011 0o3 -------wx False False
000000100 0o4 ------r-- False False
000000101 0o5 ------r-x False False
000000110 0o6 ------rw- False False
000000111 0o7 ------rwx False False
000001000 0o10 -----x--- True True
000001001 0o11 -----x--x True True... and so on. Last but not least, this Python impl. does not handle the Please let me know if you have thoughts on that @chrahunt . |
|
We don't need to feel like we have to do what We could replace this condition with something like: This has nice properties:
|
|
OK, I am more than happy to keep it simple 😉 @chrahunt. |
bsolomon1124
added a commit
to bsolomon1124/pip
that referenced
this issue
Mar 3, 2020
Closes Issue pypa#6364: use os.stat() rather than os.access() to correctly determine if srcfile has any executable bits set. os.access() will always return False in cases where the directory is mounted noexec, which can lead to a false negative and the resulting script not being executable.
nlhkabu
added
C: error messages
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Environment
pip version: pip 19.0.3
Python version: python 3.6
OS: Oracle Linux 7.6
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,relatime)
Description
Pip won't install a script as executable if /tmp is mounted as noexec. I have a wheel file that was created with setuptools. Pip unpacks it into /tmp and chmod the file to 755 which is correct and executable. However, when pip goes to copy the file from the tmp staging dir to the destination it uses access(file, X_OK) to check if the file is executable rather than looking at the mode flags. the access() returns "EACCES (permission denied)" and pip skips the chmod step.
Broken:
chmod("/tmp/pip-install-lzeaps2y/YYY/YYY-0.3.4.data/scripts/export", 0755) = 0 ... stat("/tmp/pip-install-lzeaps2y/YYY/YYY-0.3.4.data/scripts/export", {st_mode=S_IFREG|0755, st_size=3593, ...}) = 0 access("/tmp/pip-install-lzeaps2y/YYY/YYY-0.3.4.data/scripts/export", X_OK) = -1 EACCES (Permission denied) stat("/home/XXX/pt/bin/export", {st_mode=S_IFREG|0644, st_size=3593, ...}) = 0Working:
chmod("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", 0775) = 0 ... stat("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", {st_mode=S_IFREG|0775, st_size=3593, ...}) = 0 access("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", X_OK) = 0 stat("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", {st_mode=S_IFREG|0775, st_size=3593, ...}) = 0 chmod("/home/XXX/pt/bin/export", 0100775) = 0 stat("/home/XXX/pt/bin/export", {st_mode=S_IFREG|0775, st_size=3593, ...}) = 0Expected behavior
How to Reproduce
Enable tmpfs /tmp with noexec
mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime tmpfs /mntCreate a python wheel file with a script that just echos test
Output
Broken output shows the testbin is not +x
The text was updated successfully, but these errors were encountered: