Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Permissions issue with setup.py sdist bdist_wheel #258

Open
jonhue opened this issue May 17, 2019 · 23 comments
Open

Permissions issue with setup.py sdist bdist_wheel #258

jonhue opened this issue May 17, 2019 · 23 comments

Comments

@jonhue
Copy link

jonhue commented May 17, 2019

Thank you for providing feedback on Python packaging!

To help us help you, please fill out as much of the following as you can. If a question is not relevant, feel free to skip it.

  1. What is your operating system and version?

Ubuntu 18.04

  1. What is your Python version?

Python 3.6.7

  1. What version of pip do you have?

pip 9.0.1

  1. Could you describe your issue in as much detail as possible?

I was following this guide to locally package install my project. I installed the latest versions of setuptools and wheel (python3 -m pip install --user --upgrade setuptools wheel) and then tried to generate distribution archives. This step is where my problem lies. When running sudo python3 setup.py sdist bdist_wheel, I get a lot of Copying ... and then

running install_scripts 
error: [('build/bdist.linux-x86_64/wheel/few-0.1.0a0-py3.6.egg-info', 'build/bdist.linux-x86_64/wheel/few-0.1.0a0.dist-info', "[Errno 13] Permission denied: 'build/bdist.linux-x86_64/wheel/few-0.1.0a0.dist-info'")]

Do you have an idea what exactly the problem is?

@jonhue
Copy link
Author

jonhue commented May 17, 2019

I was able to successfully run the same command a month ago. Don't know what changed though...

@dmtucker
Copy link

@jonhue Any chance you are running this on an NFS-mounted filesystem? Why are you using sudo?

@jonhue
Copy link
Author

jonhue commented May 17, 2019

@dmtucker No, not using NFS. I used sudo because when not using it, I got an error saying that permission was denied trying to access a log file.

@CyrusNajmabadi
Copy link

CyrusNajmabadi commented Jun 5, 2019

I'm getting a very similar issue.

In our case, it's:

pipenv run python setup.py build bdist_wheel --universal

// a lot of copying

running install_scripts
error: [('build/bdist.linux-x86_64/wheel/pulumi-0.17.16.dev1559756548-py3.6.egg-info', 'build/bdist.linux-x86_64/wheel/pulumi-0.17.16.dev1559756548.dist-info', "[Errno 13] Permission denied: 'build/bdist.linux-x86_64/wheel/pulumi-0.17.16.dev1559756548.dist-info'")]

I'm not running sudo. The build/bdist.linux-x86_64/wheel/pulumi-0.17.16.dev1559756548.dist-info directory exists and is fully owned by me. I'm not on NFS.

  1. What is your operating system and version?

Ubuntu 18.04

  1. What is your Python version?

Python 3.6.7

  1. What version of pip do you have?

pip 19.0.1

@ryderjgillen
Copy link

Yup. Seems this is indeed broken... even running as full ROOT in home directory fails. Epic fail... :(

Ubuntu/Python 3.6.7/Pip 19.0.1

@pradyunsg
Copy link
Member

Hey @pganssle, any idea what's happening here?

@pganssle pganssle changed the title Trouble following packaging libraries tutorial Permissions issue with setup.py sdist bdist_wheel Jun 29, 2019
@pganssle
Copy link
Member

pganssle commented Jun 29, 2019

@pradyunsg No idea. In the past I have found that most permissions issues are highly system-specific.

If possible, I recommend adding a pyproject.toml file and using pep517.build to build your sdist and wheels.

In any case, there does not seem to be enough information in this ticket to recreate the issue. Please consider crafting a minimal reproducible example. I am guessing that the original poster's "few" package has a naming collision with this project, since the author seems to be different, and pulumi only seems to ship wheels and the github link points to a project written in Go, so I can't even try reproducing either of these problems.

@CyrusNajmabadi If you are involved in the "pulumi" project, you should also note that I think you have your find_packages misconfigured, because the latest pulumi wheel installs a package called "test" in addition to the package called "pulumi" (Edit: Created an issue for this.)

Edit: I figured out where the Python pulumi is kept, but I can't reproduce the issue. python setup.py build bdist_wheel works just fine for me.

@ryderjgillen
Copy link

ryderjgillen commented Jun 29, 2019

Thanks for looking into this. I can provide a MRE in the next day or so (hopefully via a Docker image)

As it currently stands I can provide these additional details about the environment that produced the error...

I was attempting to build out a new developer laptop so at the time my system was rather 'clean'.

I DO NOT have Python2.x installed
Python 3.6.7 is ALSO installed for Windows

Ubuntu 18.04 via WSL on Windows 10 Enterprise, 1903.
apt-get install -y python3
/usr/bin/python3 -m pip install --user --upgrade setuptools wheel
/usr/bin/python3 setup.py sdist --format gztar bdist_wheel

  • The user running the commands owns all the files and has full admin privileges. CWD = $HOME
  • I tried running directly as ROOT Account
  • I tried sudo using default admin account
  • I tried custom user with full ownership

All above scenarios result in same outcome.... however if the user running the packaging command does not own the files/ does not have write access to the folders (or sudo) it fails almost immediately.

Upon detailed inspection of the folder perms after error it seems the final folders do not have write perms. All above folders in the tree DO have write perms. I tried explicitly changing the perms and rerunning the packaging command but it seems the permissions are reset and the error continues to occur.

  • build/bdist.linux-x86_64/wheel/mypackage-1.0.0.dist-info == dr-xr-xr-x
  • build/bdist.linux-x86_64/wheel/mypackage-1.0.0.egg-info == dr-xr-xr-x

@pganssle
Copy link
Member

pganssle commented Jun 29, 2019

If I am correct that @jonhue was trying to build this few-lang repo, then I notice that he also erroneously is creating a package named test, because he's not using a src layout.

@rgillen Can you share the repo that this is failing for? I suspect that this permissions issue is arising because you're trying to build a package called "test". Not sure why that would cause a problem, but it seems like a pretty big coincidence (and easy enough to fix).

@ryderjgillen
Copy link

ryderjgillen commented Jun 29, 2019

@pganssle - I am not trying to build a package named 'test', the package is named 'cogid'. It is a wrapper for boto3 Coginito Identity.

I was able to get the commands to work by running them out of my $HOME directory. Interestingly I tried this last night as ROOT and it didn't work (it was very late, I could have made an error in my test case).

I compared the permissions on the 2 folder structures and they are identical. The only difference is one is a WSL.config mount to /e and the other is the WSL internal path @ \\wsl$\Ubuntu\home\svt

So in summary...

Fails
w/CWD = /e/Git/Repos/pypi/cogid (mapped via: wsl.config [automount] ROOT = /)

Succeeds
w/CWD = ~/cogid (maps to: \\wsl$\Ubuntu\home\svt\cogid)

Not sure if this is relevant but I am running Windows 1903 which had an update for WSL file-system access: https://devblogs.microsoft.com/commandline/whats-new-for-wsl-in-windows-10-version-1903/

Output of mount -l
rootfs on / type lxfs (rw,noatime)
none on /dev type tmpfs (rw,noatime,mode=755)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,noatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,noatime)
devpts on /dev/pts type devpts (rw,nosuid,noexec,noatime,gid=5,mode=620)
none on /run type tmpfs (rw,nosuid,noexec,noatime,mode=755)
none on /run/lock type tmpfs (rw,nosuid,nodev,noexec,noatime)
none on /run/shm type tmpfs (rw,nosuid,nodev,noatime)
none on /run/user type tmpfs (rw,nosuid,nodev,noexec,noatime,mode=755)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
cgroup on /sys/fs/cgroup type tmpfs (rw,relatime,mode=755)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,relatime,devices)
C:\ on /c type drvfs (rw,noatime,uid=1000,gid=1000,metadata,case=off)
D:\ on /d type drvfs (rw,noatime,uid=1000,gid=1000,metadata,case=off)
E:\ on /e type drvfs (rw,noatime,uid=1000,gid=1000,metadata,case=off)

Next I will try and run some tests on a Ubuntu Droplet in Digital Ocean and compare the results.

@pganssle
Copy link
Member

I am not trying to build a package named 'test', the package is named 'cogid'. It is a wrapper for boto3 Coginito Identity.

I don't think you or anyone in this thread is intentionally trying to build a package called test. I know the other two people are doing it by accident. Try building a wheel and opening the wheel file (it's a zip file) - if you're not accidentally building any additional packages, you'll only see 2 folders, a cogid and a cogid-XXX.dist-info folder. If you see a folder called test or examples or tests or anything like that, you're accidentally building an extra package.

@Mimetis
Copy link

Mimetis commented Jul 24, 2019

I have the exact same issue.
Trying to build a simple whl package results in the same kind of error.
I'm using Ubuntu through WSL on my Windows 10 machine (up to date)

My setup.py is pretty straightforward:

# -*- coding: utf-8 -*-

from setuptools import setup, find_packages

setup(name='southridge',
      version='0.9',
      description='Cleanse southridge datas',
      url='http://github.com/Mimetis',
      author='Sébastien Pertus',
      author_email='spertus@microsoft.com',
      license='MIT',
      packages=['southridge'],
      zip_safe=False)

and the result when I tried to build a bdist_wheel is:

spertus@MSI2019:/mnt/c/PROJECTS/DATA/sp/src$ sudo python3 setup.py bdist_wheel --universal
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
running install_egg_info
running egg_info
writing top-level names to southridge.egg-info/top_level.txt
writing dependency_links to southridge.egg-info/dependency_links.txt
writing southridge.egg-info/PKG-INFO
reading manifest file 'southridge.egg-info/SOURCES.txt'
writing manifest file 'southridge.egg-info/SOURCES.txt'
removing 'build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info' (and everything under it)
Copying southridge.egg-info to build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info
running install_scripts
error: [('build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info', 'build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info', "[Errno 13] Permission denied: 'build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info'")]

I then tried to copy my folder into my /home folder and it worked here....

spertus@MSI2019:/mnt/c/PROJECTS/DATA/sp$ sudo cp -r /mnt/c/PROJECTS/DATA/sp /home/sp
spertus@MSI2019:/home/sp/src$ sudo python3 setup.py bdist_wheel --universal
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
running install_egg_info
running egg_info
writing southridge.egg-info/PKG-INFO
writing dependency_links to southridge.egg-info/dependency_links.txt
writing top-level names to southridge.egg-info/top_level.txt
reading manifest file 'southridge.egg-info/SOURCES.txt'
writing manifest file 'southridge.egg-info/SOURCES.txt'
removing 'build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info' (and everything under it)
Copying southridge.egg-info to build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info/WHEEL

finally a last test in the folder where the build is not working (/mnt/c/DATA.....) where i tried to buils a sdist instead of bdist_wheel seems to work !

spertus@MSI2019:/mnt/c/PROJECTS/DATA/sp$ sudo python3 setup.py build sdist
running build
running build_py
running sdist
running egg_info
writing dependency_links to southridge.egg-info/dependency_links.txt
writing southridge.egg-info/PKG-INFO
writing top-level names to southridge.egg-info/top_level.txt
reading manifest file 'southridge.egg-info/SOURCES.txt'
writing manifest file 'southridge.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt

running check
creating southridge-0.9
creating southridge-0.9/southridge
creating southridge-0.9/southridge.egg-info
making hard links in southridge-0.9...
hard linking setup.py -> southridge-0.9
hard linking southridge/__init__.py -> southridge-0.9/southridge
hard linking southridge/cleanse.py -> southridge-0.9/southridge
hard linking southridge.egg-info/PKG-INFO -> southridge-0.9/southridge.egg-info
hard linking southridge.egg-info/SOURCES.txt -> southridge-0.9/southridge.egg-info
hard linking southridge.egg-info/dependency_links.txt -> southridge-0.9/southridge.egg-info
hard linking southridge.egg-info/not-zip-safe -> southridge-0.9/southridge.egg-info
hard linking southridge.egg-info/top_level.txt -> southridge-0.9/southridge.egg-info
Writing southridge-0.9/setup.cfg
creating dist
Creating tar archive
removing 'southridge-0.9' (and everything under it)

Finally, i even tried to re-install my ubuntu distribution (tried 16 & 18) but the results still remains the same ....

@Mimetis
Copy link

Mimetis commented Jul 25, 2019

I made some others tests.

Another way to make it working, is to move everything to the folder inside the Current User folder.

I've tested it, and it worked as expected

$ cp -r /mnt/c/PROJECTS/DATA/sp /mnt/c/Users/spertus.EUROPE/Documents
$ cd /mnt/c/Users/spertus.EUROPE/Documents/sp/src
$ python setup.py bdist_wheel

...
# package generated as expected
...

I've checked the permissions on every folders (even the generated ones) in each directories, and the results are always the same

$ cd /mnt/c/Users/spertus.EUROPE/Documents/sp/src
$ ls -l
drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:11 build
-rwxrwxrwx 1 spertus spertus  364 Jul 25 09:11 setup.py
drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:11 southridge
drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:11 southridge.egg-info

$ cd /mnt/c/PROJECTS/DATA/sp/src
$ ls -l
 drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:10 build
-rwxrwxrwx 1 spertus spertus  364 Jul 24 23:34 setup.py
drwxrwxrwx 1 spertus spertus 4096 Jul 24 23:45 southridge
drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:10 southridge.egg-info

According to the error, it seems I have no permissions on [Errno 13] Permission denied: 'build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info'"
But my permissions seems to be good.

spertus@MSI2019:/mnt/c/PROJECTS/DATA/sp/src/build/bdist.linux-x86_64/wheel$ ls -l
total 0
drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:10 southridge
drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:10 southridge-0.9.dist-info
drwxrwxrwx 1 spertus spertus 4096 Jul 25 09:10 southridge-0.9.egg-info

spertus@MSI2019:/mnt/c/PROJECTS/OPENHACK/DATA/sp/src/build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info$ ls -l
total 0
-rwxrwxrwx 1 spertus spertus  1 Jul 25 09:10 dependency_links.txt
-rwxrwxrwx 1 spertus spertus 11 Jul 25 09:10 top_level.txt

Continuing my investigations...
Any advice would be greatly appreciated !

@Mimetis
Copy link

Mimetis commented Jul 25, 2019

Adding a full control to Authenticated Users seems to resolve the issue

image

After adding this new permissions, everything worked correctly, finally !

spertus@MSI2019:/mnt/c/PROJECTS/DATA/SouthridgeProject/src$ python3 setup.py bdist_wheel --universal clean -a
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/cleanse.py -> build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/__init__.py -> build/bdist.linux-x86_64/wheel/southridge
running install_egg_info
running egg_info
writing top-level names to southridge.egg-info/top_level.txt
writing dependency_links to southridge.egg-info/dependency_links.txt
writing southridge.egg-info/PKG-INFO
reading manifest file 'southridge.egg-info/SOURCES.txt'
writing manifest file 'southridge.egg-info/SOURCES.txt'
Copying southridge.egg-info to build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info/WHEEL
running clean
removing 'build/lib' (and everything under it)
removing 'build/bdist.linux-x86_64' (and everything under it)
'build/scripts-3.5' does not exist -- can't clean it
removing 'build'

@blairg23
Copy link

blairg23 commented Aug 7, 2019

Adding a full control to Authenticated Users seems to resolve the issue

image

After adding this new permissions, everything worked correctly, finally !

spertus@MSI2019:/mnt/c/PROJECTS/DATA/SouthridgeProject/src$ python3 setup.py bdist_wheel --universal clean -a
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/cleanse.py -> build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/__init__.py -> build/bdist.linux-x86_64/wheel/southridge
running install_egg_info
running egg_info
writing top-level names to southridge.egg-info/top_level.txt
writing dependency_links to southridge.egg-info/dependency_links.txt
writing southridge.egg-info/PKG-INFO
reading manifest file 'southridge.egg-info/SOURCES.txt'
writing manifest file 'southridge.egg-info/SOURCES.txt'
Copying southridge.egg-info to build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info/WHEEL
running clean
removing 'build/lib' (and everything under it)
removing 'build/bdist.linux-x86_64' (and everything under it)
'build/scripts-3.5' does not exist -- can't clean it
removing 'build'

Can confirm this fixes the issue using Ubuntu on the Linux for Windows Subsystem. I swear, I have had more issues with Windows permissions when developing using Windows... Hoped Linux for Windows Subsystem would make these issues go away...

@Aperion
Copy link

Aperion commented Oct 9, 2019

Another confirmation that the above fix indeed works. It worked fine under windows, but in the WSL instance in the same directory I would get the permission denied error. Adding full control to Authenticated Users solved. I encountered this issue while working with scikit-build

@glinders
Copy link

A workaround that worked for me on WSL is to add option '--bdist-dir ~/temp/bdistwheel'

@hellzxmaker
Copy link

When I go to add full permissions to my project dir, I am getting an error that I am not authorized to change the permissions of the egg-info file. Has anyone been able to work around this?

@cbr-repo
Copy link

cbr-repo commented Mar 9, 2020

Simply re-installed wheel package and the command worked fine.

  1. Install wheel
    pip install wheel
  2. python setup.py build bdist_wheel

@cbr-repo
Copy link

cbr-repo commented Mar 9, 2020

Adding a full control to Authenticated Users seems to resolve the issue

image

After adding this new permissions, everything worked correctly, finally !

spertus@MSI2019:/mnt/c/PROJECTS/DATA/SouthridgeProject/src$ python3 setup.py bdist_wheel --universal clean -a
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/cleanse.py -> build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/__init__.py -> build/bdist.linux-x86_64/wheel/southridge
running install_egg_info
running egg_info
writing top-level names to southridge.egg-info/top_level.txt
writing dependency_links to southridge.egg-info/dependency_links.txt
writing southridge.egg-info/PKG-INFO
reading manifest file 'southridge.egg-info/SOURCES.txt'
writing manifest file 'southridge.egg-info/SOURCES.txt'
Copying southridge.egg-info to build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info/WHEEL
running clean
removing 'build/lib' (and everything under it)
removing 'build/bdist.linux-x86_64' (and everything under it)
'build/scripts-3.5' does not exist -- can't clean it
removing 'build'

Here is a simpler solution.
#258 (comment)

@SsmallWinds
Copy link

Adding a full control to Authenticated Users seems to resolve the issue

image

After adding this new permissions, everything worked correctly, finally !

spertus@MSI2019:/mnt/c/PROJECTS/DATA/SouthridgeProject/src$ python3 setup.py bdist_wheel --universal clean -a
running bdist_wheel
running build
running build_py
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/cleanse.py -> build/bdist.linux-x86_64/wheel/southridge
copying build/lib/southridge/__init__.py -> build/bdist.linux-x86_64/wheel/southridge
running install_egg_info
running egg_info
writing top-level names to southridge.egg-info/top_level.txt
writing dependency_links to southridge.egg-info/dependency_links.txt
writing southridge.egg-info/PKG-INFO
reading manifest file 'southridge.egg-info/SOURCES.txt'
writing manifest file 'southridge.egg-info/SOURCES.txt'
Copying southridge.egg-info to build/bdist.linux-x86_64/wheel/southridge-0.9.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/southridge-0.9.dist-info/WHEEL
running clean
removing 'build/lib' (and everything under it)
removing 'build/bdist.linux-x86_64' (and everything under it)
'build/scripts-3.5' does not exist -- can't clean it
removing 'build'

its works well!

@unfor19
Copy link

unfor19 commented Oct 31, 2020

Embarrassed to admit, but my issue was - tar.gz file was occupied by 7-Zip, so sdist couldn't really re-create it 🤦🏻

@mwilcox12014
Copy link

Environment: Windows 11, Conda 22.9.0, pip 22.2.2, python 3.9.13

Need modules from Allied Vision\Vimba_6.0 software
Run vimba setup.exe

C: pip -v install .
....
'"'"''"'"''"'"' % ('"'"'C:\Program Files\Allied Vision\Vimba_6.0\VimbaPython\Source\setup.py'"'"',), "", "exec"))' bdist_wheel -d 'C:\Users\SOD\AppData\Local\Temp\pip-wheel-wjyrgxcs'
cwd: C:\Program Files\Allied Vision\Vimba_6.0\VimbaPython\Source
Building wheel for VimbaPython (setup.py) ... error

Issue is Access denied on: C:\Program Files\Allied Vision\Vimba_6.0\VimbaPython\Source

As with others, Changing 'Users' permissions to 'Allow ' on Full Control fixed the problem.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests