Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lib64 directory is no longer symlinked to lib? #1751

Closed
abn opened this issue Mar 26, 2020 · 4 comments
Closed

lib64 directory is no longer symlinked to lib? #1751

abn opened this issue Mar 26, 2020 · 4 comments
Labels
question

Comments

@abn
Copy link

abn commented Mar 26, 2020

Issue

Came across this issue while debugging tox environment creation issues. It would seem that when using virtualenv==16.7.10 or simply python -m venv, the lib64 directory was a symlink to the lib directory as expected. However, in virtualenv==20.0.14, this is no longer the case. Here, lib64 is created as a directory.

For example, the following will reproduce the issue.

docker run --rm -i fedora:31 <<EOF
set -x
dnf --quiet install -y pip
for v in 16.7.10 20.0.14; do
  pip install virtualenv==\$v >/dev/null 2>&1
  virtualenv venv-\$v
  ls -all venv-\$v
done
EOF

The following, is a snippet from the output.

+ pip install virtualenv==16.7.10
+ virtualenv venv-16.7.10
Using base prefix '/usr'
New python executable in /venv-16.7.10/bin/python3
Also creating executable in /venv-16.7.10/bin/python
Installing setuptools, pip, wheel...
done.
+ ls -all venv-16.7.10
total 28
drwxr-xr-x. 5 root root 4096 Mar 26 03:35 .
drwxr-xr-x. 1 root root 4096 Mar 26 03:35 ..
drwxr-xr-x. 2 root root 4096 Mar 26 03:36 bin
drwxr-xr-x. 2 root root 4096 Mar 26 03:35 include
drwxr-xr-x. 3 root root 4096 Mar 26 03:35 lib
lrwxrwxrwx. 1 root root    3 Mar 26 03:35 lib64 -> lib
...
+ pip install virtualenv==20.0.14
+ virtualenv venv-20.0.14
created virtual environment CPython3.7.6.final.0-64 in 249ms
  creator CPython3Posix(dest=/venv-20.0.14, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
+ ls -all venv-20.0.14
total 32
drwxr-xr-x. 5 root root 4096 Mar 26 03:36 .
drwxr-xr-x. 1 root root 4096 Mar 26 03:36 ..
drwxr-xr-x. 2 root root 4096 Mar 26 03:36 bin
drwxr-xr-x. 3 root root 4096 Mar 26 03:36 lib
drwxr-xr-x. 3 root root 4096 Mar 26 03:36 lib64
-rw-r--r--. 1 root root  202 Mar 26 03:36 pyvenv.cfg

Environment

Provide at least:

  • OS: Fedora release 31 (Thirty One), Linux 7ab2b964bf3a 5.5.10-200.fc31.x86_64 #1 SMP Wed Mar 18 14:21:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

  • pip list of the host python where virtualenv is installed:

    Package            Version
------------------ -------
appdirs            1.4.3  
distlib            0.3.0  
filelock           3.0.12 
gpg                1.13.1 
importlib-metadata 1.5.2  
libcomps           0.1.14 
pip                19.1.1 
rpm                4.15.1 
setuptools         41.6.0 
six                1.14.0 
virtualenv         20.0.14
zipp               3.1.0

Output of the virtual environment creation

Make sure to run the creation with -vvv --with-traceback:

75 setup logging to NOTSET [DEBUG report:43]
75 created app data folder /root/.local/share/virtualenv [DEBUG app_data:52]
84 find interpreter for spec PythonSpec(path=/usr/bin/python3) [INFO builtin:44]
84 proposed PythonInfo(spec=CPython3.7.6.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.7.6 (default, Jan 30 2020, 09:44:41) \n[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]', encoding_fs_io=utf-8-UTF-8) [INFO builtin:50]
84 accepted PythonInfo(spec=CPython3.7.6.final.0-64, exe=/usr/bin/python3, platform=linux, version='3.7.6 (default, Jan 30 2020, 09:44:41) \n[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]', encoding_fs_io=utf-8-UTF-8) [DEBUG builtin:52]
91 filesystem is case-sensitive [DEBUG info:28]
160 create virtual environment via CPython3Posix(dest=/venv-20.0.14, clear=False, global=False) [INFO session:52]
160 create folder /venv-20.0.14/bin [DEBUG _sync:21]
160 create folder /venv-20.0.14/lib/python3.7/site-packages [DEBUG _sync:21]
160 create folder /venv-20.0.14/lib64/python3.7/site-packages [DEBUG _sync:21]
161 write /venv-20.0.14/pyvenv.cfg [DEBUG pyenv_cfg:34]
161     home = /usr [DEBUG pyenv_cfg:38]
161     implementation = CPython [DEBUG pyenv_cfg:38]
161     version_info = 3.7.6.final.0 [DEBUG pyenv_cfg:38]
161     virtualenv = 20.0.14 [DEBUG pyenv_cfg:38]
161     include-system-site-packages = false [DEBUG pyenv_cfg:38]
161     base-prefix = /usr [DEBUG pyenv_cfg:38]
161     base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
161     base-executable = /usr/bin/python3 [DEBUG pyenv_cfg:38]
161 symlink /usr/bin/python3 to /venv-20.0.14/bin/python [DEBUG _sync:40]
162 create virtualenv import hook file /venv-20.0.14/lib/python3.7/site-packages/_virtualenv.pth [DEBUG api:76]
163 create /venv-20.0.14/lib/python3.7/site-packages/_virtualenv.py [DEBUG api:79]
163 ============================== target debug ============================== [DEBUG session:54]
163 debug via /venv-20.0.14/bin/python /usr/local/lib/python3.7/site-packages/virtualenv/create/debug.py [DEBUG creator:187]
163 {
  "sys": {
    "executable": "/venv-20.0.14/bin/python",
    "_base_executable": null,
    "prefix": "/venv-20.0.14",
    "base_prefix": "/usr",
    "real_prefix": null,
    "exec_prefix": "/venv-20.0.14",
    "base_exec_prefix": "/usr",
    "path": [
      "/usr/lib64/python37.zip",
      "/usr/lib64/python3.7",
      "/usr/lib64/python3.7/lib-dynload",
      "/venv-20.0.14/lib64/python3.7/site-packages",
      "/venv-20.0.14/lib/python3.7/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "UTF-8"
  },
  "version": "3.7.6 (default, Jan 30 2020, 09:44:41) \n[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)]",
  "os": "<module 'os' from '/usr/lib64/python3.7/os.py'>",
  "site": "<module 'site' from '/usr/lib64/python3.7/site.py'>",
  "datetime": "<module 'datetime' from '/usr/lib64/python3.7/datetime.py'>",
  "math": "<module 'math' from '/usr/lib64/python3.7/lib-dynload/math.cpython-37m-x86_64-linux-gnu.so'>",
  "json": "<module 'json' from '/usr/lib64/python3.7/json/__init__.py'>"
} [DEBUG session:55]
201 add seed packages via FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1) [INFO session:59]
202 Attempting to acquire lock 140367118139024 on /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/wheels.lock [DEBUG filelock:270]
202 Lock 140367118139024 acquired on /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/wheels.lock [INFO filelock:274]
203 get bundled wheel /usr/local/lib/python3.7/site-packages/virtualenv/seed/embed/wheels/pip-20.0.2-py2.py3-none-any.whl [DEBUG acquire:51]
204 get bundled wheel /usr/local/lib/python3.7/site-packages/virtualenv/seed/embed/wheels/setuptools-46.1.1-py3-none-any.whl [DEBUG acquire:51]
204 get bundled wheel /usr/local/lib/python3.7/site-packages/virtualenv/seed/embed/wheels/wheel-0.34.2-py2.py3-none-any.whl [DEBUG acquire:51]
209 install wheel from wheel /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/wheels/wheel-0.34.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
210 install setuptools from wheel /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/wheels/setuptools-46.1.1-py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
210 build install image to /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any of wheel-0.34.2-py2.py3-none-any.whl [DEBUG base:57]
210 install pip from wheel /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/wheels/pip-20.0.2-py2.py3-none-any.whl via CopyPipInstall [DEBUG via_app_data:46]
210 build install image to /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/setuptools-46.1.1-py3-none-any of setuptools-46.1.1-py3-none-any.whl [DEBUG base:57]
211 build install image to /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any of pip-20.0.2-py2.py3-none-any.whl [DEBUG base:57]
236 changing mode of /tmp/tmppzoh5m0f/wheel-3.7 to 755 [INFO util:566]
237 changing mode of /tmp/tmppzoh5m0f/wheel to 755 [INFO util:566]
238 changing mode of /tmp/tmppzoh5m0f/wheel3 to 755 [INFO util:566]
243 copy directory /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel to /venv-20.0.14/lib/python3.7/site-packages/wheel [DEBUG _sync:48]
256 copy directory /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.dist-info to /venv-20.0.14/lib/python3.7/site-packages/wheel-0.34.2.dist-info [DEBUG _sync:48]
262 copy /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/wheel-0.34.2-py2.py3-none-any/wheel-0.34.2.virtualenv to /venv-20.0.14/lib/python3.7/site-packages/wheel-0.34.2.virtualenv [DEBUG _sync:48]
263 changing mode of /venv-20.0.14/bin/wheel-3.7 to 755 [INFO util:566]
264 changing mode of /venv-20.0.14/bin/wheel to 755 [INFO util:566]
264 changing mode of /venv-20.0.14/bin/wheel3 to 755 [INFO util:566]
265 generated console scripts wheel-3.7 wheel wheel3 [DEBUG base:53]
296 changing mode of /tmp/tmp1h0cn9gf/easy_install to 755 [INFO util:566]
297 changing mode of /tmp/tmp1h0cn9gf/easy_install3 to 755 [INFO util:566]
297 changing mode of /tmp/tmp1h0cn9gf/easy_install-3.7 to 755 [INFO util:566]
306 copy /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools-46.1.1.virtualenv to /venv-20.0.14/lib/python3.7/site-packages/setuptools-46.1.1.virtualenv [DEBUG _sync:48]
307 copy directory /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/pkg_resources to /venv-20.0.14/lib/python3.7/site-packages/pkg_resources [DEBUG _sync:48]
313 copy /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/easy_install.py to /venv-20.0.14/lib/python3.7/site-packages/easy_install.py [DEBUG _sync:48]
313 copy directory /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools-46.1.1.dist-info to /venv-20.0.14/lib/python3.7/site-packages/setuptools-46.1.1.dist-info [DEBUG _sync:48]
317 copy directory /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/setuptools-46.1.1-py3-none-any/setuptools to /venv-20.0.14/lib/python3.7/site-packages/setuptools [DEBUG _sync:48]
344 changing mode of /venv-20.0.14/bin/easy_install to 755 [INFO util:566]
345 changing mode of /venv-20.0.14/bin/easy_install3 to 755 [INFO util:566]
345 changing mode of /venv-20.0.14/bin/easy_install-3.7 to 755 [INFO util:566]
345 generated console scripts easy_install easy_install-3.7 easy_install3 [DEBUG base:53]
383 changing mode of /tmp/tmp483fsq8a/pip-3.7 to 755 [INFO util:566]
383 changing mode of /tmp/tmp483fsq8a/pip3.7 to 755 [INFO util:566]
384 changing mode of /tmp/tmp483fsq8a/pip to 755 [INFO util:566]
384 changing mode of /tmp/tmp483fsq8a/pip3 to 755 [INFO util:566]
407 copy /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.virtualenv to /venv-20.0.14/lib/python3.7/site-packages/pip-20.0.2.virtualenv [DEBUG _sync:48]
408 copy directory /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip to /venv-20.0.14/lib/python3.7/site-packages/pip [DEBUG _sync:48]
460 copy directory /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/image/CopyPipInstall/pip-20.0.2-py2.py3-none-any/pip-20.0.2.dist-info to /venv-20.0.14/lib/python3.7/site-packages/pip-20.0.2.dist-info [DEBUG _sync:48]
461 changing mode of /venv-20.0.14/bin/pip-3.7 to 755 [INFO util:566]
461 changing mode of /venv-20.0.14/bin/pip3.7 to 755 [INFO util:566]
461 changing mode of /venv-20.0.14/bin/pip to 755 [INFO util:566]
462 changing mode of /venv-20.0.14/bin/pip3 to 755 [INFO util:566]
462 generated console scripts pip-3.7 pip3.7 pip3 pip [DEBUG base:53]
462 Attempting to release lock 140367118139024 on /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/wheels.lock [DEBUG filelock:315]
462 Lock 140367118139024 released on /root/.local/share/virtualenv/seed-app-data/v1.0.1/3.7/wheels.lock [INFO filelock:318]
462 add activators for Bash, CShell, Fish, PowerShell, Python, Xonsh [INFO session:65]
464 write /venv-20.0.14/pyvenv.cfg [DEBUG pyenv_cfg:34]
465     home = /usr [DEBUG pyenv_cfg:38]
465     implementation = CPython [DEBUG pyenv_cfg:38]
465     version_info = 3.7.6.final.0 [DEBUG pyenv_cfg:38]
465     virtualenv = 20.0.14 [DEBUG pyenv_cfg:38]
465     include-system-site-packages = false [DEBUG pyenv_cfg:38]
465     base-prefix = /usr [DEBUG pyenv_cfg:38]
465     base-exec-prefix = /usr [DEBUG pyenv_cfg:38]
465     base-executable = /usr/bin/python3 [DEBUG pyenv_cfg:38]
465 created virtual environment CPython3.7.6.final.0-64 in 392ms
  creator CPython3Posix(dest=/venv-20.0.14, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/root/.local/share/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator [WARNING __main__:21]
@abn abn added the bug label Mar 26, 2020
@gaborbernat
Copy link
Contributor

gaborbernat commented Mar 26, 2020 •
edited
Loading

@abn this is the intention and by design. The lib does not have to be linked to the lib64 directory. There are two separate library paths, called platlib and purelib. This might differ and virtualenv < 20 solution was to link them, and as such merge them into the same; I consider this a bad solution as you're mixing those two together, so you might end up with conflicts where tools expect them to be separate; so with virtualenv 20 we handle them separately, as most tools also do.

@gaborbernat gaborbernat added question and removed bug labels Mar 26, 2020
@abn
Copy link
Author

abn commented Mar 26, 2020 •
edited
Loading

@gaborbernat thanks for the clarification. I was unsure if this was by design. The confusion arose as the python3.7 -m venv behaviour was the same as with < 20. Will deal with both paths downstream.

@gaborbernat
Copy link
Contributor

Not sure who added the downstream changes; is it an OS patch or upstream CPython? I think the intention here is to keep them separate; that being said merging them can also have some merit 🤷‍♂ though I consider it more error-prone.

@abn
Copy link
Author

abn commented Mar 26, 2020

I am not entirely sure on that one. But I do agree that keeping them separate is probably the better choice here.

@abn abn changed the title regression: lib64 directory is no longer symlinked to lib lib64 directory is no longer symlinked to lib? Mar 26, 2020
@abn abn mentioned this issue Mar 26, 2020
Closed
3 tasks
@pypa pypa locked and limited conversation to collaborators Jan 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@abn @gaborbernat