-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Packages installed by pip in lib64 are not recognized by pip #4464
Comments
I am seeing this exact issue as well on Amazon Linux. pip is installing everything to So my questions are:
|
Probably a long shot, but maybe @dstufft can ping someone at Amazon to take a look at this since this seems to be limited to the combination of pip + Amazon Linux. |
This is likely an issue with the python install provided by Amazon Linux as pip "asks" (via distutils) the Python installation where it should install packages, cf https://github.com/pypa/pip/blob/9.0.1/pip/locations.py#L124-L182 I'd love to see the output of :
|
@xavfernandez Here's the output on my EC2 instance:
|
Same problem here with pip 9.0.1 and python 3.6. My information for what @xavfernandez asked for is substantially similar to what @jpotterm supplied. I too was able to work around the problem by using "pip install --upgrade --target /path/to/my/sandbox/lib/python3.6/dist-packages". That seems to force everything into lib that was once isntalled in lib64. Now pip and python can find those packages. |
I think that this python issue is probably related: https://bugs.python.org/issue1294959 And perhaps the Amazon image includes a version of the fedora patch that they mention, since platlib for us is pointing to /lib64/: http://pkgs.fedoraproject.org/cgit/rpms/python3.git/plain/00102-lib64.patch |
I was having this same issue, and initially worked around it by forcing everything into the /lib/ directory as @morrone suggested, but one application in particular started having namespace conflicts when I did that. I was able to resolve that by adding the /lib64/Python3.5/dist-packages directory to my $PYTHONPATH in the .bash_profile, and that seems to have fixed everything. Just wish I had come to this solution earlier. I'll be testing upgrading our AMI to the 2017.09 release in the near future (as described here) -- hopefully that includes the fedora patch, will update if it resolves this. |
@data-wrangler I am in the same place as you - I upgraded Amazon Linux to latest version and now it installs a couple of packages into /opt/python/run/venv/lib64/python2.7/site-packages/ in ElasticBeanstalk. How did you solve this issue? |
@axper I didn't find subsequent AMI updates affected the issue, I fixed it with adding the missing directory to the |
I was able to solve the issue in Lambda by running |
unset PYTHON_INSTALL_LAYOUT That's a neat solution. Seems to be the most lightweight one proposed here. Something else for people to consider: I just tried Amazon Linux 2 and it appears to solve this problem. The Amazon Linux 2 AMI IDs are listed here. [ec2-user@ip-10-97-56-131 ~]$ python3 -m venv venv
[ec2-user@ip-10-97-56-131 ~]$ source venv/bin/activate
(venv) [ec2-user@ip-10-97-56-131 ~]$ pip install -U pip
<snipped>
(venv) [ec2-user@ip-10-97-56-131 ~]$ pip --version
pip 10.0.1 from /home/ec2-user/venv/lib64/python3.7/site-packages/pip (python 3.7)
(venv) [ec2-user@ip-10-97-56-131 ~]$ pip install numpy jellyfish psycopg2 requests sqlparse
<snipped>
(venv) [ec2-user@ip-10-97-56-131 ~]$ pip list
Package Version
---------- ---------
certifi 2018.4.16
chardet 3.0.4
idna 2.7
jellyfish 0.6.1
numpy 1.14.5
pip 10.0.1
psycopg2 2.7.5
regex 2018.6.21
requests 2.19.1
setuptools 38.4.0
sqlparse 0.2.4
urllib3 1.23
(venv) [ec2-user@ip-10-97-56-131 ~]$ pip show psycopg2
Name: psycopg2
Version: 2.7.5
Summary: psycopg2 - Python-PostgreSQL Database Adapter
Home-page: http://initd.org/psycopg/
Author: Federico Di Gregorio
Author-email: fog@initd.org
License: LGPL with exceptions or ZPL
Location: /home/ec2-user/venv/lib/python3.7/site-packages
Requires:
Required-by:
(venv) [ec2-user@ip-10-97-56-131 ~]$ pip show jellyfish
Name: jellyfish
Version: 0.6.1
Summary: a library for doing approximate and phonetic matching of strings.
Home-page: http://github.com/jamesturk/jellyfish
Author: UNKNOWN
Author-email: UNKNOWN
License: UNKNOWN
Location: /home/ec2-user/venv/lib/python3.7/site-packages
Requires:
Required-by: So it looks like everything is now being installed to |
To test whether the fix was from Python 3.7 or from Amazon Linux, I tried the above test with Python 3.4, Amazon Linux 2, pip 10.0.1, and psycopg2, which @paulcyr had reported in the initial post as problematic. It still worked, which suggests to me that the fix is coming specifically from Amazon Linux 2. |
It looks like the issue was coming from Amazon Linux and is now fixed. |
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Description:
I am working in a virtual environment created with virtualenv. There are a few packages that I am trying to install, however two of them,
psycopg2
andpillow
, are not working correctly.When I run
pip install <package>
, it works as expected. However, neither of the two modules are shown usingpip show <package>
orpip list
.pip install <package>
can be run again with the same result.It turns out that the two packages were being installed in
env/lib64/python3.5/dist-packages
. All the other packages were being installed inenv/lib/python3.5/dist-packages
For some reason, pip is not seeing the packages it has installed in the lib64 folder. Packages installed in the lib folder are recognized correctly.
What I've run:
The text was updated successfully, but these errors were encountered: