Installation issue with Ubuntu 14.04 from wheel

[amitsaha]

I am facing a weird issue on Ubuntu (VERSION="14.04.5 LTS, Trusty Tahr") + Python 2.7.6. Installation completed successfully, but, the executable bit is not set on the aws script:

$ ls -lrt amit/bin/aws*
-rw-rw-r-- 1 ubuntu ubuntu 1807 Sep 14 02:29 amit/bin/aws_zsh_completer.sh
-rw-rw-r-- 1 ubuntu ubuntu 1432 Sep 14 02:29 amit/bin/aws.cmd
-rw-rw-r-- 1 ubuntu ubuntu  204 Sep 14 02:29 amit/bin/aws_bash_completer
-rw-rw-r-- 1 ubuntu ubuntu 1148 Sep 14 02:29 amit/bin/aws_completer
-rw-rw-r-- 1 ubuntu ubuntu  827 Sep 14 02:29 amit/bin/aws

The weird part is, I cannot reproduce the issue:

• On another Ubuntu 14.04 host with the same versions of pip, awscli, setuptools
• If I clone the repository and do a manual install
• If I don't use the wheel to install ( pip install awscli --no-binary :all:)

Relevant package versions:

awscli (1.11.152)
botocore (1.7.10)
colorama (0.3.7)
docutils (0.14)
futures (3.1.1)
jmespath (0.9.3)
pip (9.0.1)
pyasn1 (0.3.4)
python-dateutil (2.6.1)
PyYAML (3.12)
rsa (3.4.2)
s3transfer (0.1.11)
setuptools (36.4.0)
six (1.10.0)
wheel (0.30.0)

[stealthycoin]

Thats a really strange issue. It will be difficult to track down if you can't reproduce it. What is the affected machine's umask?

[amitsaha]

$ umask 
0002

This is same as another 14.04 host where everything worked :-(

[dstufft]

Could you give me the output of pip --version on the machine you're having issues on? I'm attempting to sort out what could be causing this and I want to double check if you're using a distro or pip provided pip.

[amitsaha]

@dstufft does the pip version installed in the virtual environment help? If so, as per my original post, it's pip (9.0.1)

[dstufft]

@amitsaha I was actually looking for the path part included in the output of pip --version, bur if you're in a virtual environment then that wouldn't help anyways.

Some more questions:

• Given you're in a virtual environment, are you able to reproduce this in a new virtual environment on the same machine?
• If yes to do the above, in that new virtual environment if you do pip install --upgrade --force-reinstall pip are you still able to reproduce it?

[amitsaha]

Given you're in a virtual environment, are you able to reproduce this in a new virtual environment on the same machine?

Yes, I could.

I will see if we still have the same machine alive and get back to you regarding the second question.

[thaiphv]

@dstufft, I tried your command in a new virtual environment but it still happened:

(newaws) ubuntu@ip-10-6-50-98:~/newaws$ pip install --upgrade --force-reinstall pip
Collecting pip
  Using cached pip-9.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
      Successfully uninstalled pip-9.0.1
Successfully installed pip-9.0.1
(newaws) ubuntu@ip-10-6-50-98:~/newaws$ pip install awscli
Collecting awscli
  Using cached awscli-1.11.155-py2.py3-none-any.whl
Collecting botocore==1.7.13 (from awscli)
  Using cached botocore-1.7.13-py2.py3-none-any.whl
Collecting PyYAML<=3.12,>=3.10 (from awscli)
/home/ubuntu/newaws/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/home/ubuntu/newaws/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Collecting colorama<=0.3.7,>=0.2.5 (from awscli)
  Using cached colorama-0.3.7-py2.py3-none-any.whl
Collecting docutils>=0.10 (from awscli)
  Using cached docutils-0.14-py2-none-any.whl
Collecting s3transfer<0.2.0,>=0.1.9 (from awscli)
  Using cached s3transfer-0.1.11-py2.py3-none-any.whl
Collecting rsa<=3.5.0,>=3.1.2 (from awscli)
  Using cached rsa-3.4.2-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.7.13->awscli)
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from botocore==1.7.13->awscli)
  Using cached jmespath-0.9.3-py2.py3-none-any.whl
Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" (from s3transfer<0.2.0,>=0.1.9->awscli)
  Using cached futures-3.1.1-py2-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa<=3.5.0,>=3.1.2->awscli)
  Downloading pyasn1-0.3.5-py2.py3-none-any.whl (63kB)
    100% |████████████████████████████████| 71kB 8.7MB/s 
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.7.13->awscli)
  Downloading six-1.11.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, docutils, jmespath, botocore, PyYAML, colorama, futures, s3transfer, pyasn1, rsa, awscli
Successfully installed PyYAML-3.12 awscli-1.11.155 botocore-1.7.13 colorama-0.3.7 docutils-0.14 futures-3.1.1 jmespath-0.9.3 pyasn1-0.3.5 python-dateutil-2.6.1 rsa-3.4.2 s3transfer-0.1.11 six-1.11.0
(newaws) ubuntu@ip-10-6-50-98:~/newaws$ ll bin/aws*
-rw-rw-r-- 1 ubuntu ubuntu  829 Sep 19 08:58 bin/aws
-rw-rw-r-- 1 ubuntu ubuntu  204 Sep 19 08:58 bin/aws_bash_completer
-rw-rw-r-- 1 ubuntu ubuntu 1432 Sep 19 08:58 bin/aws.cmd
-rw-rw-r-- 1 ubuntu ubuntu 1150 Sep 19 08:58 bin/aws_completer
-rw-rw-r-- 1 ubuntu ubuntu 1807 Sep 19 08:58 bin/aws_zsh_completer.sh
(newaws) ubuntu@ip-10-6-50-98:~/newaws$ 

[dstufft]

Could I get you to apply this patch to your copy of pip:

diff --git a/pip/wheel.py b/pip/wheel.py
index 9ac9dffe..d29736a9 100644
--- a/pip/wheel.py
+++ b/pip/wheel.py
@@ -330,11 +330,13 @@ def move_wheel_files(name, req, wheeldir, user=False, home=None, root=None,
 
                 # If our file is executable, then make our destination file
                 # executable.
+                logger.debug("%s has executable bit: %s", srcfile, os.access(srcfile, os.X_OK))
                 if os.access(srcfile, os.X_OK):
                     st = os.stat(srcfile)
                     permissions = (
                         st.st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH
                     )
+                    logger.debug("Chmodding %s to: %s", destfile, permissions)
                     os.chmod(destfile, permissions)
 
                 changed = False

And then run pip install -vvvv awscli, make sure that it installed without the executable permission, and then email me the logs that it produced to dstufft@amazon.com?

[thaiphv]

@dstufft I've sent the logs to your email address after running the command using your patch.

[dstufft]

Is your /tmp mounted noexec?

[thaiphv]

Yes, it is.

tmpfs on /tmp type tmpfs (rw,noexec,nosuid,nodev,noatime,mode=0777)

[dstufft]

Ok, the underlying issue here is the noexec, when pip unpacks the wheel it finds that none of the files have the executable bit (because of noexec) so it doesn't chmod the final files to make sure they are executable.

If you set the TMPDIR environment variable to another location it should work then, I've also sent #2844 which I believe should also fix this. Beyond that there's not a whole lot awscli can do here.

[thaiphv]

@dstufft, thanks a lot for your help.