Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pip list deprecation warning #187

Merged
merged 3 commits into from
Apr 25, 2017
Merged

Conversation

mal
Copy link
Contributor

@mal mal commented Mar 31, 2017

While working on another PR I noticed the following deprecation warning in the output:

[...]
Successfully installed pip-9.0.1
+ awk -F [ ()]+ $1 == "pip" { print $2; exit }
+ tac
+ tac
+ pip list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
+ [ 9.0.1 = 9.0.1 ]
[...]

To pre-empt this change I've moved the check over to using pip freeze. I think pip freeze --all is a better candidate for this check as it not only has an easier to parse explicitly defined format, but is also less likely to change.


As an aside, I do not understand the purpose of the |tac|tac| pipe in the affected line (maybe buffering?), but have left it in place as history seems to suggest it was intentionally added during the move to templates, unfortunately no explicit reasoning is given. In some cursory testing it appeared to make no difference, but if this is some sort of intentional trick I would very much like to understand it, if anyone is willing to share their insight. 😄

Copy link
Member

@tianon tianon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, just one minor nit! (and added some clarification about |tac|tac| below)

Overall, looks like pip freeze was what we wanted in the first place when we went for pip list -- I wonder though if we can duplicate the issue from #100 and make sure this still successfully catches it?

2.7/Dockerfile Outdated
&& rm /tmp/get-pip.py \
&& wget -O /tmp/get-pip.py 'https://bootstrap.pypa.io/get-pip.py' \
&& python2 /tmp/get-pip.py "pip==$PYTHON_PIP_VERSION" \
&& rm /tmp/get-pip.py \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like it was unintentionally added here -- these are kept indented so that diff between this version and the other versions is smaller (no whitespace changes, only removed lines, essentially).

2.7/Dockerfile Outdated
# https://github.com/docker-library/python/pull/100
&& [ "$(pip list |tac|tac| awk -F '[ ()]+' '$1 == "pip" { print $2; exit }')" = "$PYTHON_PIP_VERSION" ] \
&& [ "$(pip freeze --all |tac|tac| awk -F '==' '$1 == "pip" { print $2; exit }')" = "$PYTHON_PIP_VERSION" ] \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice -- pip freeze is indeed a much better solution. 👍

The |tac|tac| is a cheeky trick to ensure the entire output of pip freeze is read completely -- some applications get really grumpy when we do exit from awk too quickly and standard input is then closed (causing pip freeze to have standard output closed). It's easier to illustrate with curl+head:

$ curl -fsSL 'https://google.com' | head -1 > /dev/null
curl: (23) Failed writing body (867 != 1339)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aha! Thanks for the clarification 👍

@mal
Copy link
Contributor Author

mal commented Apr 5, 2017

Removed the 2.7 whitespace commit. I also had a brief look at replicating the issue in #100 but wasn't able to, due to ensurepip currently installing 9.0.1, the same version specified in the Dockerfile.

@mal
Copy link
Contributor Author

mal commented Apr 7, 2017

I just built a 3.5 image from e327311 (commit prior to the fix in #100) and was unable to reproduce the issue at all:

[... snip ...]
Successfully installed pip-7.1.2 setuptools-18.2
+ ldconfig
+ pip3 install --no-cache-dir --upgrade --ignore-installed pip==8.1.1
Collecting pip==8.1.1
  Downloading pip-8.1.1-py2.py3-none-any.whl (1.2MB)
Installing collected packages: pip
Successfully installed pip-8.1.1
[... snip ...]
# pip freeze
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
# pip3 freeze
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

I was expecting to see 7.1.2, however version 9.0.1 is available. in order for it to be consistent with the original report, but seeing 8.1.1 means the correct version for that image is both installed and recognised.

Digging a little deeper I found references to 7.1.2 but none that actively surfaced in the manner described in the report:

# pip3 -V      
pip 8.1.1 from /usr/local/lib/python3.5/site-packages (python 3.5)
# ls -l /usr/local/lib/python3.5/site-packages
total 40
-rw-r--r--  1 root staff  119 Apr  7 20:49 README
drwxr-sr-x  2 root staff 4096 Apr  7 20:49 __pycache__
drwxr-sr-x  3 root staff 4096 Apr  7 20:49 _markerlib
-rw-r--r--  1 root staff  126 Apr  7 20:49 easy_install.py
drwxr-sr-x 20 root staff 4096 Apr  7 20:50 pip
drwxr-sr-x  2 root staff 4096 Apr  7 20:49 pip-7.1.2.dist-info
drwxr-sr-x  2 root staff 4096 Apr  7 20:49 pip-8.1.1.dist-info
drwxr-sr-x  4 root staff 4096 Apr  7 20:49 pkg_resources
drwxr-sr-x  4 root staff 4096 Apr  7 20:49 setuptools
drwxr-sr-x  2 root staff 4096 Apr  7 20:49 setuptools-18.2.dist-info

At this point I'm at a bit of a loss. The proposed change makes sense in the context of updating for deprecation, but it seems like the reason for this check being present at all could use some further investigation, probably from someone with more familiarity with pip and python module internals than me I'm afraid 😞

@tianon
Copy link
Member

tianon commented Apr 10, 2017

Hmm, OK -- maybe then we should simply remove the check entirely and revisit if the problem comes back. Sound sane?

@mal
Copy link
Contributor Author

mal commented Apr 10, 2017

Think so, yes. If nothing else the --force-install option on the line above wasn't present at the time this existed as an issue, and I'd expect that to prevent it recurring. I'll update this PR 👍

mal added 3 commits April 10, 2017 18:34
Following discussion in docker-library#187 it was decided that
rather than refactor this check, it should be removed. This was because
the original error condition it was designed to prevent could not be
reproduced. This is done with the understanding that it may need to be
revisited if any recurrance (of docker-library#100) is
encountered in future.
@tianon tianon merged commit 089f8ce into docker-library:master Apr 25, 2017
tianon added a commit to infosiftr/stackbrew that referenced this pull request Apr 26, 2017
- `docker`: experimental multiarch (docker-library/docker#52)
- `golang`: experimental multiarch (docker-library/golang#158)
- `nextcloud`: 10.0.5, 11.0.3, 9.0.58, update via container updates (nextcloud/docker#65)
- `postgres`: ensure `postgres` user's HOME has decent permissions (docker-library/postgres#277)
- `python`: fix `pip` install to also install `setuptools` and `wheel` (docker-library/python#186, docker-library/python#187)
tao12345666333 pushed a commit to tao12345666333/python that referenced this pull request Jun 28, 2018
Following discussion in docker-library#187 it was decided that
rather than refactor this check, it should be removed. This was because
the original error condition it was designed to prevent could not be
reproduced. This is done with the understanding that it may need to be
revisited if any recurrance (of docker-library#100) is
encountered in future.
tao12345666333 pushed a commit to tao12345666333/python that referenced this pull request Jun 28, 2018
VenusPR added a commit to VenusPR/Python_Richard that referenced this pull request Mar 9, 2023
Following discussion in docker-library/python#187 it was decided that
rather than refactor this check, it should be removed. This was because
the original error condition it was designed to prevent could not be
reproduced. This is done with the understanding that it may need to be
revisited if any recurrance (of docker-library/python#100) is
encountered in future.
ProActiveSpirit pushed a commit to ProActiveSpirit/python that referenced this pull request Mar 1, 2024
Following discussion in docker-library/python#187 it was decided that
rather than refactor this check, it should be removed. This was because
the original error condition it was designed to prevent could not be
reproduced. This is done with the understanding that it may need to be
revisited if any recurrance (of docker-library/python#100) is
encountered in future.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants