Release PR: pip 20.0

[pradyunsg]

Filing this PR as a sanity check to make sure that the CI passes with the version bumped.

It contains the commit that'll be tagged as 20.0 -- don't merge this unless you're @pradyunsg.

[pradyunsg]

Looks like the Windows CI failed to checkout a git commit, after building the distributions properly. I don't think there's much point in exploring this right now but we should look into this at a later time.

For now, I'll go ahead with the rest of the do-the-release tasks.

2020-01-21T11:18:57.5608454Z ##[section]Starting: Generate distributions for the dummy release
2020-01-21T11:18:57.5708205Z ==============================================================================
2020-01-21T11:18:57.5708499Z Task         : Bash
2020-01-21T11:18:57.5708621Z Description  : Run a Bash script on macOS, Linux, or Windows
2020-01-21T11:18:57.5708784Z Version      : 3.159.3
2020-01-21T11:18:57.5708933Z Author       : Microsoft Corporation
2020-01-21T11:18:57.5709061Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2020-01-21T11:18:57.5709204Z ==============================================================================
2020-01-21T11:18:57.7926855Z Generating script.
2020-01-21T11:18:57.8148996Z Script contents:
2020-01-21T11:18:57.8149451Z nox -s build-release -- 99.9
2020-01-21T11:18:57.8208593Z [command]"C:\Program Files\Git\usr\bin\bash.exe" --noprofile --norc -c pwd
2020-01-21T11:18:57.8450636Z /d/a/_temp
2020-01-21T11:18:57.8462242Z 
2020-01-21T11:18:57.8500139Z ========================== Starting Command Output ===========================
2020-01-21T11:18:57.8507006Z [command]"C:\Program Files\Git\usr\bin\bash.exe" --noprofile --norc /d/a/_temp/d0287815-6339-4d68-83c4-7a41360360a2.sh
2020-01-21T11:18:58.0930723Z Running session build-release
2020-01-21T11:18:58.0957906Z Creating virtual environment (virtualenv) using python.exe in .nox\build-release
2020-01-21T11:19:05.8956998Z # Ensure no files in dist/
2020-01-21T11:19:05.8959021Z # Install dependencies
2020-01-21T11:19:05.8959573Z pip install setuptools wheel twine
2020-01-21T11:19:15.4692318Z # Checkout the tag
2020-01-21T11:19:15.4734270Z git checkout 99.9
2020-01-21T11:19:15.5168775Z # Build distributions
2020-01-21T11:19:15.5170864Z python setup.py sdist bdist_wheel
2020-01-21T11:19:18.1684405Z # Verify distributions
2020-01-21T11:19:18.1690550Z twine check dist\pip-99.9-py2.py3-none-any.whl dist\pip-99.9.tar.gz
2020-01-21T11:19:19.0073373Z # Checkout the master branch
2020-01-21T11:19:19.0073892Z git checkout master
2020-01-21T11:19:19.0331592Z Command git checkout master failed with exit code 1:
2020-01-21T11:19:19.0332570Z error: Your local changes to the following files would be overwritten by checkout:
2020-01-21T11:19:19.0333191Z 	AUTHORS.txt
2020-01-21T11:19:19.0333765Z Please commit your changes or stash them before you switch branches.
2020-01-21T11:19:19.0334377Z Aborting
2020-01-21T11:19:19.0334983Z Session build-release failed.
2020-01-21T11:19:19.0571448Z 
2020-01-21T11:19:19.0683330Z ##[error]Bash exited with code '1'.
2020-01-21T11:19:19.0700397Z ##[section]Finishing: Generate distributions for the dummy release

[pfmoore]

2020-01-21T11:19:19.0332570Z error: Your local changes to the following files would be overwritten by checkout:
2020-01-21T11:19:19.0333191Z AUTHORS.txt

Line ending issue?

Why are we running these steps under git bash? That sounds risky, as it's a msys (cygwin-like) environment and could well behave differently than "native" Windows. (I don't necessarily think that's the issue here, I just dislike using msys/cygwin as a general rule).

[honnix]

I think this release has issue.

pip --version
pip 20.0 from /Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip (python 3.7)

$ pip list
Traceback (most recent call last):
  File "/Users/honnix/.virtualenvs/test-pip/bin/pip", line 8, in <module>
    sys.exit(main())
  File "/Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/cli/main.py", line 73, in main
    command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
  File "/Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/commands/__init__.py", line 96, in create_command
    module = importlib.import_module(module_path)
  File "/Users/honnix/.virtualenvs/test-pip/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/commands/list.py", line 13, in <module>
    from pip._internal.cli.req_command import IndexGroupCommand
  File "/Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 20, in <module>
    from pip._internal.operations.prepare import RequirementPreparer
  File "/Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 16, in <module>
    from pip._internal.distributions import (
  File "/Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/distributions/__init__.py", line 1, in <module>
    from pip._internal.distributions.source import SourceDistribution
ImportError: cannot import name 'SourceDistribution' from 'pip._internal.distributions.source' (/Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/distributions/source/__init__.py)

ls /Users/honnix/.virtualenvs/test-pip/lib/python3.7/site-packages/pip/_internal/distributions

__init__.py  __pycache__  base.py      installed.py source       source.py    wheel.py

Is it intended that both source directory and source.py exist?

[pfmoore]

Line ending issue?

@pradyunsg Looking at this issue, I'm pretty sure it is a line ending issue.

1. tools.automation.release.generate_authors() writes AUTHORS.txt with an explicit \n line ending.
2. On Windows, by default git has core.autocrlf=true, so that when checked out, AUTHORS.txt will have \r\n line endings.
3. Nox (in prepare_release) checks for changed files using git diff --no-patch --exit-code. Because of autocrlf, this will not notice the difference between \r\n and \n line endings (it will write a warning, but won't treat the situation as "files have changed").
   4 However, git does consider the file as modified. This is probably what results in the "your local changes will be overwritten" message.

It looks like this was an unintended consequence of #7600. That PR was addressing people running with the unusual setting of core.autocrlf=false, so maybe it's better to revert it and find a better solution. Could we maybe have a per-repository override that forces pip to always use autocrlf=true? (I don't now enough git to know if that's an option). @uranusjr what do you think (as you were the one affected by the original issue)?

I haven't been able to recreate the issue locally, yet, as it needs a combination of a Unix machine and a Windows one, and I don't have a Unix VM set up right now. But I'm pretty sure the analysis above is accurate.

[pradyunsg]

Ok, great; it's exactly what I thought it was -- a non-critical failure due to some interaction of Windows+git+CI on newlines. :)

---

#7600 was squash merged, so anyone who wants to file a PR reverting it only needs to do git revert <that-commit-hash>.

[uranusjr]

Setting per-repository config (and reverting the changes in the Nox script) sounds like a good solution to me. I’m guessing it would also benefit #7617 as well. Continuing the discussion there.