Vendoring process fails on Windows

[pradyunsg]

Environment

• pip version: master
• Python version: 3.8
• OS: Windows 10

Description

Running tox -e vendoring fails on Windows.

The reason for the failure seems to be newline normalization -- see https://github.com/pypa/pip/commit/5563141d58bc17d5fab3bf4ca1649e136e25696c/checks/396942169/logs (from #7611). The patching process failed and vendoring (the tool) did not error out immediately. The git diff --exit-code catches the problem though. :)

Expected behavior

tox -e vendoring should work on all platforms.

How to Reproduce

1. Checkout pip's current master on Windows.
2. Run tox -e vendoring
3. An error occurs.

Output

From the logs linked above, the relevant lines are:

2020-01-18T18:29:27.6652039Z   Running git apply --verbose 'tools\automation\vendoring\patches\requests.patch'
2020-01-18T18:29:27.6652131Z     error: corrupt patch at line 7

[pfmoore]

I can take a look at this later this week, if no-one gets to it before me. @pradyunsg I see you self-assigned it - I'm fine if you prefer to deal with it yourself.

It may be as simple as marking the patch files as "binary" in .gitattributes to stop git using platform line endings in them.

[pradyunsg]

There's 2 things to do here:

1. Making vendoring fail when the patching fails (which is what I assigned myself for)
2. Properly handling line endings for these patches (please feel free to self assign this bit to yourself @pfmoore).

---

I'm not 100% sure what the problem with the line endings is - if it's the line endings changing, or it might be a difference of line ending between the two files. I'm pretty sure that you'd be much better at figuring this out than me. :)

[uranusjr]

This plus #7600 means we’ll need to figure out a universal setting for all these things. There are two settings affecting this: core.autocrlf and core.eol.

The “easiest” (in quotes) solution would be to punt and always use UNIX line endings (core.autocrlf=false, core.eol=lf). I can think of two downsides: we need to always write automation scripts with newline='\n', and some Windows folks using archaic editors may have problems editing the files. The solution at the other end of the spectrum is to always stick to the platform default, which would eliminate us from needing to stick newline='\n', but could have other problems (as the one we are seeing here).

[pfmoore]

I'd prefer to go with native line endings, as that's less likely to have odd side-effects (even though most editors work fine with \n line endings, they typically default to \r\n for new files).

Treating patch files as special makes more sense to me, because, well, they are - the patch utilities are what don't handle mixed line endings cleanly.

Actually, it looks like the problem is far simpler than all this - when we moved the patches from tasks to tools/automation, the .gitattributes file wasn't updated! I've just filed #7641 to fix this - it works on my PC, but if anyone else can test it as well, that would be great (I may not wait for confirmation from others before merging - it's the right change to make regardless).

[webknjaz]

It looks like this can be closed. Confirmed by rebasing #7611.

[pradyunsg]

Yup!