Wheel scripts don't get command line wrappers on Windows

[takluyver]

When installing a wheel on Windows, if it specifies setuptools console_scripts or gui_scripts entry points, pip creates an exe wrapper so that it can be directly invoked at the command line. Scripts supplied in (data)/scripts/ with a #!python shebang, as described in PEP 427, however, do not get such a wrapper. Generating them is a recommend feature for installers according to the PEP.

I'm happy to work on this, since I'm creating a wheel-building tool that puts scripts in the (data)/scripts/ directory.

Side note: at least for console scripts, is there a good reason to wrap them in a .exe rather than a .cmd file, which would be easier to inspect? In my experience, writing a script.cmd file that does "path\to\python.exe" "%~dp0\script" serves to make a Python script executable from the command line.

Crosslink pypa/flit#1.

[pfmoore]

Regarding wrapping in an exe, yes it is important to do this. I tried to explain why in http://paul-moores-notes.readthedocs.org/en/latest/wrappers.html but basically Windows doesn't treat anything other than .exe files as first class citizens, so you'll hit problems somewhere if you use anything else. (The problem cases may not be important to you, and often aren't, but it matters for a general tool like pip).

As regards {data}/scripts scripts, they are typically "legacy" usage, and generally aren't recommended these days (I'd be hard pressed to find anything that specifically says this, but it's a common theme in discussions). I don't see any problem with adding support like you mention.

Just as a note, on Windows the "correct interpreter" should be written with double quotes if needed to protect any spaces in the filename, and should be encoded in UTF-8 assuming there's no BOM in the file (see https://hg.python.org/cpython/file/1aad901e0307/PC/launcher.c#l1140). On Unix it should probably use sys.getfilesystemencoding() but a Unix expert may be able to confirm that.

(In practice, I've had problems getting a UTF-8 encoded shebang to work with non-ASCII characters on Windows, so be prepared to hit bugs in that area. You wouldn't be alone, it looks like distlib's exe wrappers, which pip uses for console_scripts are also broken...)

[takluyver]

Thanks, that piece was interesting reading; I may find a use for shimmy at some point.

I'll look into adding shim exes for script files.

[takluyver]

I started looking into this, and I realised that installing wheels with entry points now creates a script that does a direct import and call instead of going through pkg_resources 🎉 🎈 ✨ That was my main objection to using entry points, so I'll probably just do that instead of putting script files in the wheel.

However, is there already an issue for this: there is an extra blank line being inserted after the shebang, so the coding comment ends up on the third line:

#!/home/takluyver/.virtualenvs/ipy-release/bin/python3

# -*- coding: utf-8 -*-

I believe the coding comment only takes effect if it's on the first two lines, so this will be ineffective. I don't think it would ever actually be a problem - on Python 3, UTF-8 is already the default, and on Python 2, the module name and function name have to be ASCII anyway. But if you're putting the encoding comment in, you may as well do it properly.

[guyverthree]

Hello,
does that mean this is no longer an issue or no longer an issue for you ?

I'm changing the way I package the python modules for my application and we are using windows and have systems built around the .exe in the script folder. Is this being looked into ?

takluyer if not could you give me a part to look at as from pfmoor's comment it seem that it is an important part of the install process for windows.

[guyverthree]

In fact the latest PIP does create them if installing to a default directory but not if you use the --target== options. otherwise just a single .bat file is created.

I think i can work around this locally now.

[takluyver]

@guyverthree it's just not a problem for me any more, because I decided to use entry_points.txt rather than putting separate script files into the wheels. I hope a future version of the wheel spec will formally standardise this practice.

[pradyunsg]

Is this resolved? If not, what's the appropriate action to be done toward fixing this?

[takluyver]

From my point of view, the appropriate action is to include entry points (and specifically console_scripts) in the next version of the wheel spec, so they're a proper standard rather than something defined by usage. Then that could be the recommended way to package command line interfaces, rather than putting separate script files in the distribution.

[takluyver]

A brief follow-up: while entry points are not yet part of any wheel spec, we did specify their mechanism in the Python Packaging User Guide. So the de-facto spec is a bit less implementation defined now:

https://packaging.python.org/specifications/entry-points/

I'm still inclined to say that a future version of the wheel spec should deprecate the scripts directory, but if you want to close this as a pip issue, I won't object.