[Windows] installations should include pythonX.exe and pythonX.Y.exe executables

[edmorley]

BPO	21506
Nosy	@loewis, @pfmoore, @tjguk, @zware, @eryksun, @zooba, @edmorley

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2014-05-14.11:36:10.643>
labels = ['type-feature', 'expert-installation', '3.10', 'OS-windows']
title = '[Windows] installations should include pythonX.exe and pythonX.Y.exe executables'
updated_at = <Date 2021-03-18.22:07:25.470>
user = 'https://github.com/edmorley'

bugs.python.org fields:

activity = <Date 2021-03-18.22:07:25.470>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Installation', 'Windows']
creation = <Date 2014-05-14.11:36:10.643>
creator = 'edmorley'
dependencies = []
files = []
hgrepos = []
issue_num = 21506
keywords = []
message_count = 14.0
messages = ['218521', '218522', '218535', '218537', '218549', '218553', '218554', '218556', '218565', '218572', '251552', '251559', '251820', '389043']
nosy_count = 8.0
nosy_names = ['loewis', 'paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'edmorley', 'ahiijny']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21506'
versions = ['Python 3.10']

[edmorley]

A python.org 2.7.6 release of the Windows MSI installer, results in only the following python binaries in the installation directory:
C:\Python27\python.exe
C:\Python27\pythonw.exe

In Mozilla bug https://bugzilla.mozilla.org/show_bug.cgi?id=957721 we would like to be able to force our 'mach' script to use Python2.7 (due to the assorted Python versions present on our CI machines), however doing so via the script shebang line breaks local developer workflows on Windows, since the default Python install doesn't include python2.7.exe and so the binary isn't found.

As such, it would be great if python.exe could be symlinked to pythonX.Y.exe (and also I guess pythonX.exe) as part of the Windows installation/build - giving us parity with Unix based platforms.

This can be done on Windows using mklink (http://ss64.com/nt/mklink.html), eg:

c:\Python27>mklink python2.7.exe python.exe
symbolic link created for python2.7.exe <<===>> python.exe

c:\Python27>mklink python2.exe python.exe
symbolic link created for python2.exe <<===>> python.exe

c:\Python27>dir python*

10/11/2013 19:24 26,624 python.exe
14/05/2014 12:04 <SYMLINK> python2.exe [python.exe]
14/05/2014 12:04 <SYMLINK> python2.7.exe [python.exe]
10/11/2013 19:24 27,136 pythonw.exe

Alternatively, just a plain copy of the binary prior to creating the MSI would be just as helpful for us too.

I searched for a while to see if there were any bugs filed for this already, but the closest I could find was:

• http://legacy.python.org/dev/peps/pep-0397/ - which is about a smart launcher that uses the shebang lines to run the correct Python version when multiple are installed (and thus is quite a different request to that in this bug).
• and https://mail.python.org/pipermail/python-dev/2011-March/108850.html which suggests the idea as in this bug summary here, but it seems like a bug for it was never filed.

Many thanks :-)

[edmorley]

Meant to add: the ActivePython release does this already - but it would be great if upstream did too.

[zware]

A similar thought I've had is to add a feature to the PEP-397 launcher, allowing it to figure out Python version from a suffix on its executable name and add symlinks to it for each installed version of Python. That way PATH wouldn't need to be modified to make every installed version accessible from a single optionless command.

[loewis]

I don't deal with Python 2.7 anymore, so I have no opinion on this matter.

[zooba]

As far as Python 2.7 is concerned, I will be avoiding making any changes to the installer at all.

For Python 3.5 I'm trying to work out a few of the install issues, one of them being not installing into Program Files. I think symlinks may be the way to make that change work, in which case it will be trivial to add two more. That said, I prefer the py.exe launcher anyway (especially since it handles 32/64-bit slightly better than this would), mostly since I intensely dislike permanently modifying PATH for anybody :)

[eryksun]

How about using hard links (mklink /H) instead? It's a minor issue, but there's a bug in Explorer that breaks opening symbolic links to EXE, COM, CMD, and BAT files (exefile, comfile, cmdfile, batfile). Explorer displays the error message "the specified path does not exist". These file types have shell\open\command set to "%1" %*, i.e. the file itself (%1) is used as the command.

C:\>reg query hkcr\exefile\shell\open\command /ve

HKEY_CLASSES_ROOT\exefile\shell\open\command
    (Default)    REG_EXPAND_SZ    "%1" %*

I assume that's because these file types can be passed directly to CreateProcess. However, the Explorer bug isn't a problem with CreateProcess, or even ShellExecuteEx. Maybe it's due to an overly strict security policy. Anyway, my 2 cents is use hard links.

[zooba]

there's a bug in Explorer that breaks opening symbolic links to EXE, COM, CMD, and BAT files

Do you know which versions of Windows this applies to? It works fine for me (though I'm up to date), and the file associations have not changed.

I've seen some suggestions that creating a symlink without the right file extension (e.g. python2 <<===>> python.exe rather than python2.exe) will fail in some cases, but that's easily avoided by getting the symlink right.

[eryksun]

Here's a related superuser question:

http://superuser.com/q/304093

The message box shown in the older answer is what I get:

http://i.stack.imgur.com/bqXBs.png

The problem is only with Explorer. I can execute a symbolic link to an EXE via cmd, subprocess.Popen, os.startfile (ShellExecute), and so on. Also, Explorer only does this for the executable file types that I mentioned. It has no problem opening symbolic links to .py files; the difference is the open command line for a Python.File includes a program to run:

C:\>ftype Python.File
Python.File="C:\Windows\py.exe" "%1" %*

It works if I change the shell\open\command for the exefile type to run via cmd /c. But that's just silly.

[zooba]

Apparently there's a distinction between absolute and relative symlinks. Do you see a difference between:

mklink python2.7.exe python.exe

and

mklink python2.7.exe C:\Python27\python.exe

?

[eryksun]

Using a relative target in the link isn't the problem for me here. (See bpo-13702 for another problem with relative links.) I'm in the same boat as commenter weberc2. Same symptom; apparently a different problem.

I'm using an absolute path for the target:

C:\Program Files\Python\Scripts>mklink python2.7.exe "C:\Program Files\Python\Python27\python.exe"
symbolic link created for python2.7.exe <<===>> C:\Program Files\Python\Python27\python.exe

(Mirror the --user layout in %AppData%.)

This works fine from cmd, but not Explorer. Anyway, the link would be added to support shell scripts that use shebangs for environments such as MSYS, so executing with Explorer isn't the goal anyway. The shortcut in the start menu would target python.exe, not the symbolic/hard link.

[BreamoreBoy]

I've changed the version to 3.6 only as I just can't see this happening for any other. For all I know this might have been sorted in 3.5, I'll leave that decision to our Windows gurus.

[eryksun]

Creating a symbolic link can't be relied on for a per-user installation. Administrators have to elevate to create symbolic links, and most regular users also lack this privilege.

[zooba]

The file is so small that a second copy is just as cheap - literally everything is in the DLL. It could certainly be added to 3.6, and I'll probably get to it eventually, but right now I'm still focused on 3.5.

[eryksun]

One issues with versioned "X.Y" executables is interference with normal handling of the ".exe" extension. For example, in "python3.10" the ".10" counts as an extension, so normally ".exe" won't be appended when searching for the name.

This affects SearchPathW() searches that supply a default ".exe" extension, such as what CreateProcessW() uses to find the executable parsed from lpCommandLine. For example, subprocess.call('python3.9') won't append '.exe' to find "python3.9.exe" in the search path.

It also affects the shell API's "App Paths" search, since the shell won't append ".exe" to a name that already has an extension. (The machine and user "App Paths" keys are the Windows shell equivalent of Unix "/usr/bin" and "$HOME/.local/bin", but more powerful.) For example, the store app distribution of Python 3.9 creates a "python3.9.exe" entry in the user's "App Paths", but os.startfile('python3.9') doesn't work; it has to be os.startfile('python3.9.exe').

Using "_" instead of "." would be compatible with Windows filename conventions -- e.g. "python3_10.exe". But this is an uphill battle against a Unix convention that's probably been in use since the 1970s or 1980s.

[chrisoldwood]

zooba commented on Sep 29, 2015
The file is so small that a second copy is just as cheap - literally everything is in the DLL. It could certainly be added to 3.6, and I'll probably get to it eventually, but right now I'm still focused on 3.5.

Can I confirm that this is still a valid workaround today with 3.11? i.e. I'm assuming there are no obvious shortcomings [1] from taking this approach after installing the standard package. (There were so many potential ways to workaround this - copy binary, symbolic link, .bat file wrapper, Chocolately's shimgen, etc. that I wanted to pick one that actual Python people would choose as they would know best 🙂.)

---

[1] Aside from any permissions required, e.g. creating symbolic links, or other programs assuming it's a binary called python3.exe which would preclude a python3.{bat,cmd} wrapper script.

[zooba]

Can I confirm that this is still a valid workaround today with 3.11?

Yep. As I've mentioned elsewhere, nothing much has changed here.

In a virtual environment, you'll need to copy the venv/Scripts/python.exe and not the original install. You can also rename py.exe to python3.exe and it will behave like py.exe -V:PythonCore/3 ... (or py.exe -3 ...), which may be more to your liking, depending on your setup.

[zooba]

#99185 is a newer version of this bug, so I'd rather close this and keep that one open.