feature: shims as .exe files

[db4]

Current implementation of shims as .bat files breaks some existing use cases. E.g. CMake wrapper for conan contains the following code fragment to call Conan:

        execute_process(COMMAND ${CONAN_CMD} ${conan_args}
                        RESULT_VARIABLE return_code
                        OUTPUT_VARIABLE conan_output
                        ERROR_VARIABLE conan_output
                        WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR})

normally CONAN_CMD points to <python dir>\Scripts\conan.exe, but with pyenv-win it turns to shims\conan.bat, so execute_process() fails (it doesn't use cmd.exe to run commands)

Maybe pyenv-win could use Chocolatey Shim Generator to work around this? Although it's not OSS, they're ready to grant a license:

I want to use shimgen outside of Chocolatey.

If your project is FOSS, please contact us for a grant of a free license to do so

[db4]

The following proof of concept works for me so far:

--- pyenv-lib.vbs.orig	2021-09-16 15:40:10.000000000 +0300
+++ pyenv-lib.vbs	2022-02-16 11:15:15.096235800 +0300
@@ -256,14 +256,18 @@
 Sub WriteWinScript(baseName)
     ' WScript.echo "kkotari: pyenv-lib.vbs write win script..!"
     Dim filespec
-    filespec = strDirShims &"\"& baseName &".bat"
+    Dim objexec
+    Dim shimgen
+    Dim cmd
+    shimgen = objws.Environment("Process")("ChocolateyInstall")&"\tools\shimgen.exe"
+    cmd = objws.Environment("Process")("COMSPEC")
+    filespec = strDirShims &"\"& baseName &".exe"
     If Not objfs.FileExists(filespec) Then
-        With objfs.CreateTextFile(filespec)
-            .WriteLine("@echo off")
-            .WriteLine("chcp 1250 > NUL")
-            .WriteLine("call pyenv exec %~n0 %*")
-            .Close
-        End With
+        Set objexec = objws.Exec(shimgen & " -o " & filespec & " -p " & cmd & " -c ""/C pyenv exec " & baseName & """")
+        Do
+            WScript.StdOut.WriteLine(objExec.StdOut.ReadLine())
+        Loop While Not objexec.Stdout.atEndOfStream
+        WScript.StdOut.WriteLine(objexec.StdOut.ReadAll)
     End If
 End Sub

[kirankotari]

@db4 Hey it's good to create a pull request for it. Please go ahead.

[bkeryan]

Is it necessary to dynamically generate shim EXEs? A single, precompiled shim EXE could use argv[0] to identify which command to call.

[db4]

Is it necessary to dynamically generate shim EXEs? A single, precompiled shim EXE could use argv[0] to identify which command to call.

How this identification could work? Maintain the map in a separate file (one for every shim directory)? Yes, that's possible, but IMHO harder to administer and requires some efforts to implement (Chocolatey Shim Generator is already there)

[bkeryan]

The existing batch file implementation already uses this technique. %0 is argv[0]. %~n0 extracts the command name, omitting the drive letter, directory, and file extension. pyenv exec does the rest.

In C on Windows, you can use _splitpath_s to extract the command name:

char command[MAX_PATH] = "";
errno_t error = _splitpath_s(argv[0], NULL, 0, NULL, 0, command, sizeof(command), NULL, 0);

The result is a command name that you can pass to pyenv exec along with argv[1] through argv[argc - 1]. For example:

argv[0]	command
python	python
python3.exe	python3
C:\Users\Username\.pyenv\pyenv-win\shims\pip.exe	pip
..\..\.pyenv\pyenv-win\shims\pythonw.exe	pythonw

[db4]

@bkeryan I like your approach. I've created a helper exe in C and it seems to work with pyenv without an issue. The only problem is how to integrate it into pyenv-win distribution? C sources need Visual Studio to build them...

[db4]

No, still not entirely acceptable. FindPython3.cmake detects python the following way:

  if (_${_PYTHON_PREFIX}_EXECUTABLE AND NOT CMAKE_CROSSCOMPILING)
    if (NAME STREQUAL "PREFIX")
      execute_process (COMMAND ${_${_PYTHON_PREFIX}_INTERPRETER_LAUNCHER} "${_${_PYTHON_PREFIX}_EXECUTABLE}" -c "import sys\ntry:\n   from distutils import sysconfig\n   sys.stdout.write(';'.join([sysconfig.PREFIX,sysconfig.EXEC_PREFIX,sysconfig.BASE_EXEC_PREFIX]))\nexcept Exception:\n   import sysconfig\n   sys.stdout.write(';'.join([sysconfig.get_config_var('base') or '', sysconfig.get_config_var('installed_base') or '']))"
                       RESULT_VARIABLE _result
                       OUTPUT_VARIABLE _values
                       ERROR_QUIET
                       OUTPUT_STRIP_TRAILING_WHITESPACE)

Note these newline characters. I don't know a way to correctly escape them for pyenv.bat. Probably the only chance is to re-implement pyenv.bat exec logic in C.

[Cologler]

I created https://github.com/Cologler/exe2ps1-rust for this and used it for a couple of months. The problem is it will be removed by pyenv after calling pip install ....

[db4]

@Cologler does FindPython3.cmake from CMake distribution work correctly with your python.exe?