pyenv-sync command and unversioned and major-versioned python 3 symlinks

[capyvara]

Output of brew config

HOMEBREW_VERSION: 4.0.28-65-g2e28696
ORIGIN: https://github.com/Homebrew/brew
HEAD: 2e28696dca6bdc0c19de543eecfd144bf65f2426
Last commit: 10 days ago
Core tap origin: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 95df2fadbc831dac5fcacf53ef04c252ada80eb9
Core tap last commit: 7 weeks ago
Core tap branch: master
Core tap JSON: 13 Jul 19:05 UTC
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_EDITOR: subl -n -w
HOMEBREW_MAKE_JOBS: 16
HOMEBREW_NO_INSTALL_CLEANUP: set
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: 16-core 64-bit kabylake
Clang: 14.0.3 build 1403
Git: 2.41.0 => /usr/local/bin/git
Curl: 7.87.0 => /usr/bin/curl
macOS: 13.3.1-x86_64
CLT: N/A
Xcode: 14.3 => /Applications/Xcode14.3.app/Contents/Developer

Output of brew doctor

Please note that these warnings are just used to help the Homebrew maintainers
with debugging if you file an issue. If everything you use Homebrew for is
working fine: please don't worry or file an issue; just ignore this. Thanks!

Warning: Some installed formulae are deprecated or disabled.
You should find replacements for the following formulae:
  dep
  go@1.17
  node@10
  python@3.7
  ruby@2.7

Warning: You have the following deprecated, official taps tapped:
  Homebrew/homebrew-cask-drivers
Untap them with `brew untap`.

Warning: You have unlinked kegs in your Cellar.
Leaving kegs unlinked can lead to build-trouble and cause formulae that depend on
those kegs to fail to run properly once built. Run `brew link` on these:
  python

Description of issue

Recently the PR #15507 (link) added the pyenv-sync command that symlinks the brew installed versions into the pyenv versions directory.

However there's an issue that it fail to proper find the unversioned and major-versioned links, such as `python', 'python3', 'pip', 'pip3' and so on, example:

➜ which python3
/Users/capyvara/.pyenv/shims/python3

➜ python3 --version
Python 3.11.4

➜ export PYENV_VERSION=3.10.12

➜ python3 --version           
Python 3.11.4

Enabling PYENV_DEBUG, we can see that it fails to get the python3 executable at the bin path, and fallback to the system one installed at /usr/local/bin/python3.

pyenv-which source

(Lines omitted for brevity)

export PYENV_DEBUG=1
➜ python3 --version   
...
+(/usr/local/Cellar/pyenv/2.3.22/libexec/pyenv-which:52): for version in '"${versions[@]}"' '"$system"'
+(/usr/local/Cellar/pyenv/2.3.22/libexec/pyenv-which:68): '[' -x /Users/capyvara/.pyenv/versions/3.10.12/bin/python3 ']'
+(/usr/local/Cellar/pyenv/2.3.22/libexec/pyenv-which:68): '[' -x /usr/local/bin/python3 ']'
+(/usr/local/Cellar/pyenv/2.3.22/libexec/pyenv-which:69): break
+(/usr/local/Cellar/pyenv/2.3.22/libexec/pyenv-exec:33): PYENV_COMMAND_PATH=/usr/local/bin/python3
...

Brew Python formulas create these links at libexec/bin instead of the bin that pyenv looks for, anyone know the reason why?

Also another issue is that if you perform a pyenv install of a version already symlinked by the pyenv-sync (and select replace) instead of deleting the symlink and creating a new folder, pyenv actually overrides the contents of the brew installation at Cellar.

Some ideas:

• Maybe pyenv-sync command can create those symlinks at installation bin at Cellar? anyone seems any downside?

• Or instead of symlinking the root folder create symlinks for the subdirs instead.

• Or we can make a PR at pyenv that also seeks at libexec/bin sub-folders?

Any ideas?

Thanks

[osalbahr]

Does running brew link (as suggested by brew doctor) affect anything?

[capyvara]

No, it's just a conflict of a old python 3.7 installation, it will go away once I remove it.

[JeroenSchmidt]

I think I have a similar issue.

ENV Details

~ » brew --config
HOMEBREW_VERSION: 4.2.11-54-g76c7245
ORIGIN: https://github.com/Homebrew/brew
HEAD: 76c7245d10d4f6219a2c3ac9b584dd8d9661087d
Last commit: 3 hours ago
Core tap HEAD: fd602f0a4288b46db0dfc81e29e59fd148bfa2a6
Core tap last commit: 68 minutes ago
Core tap JSON: 06 Mar 19:24 UTC
Core cask tap HEAD: 844c1e835d6f3a2b1071e6debc647b4680a10164
Core cask tap last commit: 70 minutes ago
Core cask tap JSON: 06 Mar 19:24 UTC
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 10
HOMEBREW_SORBET_RUNTIME: set
Homebrew Ruby: 3.1.4 => /opt/homebrew/Library/Homebrew/vendor/portable-ruby/3.1.4/bin/ruby
CPU: 10-core 64-bit arm_firestorm_icestorm
Clang: 15.0.0 build 1500
Git: 2.44.0 => /opt/homebrew/bin/git
Curl: 8.4.0 => /usr/bin/curl
macOS: 14.3-arm64
CLT: 15.1.0.0.1.1700200546
Xcode: N/A
Rosetta 2: false

~ » pyenv --version
pyenv 2.3.36

~ » which pyenv
/opt/homebrew/bin/pyenv

Working Behaviour

pyenv only works if I don't use a homebrew python version

# Working
~ » pyenv global 3.11.3
~ » pyenv versions
  system
* 3.11.3 (set by /Users/<user>/.pyenv/version)
  3.11.8 --> /opt/homebrew/Cellar/python@3.11/3.11.8
  3.12.2 --> /opt/homebrew/Cellar/python@3.12/3.12.2_1

~ » python3 --version
3.11.3

~ » python --version
3.11.3

Broken Behaviour

pyenv stops working as soon as I set a homebrew python version

~ » pyenv global 3.11.8
~ » pyenv versions
  system
  3.11.3 
* 3.11.8 --> /opt/homebrew/Cellar/python@3.11/3.11.8 (set by /Users/<user>/.pyenv/version)
  3.12.2 --> /opt/homebrew/Cellar/python@3.12/3.12.2_1

~ » python3 --version
3.12.2 # I suspect that is unable to find anything so it falls back to python@3.12.2 located in my PATH

~ » python --version
pyenv: python: command not found

The `python' command exists in these Python versions:
  3.11.3 

[MikeMcQuaid]

Sounds like a legit issue that should be fixed and is probably worth filing in Homebrew/brew!

[jvsteiner]

I have this same problem. I used pyenv-sync, but the pyenv chosen version was not being used, as designed. I also ran ussing the --debug flag, and saw that it was falling back to the /opt/homebrew/bin/python3. (in my case, I have an apple silicon macbook, so that is why brew is set up in /opt) I traced where the breakdown occurred, and in my case, I discovered that the contents of ~/.pyenv/versions/3.11.9/bin included only the long-versioned names for the python interpreter, ie it had:

lrwxr-xr-x@ 1 username  admin   58 Apr  2 11:25 2to3-3.11 -> ../Frameworks/Python.framework/Versions/3.11/bin/2to3-3.11
lrwxr-xr-x@ 1 username  admin   57 Apr  2 11:25 idle3.11 -> ../Frameworks/Python.framework/Versions/3.11/bin/idle3.11
-rwxr-xr-x@ 1 username  wheel  249 Apr  3 15:20 pip3.11
lrwxr-xr-x@ 1 username  admin   58 Apr  2 11:25 pydoc3.11 -> ../Frameworks/Python.framework/Versions/3.11/bin/pydoc3.11
lrwxr-xr-x@ 1 username  admin   59 Apr  2 11:25 python3.11 -> ../Frameworks/Python.framework/Versions/3.11/bin/python3.11
lrwxr-xr-x@ 1 username  admin   66 Apr  2 11:25 python3.11-config -> ../Frameworks/Python.framework/Versions/3.11/bin/python3.11-config
-rwxr-xr-x@ 1 username  wheel  236 Apr  3 15:20 wheel3.11

but not any python3 link, which is what pyenv was looking for.

Once I added an extra python3 and pip3 symlink, for example:

ln -s /Users/username/.pyenv/versions/3.11.9/bin/python3.11 /Users/username/.pyenv/versions/3.11.9/bin/python3

everything worked as expected.