pre-commit hook with language version language_version: python2.7 is not working

[manishsharma64]

describe your issue

Getting attached error. I am on Mac OS, M1 machine

pyenv global
system
2.7.18
3.8.5

Tried after clean cache using pre-comple clean as well.
Using language version of python3 is not an option.

pre-commit --version

2.15.0

.pre-commit-config.yaml

default_stages: [commit]
repos:
  - repo: https://gitlab.com/pycqa/flake8
    rev: 3.9.2
    hooks:
      - id: flake8
        name: Flake8 (Python 2)
        language_version: python2.7
        exclude: |
          (?x)^(
            node_modules/.*|
            org/rpc/external/temporal_grpc/bin/.*
          )$

~/.cache/pre-commit/pre-commit.log (if present)

version information

pre-commit version: 2.15.0
sys.version:
    3.9.7 (default, Sep  3 2021, 12:37:55)
    [Clang 12.0.5 (clang-1205.0.22.9)]
sys.executable: /usr/local/Cellar/pre-commit/2.15.0/libexec/bin/python3.9
os.name: posix
sys.platform: darwin

error information

An unexpected error has occurred: CalledProcessError: command: ('/Users/manishsharma/.cache/pre-commit/repoba54ew84/py_env-python2.7/bin/python', '-mpip', 'install', '.')
return code: -9
expected return code: 0
stdout: (none)
stderr: (none)

Traceback (most recent call last):
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/error_handler.py", line 65, in error_handler
    yield
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/main.py", line 396, in main
    return run(args.config, store, args)
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/commands/run.py", line 417, in run
    install_hook_envs(to_install, store)
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/repository.py", line 224, in install_hook_envs
    _hook_install(hook)
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/repository.py", line 82, in _hook_install
    lang.install_environment(
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/languages/python.py", line 205, in install_environment
    helpers.run_setup_cmd(prefix, install_cmd)
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/languages/helpers.py", line 52, in run_setup_cmd
    cmd_output_b(*cmd, cwd=prefix.prefix_dir, **kwargs)
  File "/usr/local/Cellar/pre-commit/2.15.0/libexec/lib/python3.9/site-packages/pre_commit/util.py", line 154, in cmd_output_b
    raise CalledProcessError(returncode, cmd, retcode, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('/Users/manishsharma/.cache/pre-commit/repoba54ew84/py_env-python2.7/bin/python', '-mpip', 'install', '.')
return code: -9
expected return code: 0
stdout: (none)
stderr: (none)

[asottile]

exit code -9 means the process was terminated by SIGKILL -- perhaps some management software? out of memory? impossible to tell since I don't have an M1 mac (I'm surprised python2.7 even installs on an M1 mac given it end-of-lifed before M1 was released)

[manishsharma64]

pre-compile works fine if don't mention language_version, on Python 3. Doubt it's a memory issue.

default_stages: [commit]
repos:
  - repo: https://gitlab.com/pycqa/flake8
    rev: 3.9.2
    hooks:
      - id: flake8
        name: Flake8 (Python 3)
        exclude: |
          (?x)^(
            node_modules/.*|
            org/rpc/external/temporal_grpc/bin/.*
          )$

Anything else I can try?

[asottile]

try this:

git clone https://gitlab.com/pycqa/flake8
cd flake8
git checkout 3.9.2
virtualenv venv -ppython2.7
venv/bin/pip install .

that's basically all that pre-commit does

[manishsharma64]

seems to be working, last few lines of last step. What next?

Building wheels for collected packages: flake8
  Building wheel for flake8 (setup.py) ... done
  Created wheel for flake8: filename=flake8-3.9.2-py2.py3-none-any.whl size=73148 sha256=75c54625c0ac176ebba27939a57bd0372728aa40874f1389b4bcd0269eb717a7
  Stored in directory: /private/var/folders/y2/2ys95ksx3lgcm7wpbl24fq440000gp/T/pip-ephem-wheel-cache-r4jfT0/wheels/ab/b2/53/d84ea87d0f6d09c9ebb9e550efdaf07f768bee88ea690013cb
Successfully built flake8
Installing collected packages: pyflakes, pycodestyle, mccabe, enum34, typing, configparser, functools32, contextlib2, zipp, six, scandir, pathlib2, importlib-metadata, flake8
Successfully installed configparser-4.0.2 contextlib2-0.6.0.post1 enum34-1.1.10 flake8-3.9.2 functools32-3.2.3.post2 importlib-metadata-2.1.1 mccabe-0.6.1 pathlib2-2.3.6 pycodestyle-2.7.0 pyflakes-2.3.1 scandir-1.10.0 six-1.16.0 typing-3.10.0.0 zipp-1.2.0

[asottile]

can you show all the output of all the commands?

[manishsharma64]

➜  testdir2 git clone https://gitlab.com/pycqa/flake8
Cloning into 'flake8'...
warning: redirecting to https://gitlab.com/pycqa/flake8.git/
remote: Enumerating objects: 10533, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 10533 (delta 0), reused 0 (delta 0), pack-reused 10531
Receiving objects: 100% (10533/10533), 2.69 MiB | 8.09 MiB/s, done.
Resolving deltas: 100% (6770/6770), done.
➜  testdir2 cd flake8
➜  flake8 git:(master) git checkout 3.9.2
Note: switching to '3.9.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at c6e0d27 Release 3.9.2
➜  flake8 git:(c6e0d27) virtualenv venv -ppython2.7
created virtual environment CPython2.7.18.final.0-64 in 199ms
  creator CPython2Posix(dest=/Users/manishsharma/dev/repos/testdir2/flake8/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/manishsharma/Library/Application Support/virtualenv)
    added seed packages: pip==20.3.4, setuptools==44.1.1, wheel==0.37.0
  activators NushellActivator,PythonActivator,FishActivator,CShellActivator,PowerShellActivator,BashActivator
➜  flake8 git:(c6e0d27) venv/bin/pip install .
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Processing /Users/manishsharma/dev/repos/testdir2/flake8
Collecting pyflakes<2.4.0,>=2.3.0
  Using cached pyflakes-2.3.1-py2.py3-none-any.whl (68 kB)
Collecting pycodestyle<2.8.0,>=2.7.0
  Using cached pycodestyle-2.7.0-py2.py3-none-any.whl (41 kB)
Collecting mccabe<0.7.0,>=0.6.0
  Using cached mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting enum34
  Using cached enum34-1.1.10-py2-none-any.whl (11 kB)
Collecting typing
  Using cached typing-3.10.0.0-py2-none-any.whl (26 kB)
Collecting configparser
  Using cached configparser-4.0.2-py2.py3-none-any.whl (22 kB)
Processing /Users/manishsharma/Library/Caches/pip/wheels/c2/ea/a3/25af52265fad6418a74df0b8d9ca8b89e0b3735dbd4d0d3794/functools32-3.2.3.post2-py2-none-any.whl
Collecting importlib-metadata
  Using cached importlib_metadata-2.1.1-py2.py3-none-any.whl (10 kB)
Collecting zipp>=0.5
  Using cached zipp-1.2.0-py2.py3-none-any.whl (4.8 kB)
Collecting pathlib2; python_version < "3"
  Using cached pathlib2-2.3.6-py2.py3-none-any.whl (17 kB)
Collecting contextlib2; python_version < "3"
  Using cached contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Processing /Users/manishsharma/Library/Caches/pip/wheels/58/2c/26/52406f7d1f19bcc47a6fbd1037a5f293492f5cf1d58c539edb/scandir-1.10.0-cp27-cp27m-macosx_11_0_x86_64.whl
Building wheels for collected packages: flake8
  Building wheel for flake8 (setup.py) ... done
  Created wheel for flake8: filename=flake8-3.9.2-py2.py3-none-any.whl size=73148 sha256=e28e2d8215b7d10aac22235cd3d35119d3ffbb2d3681a7384d1b6d83e7a396f4
  Stored in directory: /private/var/folders/y2/2ys95ksx3lgcm7wpbl24fq440000gp/T/pip-ephem-wheel-cache-GpcSRO/wheels/6b/ba/e6/a9a6ddf0e1055668b0a6f74387b6394778b00aa47ce1fbe47c
Successfully built flake8
Installing collected packages: pyflakes, pycodestyle, mccabe, enum34, typing, configparser, functools32, contextlib2, zipp, six, scandir, pathlib2, importlib-metadata, flake8
Successfully installed configparser-4.0.2 contextlib2-0.6.0.post1 enum34-1.1.10 flake8-3.9.2 functools32-3.2.3.post2 importlib-metadata-2.1.1 mccabe-0.6.1 pathlib2-2.3.6 pycodestyle-2.7.0 pyflakes-2.3.1 scandir-1.10.0 six-1.16.0 typing-3.10.0.0 zipp-1.2.0

[asottile]

hmmm maybe try this instead? it's more exact to what pre-commit runs:

git clone https://gitlab.com/pycqa/flake8
cd flake8
git checkout 3.9.2
WD="$PWD"
cd / && virtualenv "$wd/venv" -p python2.7 && cd "$WD"
. venv/bin/activate
python -m pip install .

[manishsharma64]

looks good. Changed wd in capital here -> cd / && virtualenv "$wd/venv" -p python2.7 && cd "$WD"

➜  testdir4 git clone https://gitlab.com/pycqa/flake8
Cloning into 'flake8'...
warning: redirecting to https://gitlab.com/pycqa/flake8.git/
remote: Enumerating objects: 10533, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 10533 (delta 0), reused 0 (delta 0), pack-reused 10531
Receiving objects: 100% (10533/10533), 2.69 MiB | 6.57 MiB/s, done.
Resolving deltas: 100% (6770/6770), done.
➜  testdir4 cd flake8
➜  flake8 git:(master) git checkout 3.9.2
Note: switching to '3.9.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at c6e0d27 Release 3.9.2
➜  flake8 git:(c6e0d27) WD="$PWD"
➜  flake8 git:(c6e0d27) echo $WD
/Users/manishsharma/dev/repos/testdir4/flake8
➜  flake8 git:(c6e0d27) cd / && virtualenv "$WD/venv" -p python2.7 && cd "$WD"
created virtual environment CPython2.7.18.final.0-64 in 195ms
  creator CPython2Posix(dest=/Users/manishsharma/dev/repos/testdir4/flake8/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/manishsharma/Library/Application Support/virtualenv)
    added seed packages: pip==20.3.4, setuptools==44.1.1, wheel==0.37.0
  activators NushellActivator,PythonActivator,FishActivator,CShellActivator,PowerShellActivator,BashActivator
➜  flake8 git:(c6e0d27) . venv/bin/activate
(venv) ➜  flake8 git:(c6e0d27) python -m pip install .
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Processing /Users/manishsharma/dev/repos/testdir4/flake8
Collecting pyflakes<2.4.0,>=2.3.0
  Using cached pyflakes-2.3.1-py2.py3-none-any.whl (68 kB)
Collecting pycodestyle<2.8.0,>=2.7.0
  Using cached pycodestyle-2.7.0-py2.py3-none-any.whl (41 kB)
Collecting mccabe<0.7.0,>=0.6.0
  Using cached mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting enum34
  Using cached enum34-1.1.10-py2-none-any.whl (11 kB)
Collecting typing
  Using cached typing-3.10.0.0-py2-none-any.whl (26 kB)
Collecting configparser
  Using cached configparser-4.0.2-py2.py3-none-any.whl (22 kB)
Processing /Users/manishsharma/Library/Caches/pip/wheels/c2/ea/a3/25af52265fad6418a74df0b8d9ca8b89e0b3735dbd4d0d3794/functools32-3.2.3.post2-py2-none-any.whl
Collecting importlib-metadata
  Using cached importlib_metadata-2.1.1-py2.py3-none-any.whl (10 kB)
Collecting zipp>=0.5
  Using cached zipp-1.2.0-py2.py3-none-any.whl (4.8 kB)
Collecting pathlib2; python_version < "3"
  Using cached pathlib2-2.3.6-py2.py3-none-any.whl (17 kB)
Collecting contextlib2; python_version < "3"
  Using cached contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Processing /Users/manishsharma/Library/Caches/pip/wheels/58/2c/26/52406f7d1f19bcc47a6fbd1037a5f293492f5cf1d58c539edb/scandir-1.10.0-cp27-cp27m-macosx_11_0_x86_64.whl
Building wheels for collected packages: flake8
  Building wheel for flake8 (setup.py) ... done
  Created wheel for flake8: filename=flake8-3.9.2-py2.py3-none-any.whl size=73148 sha256=95abe7aeed935be6d6c515277cd7b68abf7cddfd704e76e8684455ddb3a1637b
  Stored in directory: /private/var/folders/y2/2ys95ksx3lgcm7wpbl24fq440000gp/T/pip-ephem-wheel-cache-gePDeH/wheels/07/d4/27/8ef0718dfd9329a0c6f511f644d5a857f4b3e02cf8f57ae6d3
Successfully built flake8
Installing collected packages: pyflakes, pycodestyle, mccabe, enum34, typing, configparser, functools32, contextlib2, zipp, six, scandir, pathlib2, importlib-metadata, flake8
Successfully installed configparser-4.0.2 contextlib2-0.6.0.post1 enum34-1.1.10 flake8-3.9.2 functools32-3.2.3.post2 importlib-metadata-2.1.1 mccabe-0.6.1 pathlib2-2.3.6 pycodestyle-2.7.0 pyflakes-2.3.1 scandir-1.10.0 six-1.16.0 typing-3.10.0.0 zipp-1.2.0
(venv) ➜  flake8 git:(c6e0d27)

[asottile]

does it reproduce as well when you do the cloning inside of ~/.cache/pre-commit ?

[manishsharma64]

yes, looks good in there too.

➜  repotestdir git clone https://gitlab.com/pycqa/flake8
Cloning into 'flake8'...
warning: redirecting to https://gitlab.com/pycqa/flake8.git/
remote: Enumerating objects: 10533, done.
remote: Counting objects: 100% (2/2), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 10533 (delta 0), reused 0 (delta 0), pack-reused 10531
Receiving objects: 100% (10533/10533), 2.69 MiB | 11.54 MiB/s, done.
Resolving deltas: 100% (6770/6770), done.
➜  repotestdir cd flake8
➜  flake8 git:(master) git checkout 3.9.2
Note: switching to '3.9.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at c6e0d27 Release 3.9.2
➜  flake8 git:(c6e0d27) WD="$PWD"
➜  flake8 git:(c6e0d27) cd / && virtualenv "$WD/venv" -p python2.7 && cd "$WD"
created virtual environment CPython2.7.18.final.0-64 in 194ms
  creator CPython2Posix(dest=/Users/manishsharma/.cache/pre-commit/repotestdir/flake8/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, wheel=bundle, setuptools=bundle, via=copy, app_data_dir=/Users/manishsharma/Library/Application Support/virtualenv)
    added seed packages: pip==20.3.4, setuptools==44.1.1, wheel==0.37.0
  activators NushellActivator,PythonActivator,FishActivator,CShellActivator,PowerShellActivator,BashActivator
➜  flake8 git:(c6e0d27) . venv/bin/activate
(venv) ➜  flake8 git:(c6e0d27) python -m pip install .
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Processing /Users/manishsharma/.cache/pre-commit/repotestdir/flake8
Collecting pyflakes<2.4.0,>=2.3.0
  Using cached pyflakes-2.3.1-py2.py3-none-any.whl (68 kB)
Collecting pycodestyle<2.8.0,>=2.7.0
  Using cached pycodestyle-2.7.0-py2.py3-none-any.whl (41 kB)
Collecting mccabe<0.7.0,>=0.6.0
  Using cached mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting enum34
  Using cached enum34-1.1.10-py2-none-any.whl (11 kB)
Collecting typing
  Using cached typing-3.10.0.0-py2-none-any.whl (26 kB)
Collecting configparser
  Using cached configparser-4.0.2-py2.py3-none-any.whl (22 kB)
Processing /Users/manishsharma/Library/Caches/pip/wheels/c2/ea/a3/25af52265fad6418a74df0b8d9ca8b89e0b3735dbd4d0d3794/functools32-3.2.3.post2-py2-none-any.whl
Collecting importlib-metadata
  Using cached importlib_metadata-2.1.1-py2.py3-none-any.whl (10 kB)
Collecting zipp>=0.5
  Using cached zipp-1.2.0-py2.py3-none-any.whl (4.8 kB)
Collecting pathlib2; python_version < "3"
  Using cached pathlib2-2.3.6-py2.py3-none-any.whl (17 kB)
Collecting contextlib2; python_version < "3"
  Using cached contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Processing /Users/manishsharma/Library/Caches/pip/wheels/58/2c/26/52406f7d1f19bcc47a6fbd1037a5f293492f5cf1d58c539edb/scandir-1.10.0-cp27-cp27m-macosx_11_0_x86_64.whl
Building wheels for collected packages: flake8
  Building wheel for flake8 (setup.py) ... done
  Created wheel for flake8: filename=flake8-3.9.2-py2.py3-none-any.whl size=73148 sha256=cf9c6c18e5179c245123c4c54475bb2ee7832aafc86526c3975211f8cb1da989
  Stored in directory: /private/var/folders/y2/2ys95ksx3lgcm7wpbl24fq440000gp/T/pip-ephem-wheel-cache-lTCk5Z/wheels/f2/c3/dd/2244c3e3f984b305cbde14bbc37d828747b8b8df277f61e8ca
Successfully built flake8
Installing collected packages: pyflakes, pycodestyle, mccabe, enum34, typing, configparser, functools32, contextlib2, zipp, six, scandir, pathlib2, importlib-metadata, flake8
Successfully installed configparser-4.0.2 contextlib2-0.6.0.post1 enum34-1.1.10 flake8-3.9.2 functools32-3.2.3.post2 importlib-metadata-2.1.1 mccabe-0.6.1 pathlib2-2.3.6 pycodestyle-2.7.0 pyflakes-2.3.1 scandir-1.10.0 six-1.16.0 typing-3.10.0.0 zipp-1.2.0

[asottile]

what if you use the virtualenv from your brew installation of pre-commit ?

[manishsharma64]

I didn't get it. Use pre-commit from virtualenv instead of brew installation?

[asottile]

ah I was meaning more to use the /usr/local/Cellar/pre-commit/2.15.0/libexec/bin/virtualenv instead of virtualenv maybe to reproduce it

but yeah maybe try pre-commit installed in a virtualenv as well?

[manishsharma64]

I ran these, inside ~/.cache/pre-commit
Successfully built flake8

git clone https://gitlab.com/pycqa/flake8
cd flake8
git checkout 3.9.2
WD="$PWD"
cd / && /usr/local/Cellar/pre-commit/2.15.0/libexec/bin/virtualenv "$WD/venv" -p python2.7 && cd "$WD"
. venv/bin/activate
python -m pip install .