Pipenv install failes because Access Denied to npm

[mrufsvold]

Issue description

When I run pipenv install in any directory, I get PermissionError: [WinError 5] Access is denied: 'C:\\WINDOWS\\system32\\config\\systemprofile\\AppData\\Roaming\\npm

Debugging already attempted

I already read this issue and tried --sequential, but it still didn't work.

I've uninstalled and reinstalled Pipenv, Pyenv, and Node.js. I tried using a different Python/Pip install to get Pipenv. I've also tried moving Python and related things to the top of my PATH to avoid every looking at NPM. The error message has remained the same.

My next step will be to fully delete everything related to Python and try from scratch, but before I go that far, I thought I'd see if anyone has expert guidance here. Especially, why Pathlib would be looking at npm AppData at all.

I checked the file system, and 'C:\WINDOWS\system32\config\` exists, but it is an empty folder.

Expected result

I expect the install to complete.

Actual result

Traceback (most recent call last):
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\USER\.pyenv\pyenv-win\versions\3.9.6\Scripts\pipenv.exe\__main__.py", line 7, in <module>      
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\cli\options.py", line 54, in main

    return super().main(*args, **kwargs, windows_expand_args=False)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 754, 
in invoke
    return __callback(*args, **kwargs)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 754, 
in invoke
    return __callback(*args, **kwargs)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\cli\command.py", line 193, in install
    do_install(
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\core.py", line 1852, in do_install
    ensure_project(
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\core.py", line 512, in ensure_project
    ensure_virtualenv(
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\core.py", line 436, in ensure_virtualenv
    python = ensure_python(project, three=three, python=python)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\core.py", line 328, in ensure_python
    path_to_python = find_a_system_python(python)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\core.py", line 293, in find_a_system_python
    return next(iter(finder.find_all_python_versions()), None)
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 312, in find_all_python_versions
    python_version_dict = getattr(self.system_path, "python_version_dict", {})
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 120, in system_path
    self._system_path = self.create_system_path()
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 82, in create_system_path
    return pyfinder_path.SystemPath.create(
  File "C:\Users\USER\.pyenv\pyenv-win\versions\3.9.6\Lib\site-packages\pipenv\vendor\pythonfinder\models\path.py", line 694, in create
    instance = instance._run_setup()
  File "C:\Users\USER\.pyenv\pyenv-win\versions\3.9.6\Lib\site-packages\pipenv\vendor\pythonfinder\models\path.py", line 241, in _run_setup
    path_order=[p.as_posix() for p in path_instances if p.exists()],
  File "C:\Users\USER\.pyenv\pyenv-win\versions\3.9.6\Lib\site-packages\pipenv\vendor\pythonfinder\models\path.py", line 241, in <listcomp>
    path_order=[p.as_posix() for p in path_instances if p.exists()],
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\pathlib.py", line 1414, in exists
    self.stat()
  File "c:\users\USER\.pyenv\pyenv-win\versions\3.9.6\lib\pathlib.py", line 1222, in stat
    return self._accessor.stat(self)
PermissionError: [WinError 5] Access is denied: 'C:\\WINDOWS\\system32\\config\\systemprofile\\AppData\\Roaming\\npm'

Steps to replicate

I do not think this is a precise way to replicate, but the related actions I've made (that I can think of) are:
Install pipenv via a Conda-based Python
Install pyenv-win using the steps given in the docs
Install Node.js
Try to run pipenv install

Please run $ pipenv --support, and paste the results here. Don't put backticks (`) around it! The output already contains Markdown formatting.
'''
Traceback (most recent call last):
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\USER.pyenv\pyenv-win\versions\3.9.6\Scripts\pipenv.exe_main.py", line 7, in
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1128, in call
return self.main(*args, **kwargs)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\cli\options.py", line 54, in main
return super().main(*args, **kwargs, windows_expand_args=False)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1053, in main
rv = self.invoke(ctx)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1637, in invoke
super().invoke(ctx)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 754,
in invoke
return __callback(*args, **kwargs)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\decorators.py", line 84, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\core.py", line 754,
in invoke
return __callback(*args, **kwargs)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\click\decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\cli\command.py", line 103, in cli
get_pipenv_diagnostics(state.project)
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\help.py", line 31, in get_pipenv_diagnostics
python_paths = finder.find_all_python_versions()
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 312, in find_all_python_versions
python_version_dict = getattr(self.system_path, "python_version_dict", {})
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 120, in system_path
self._system_path = self.create_system_path()
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\site-packages\pipenv\vendor\pythonfinder\pythonfinder.py", line 82, in create_system_path
return pyfinder_path.SystemPath.create(
File "C:\Users\USER.pyenv\pyenv-win\versions\3.9.6\Lib\site-packages\pipenv\vendor\pythonfinder\models\path.py", line 694, in create
instance = instance._run_setup()
File "C:\Users\USER.pyenv\pyenv-win\versions\3.9.6\Lib\site-packages\pipenv\vendor\pythonfinder\models\path.py", line 241, in _run_setup
path_order=[p.as_posix() for p in path_instances if p.exists()],
File "C:\Users\USER.pyenv\pyenv-win\versions\3.9.6\Lib\site-packages\pipenv\vendor\pythonfinder\models\path.py", line 241, in
path_order=[p.as_posix() for p in path_instances if p.exists()],
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\pathlib.py", line 1414, in exists
self.stat()
File "c:\users\USER.pyenv\pyenv-win\versions\3.9.6\lib\pathlib.py", line 1222, in stat
return self._accessor.stat(self)
PermissionError: [WinError 5] Access is denied: 'C:\WINDOWS\system32\config\systemprofile\AppData\Roaming\npm
'''

[mrufsvold]

I just uninstalled Anaconda and all versions of python, uninstalled Node and NPM, and deleted all associated folders that I know of. Then I installed a fresh copy of Miniconda, installed pipenv, ran pipenv install, and got the same error. Shakes fist at the sky

[matteius]

@mrufsvold what version of pipenv are you running?

[mrufsvold]

pipenv, version 2022.3.28

[matteius]

@mrufsvold I am not super familiar with Anaconda -- can you explain it briefly?

What happens if you set the environment variable PIPENV_MAX_DEPTH = 1?

[mrufsvold]

Anaconda is a collection of software tooling that supports Python. It creates a predictable environment for a python installation so that processes are more easily replicable across machines and it includes stuff like a GUI for launching Jupyter, Spyder, etc.

Basically, it includes lots of nice-to-haves that lower the learning curve for new Python users and helps enterprise level uses of python maintain consistency.

I'll try deleting it and just installing vanilla Python to see if that helps.

I tried setting that variable, but it didn't change the error.

[matteius]

@mrufsvold I had good luck installing python from the Windows store. Also I have pyenv setup on windows to pull in whatever python version pipenv needs for the project at hand.

[mrufsvold]

Welp, I'm at a loss. I cleared out Anaconda. Did a search of my entire user folder for "npm" and deleted every file that came up. Installed just Python and pipenv. Ran pipenv install and nothing.

This is a work machine so I don't have admin which means I can't dig much deeper at this point.

You're welcome to close this ticket or leave it open and I'll post if I get resolution with my IT department tomorrow.

Thanks for all your late night help, @matteius !

[matteius]

Here is what I think is happening is you are getting caught in a trap of recursively searching for a Pipfile. Do you have a Pipfile in the directory you are installing to? What is the project path or how deep is it from your root C:\ drive? Also have you tried another shell, say power shell if you were using cmd, or even git shell? If you don't have a Pipfile in the directory, perhaps just touch Pipfile first?

[matteius]

@mrufsvold Also you can try setting PIPENV_NO_INHERIT = 1 in your environmnet variables as well. EDIT: Actually that just sets PIPENV_MAX_DEPTH = 2 so I think not setting PIPENV_NO_INHERIT and setting PIPENV_MAX_DEPTH = 1 would be better but as you already said that didn't work. The other big thing I'll say is its always preferable to do a --user install so pip install pipenv --user instead of pip install pipenv. Another thing, is I have some environment variables to find the local installs of those things:

export PATH=$PATH:~/.pyenv/pyenv-win/bin/:~/.pyenv/pyenv-win/shims
export PYENV_HOME=~/.pyenv
export PYENV_ROOT=~/.pyenv
export PYENV=~/.pyenv

export PATH=$PATH:~/.local/bin

export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Projects

It looks like in the stack trace that its trying to find your system python and not finding anything because you are running in pyenv, you might be lacking having it on your path -- also are you pyenv-win because you should be.

Or maybe is it possible your pyenv is messed up?

[mrufsvold]

I was using Pyenv originally, but I removed it as I tried to reset everything back to base and I haven't added it back in. I'll try to reinstall it now!

[matteius]

@mrufsvold It would appear from your original stack trace that the bug is happening within pythonfinder (which we vendor into the project) but I think its happening while looking for your system python. Saw you have pyenv-win setup correctly and called out in your Pipfile a specific python version that it would use pyenv to get/install it, I wonder if that would work smoother.

Also did you try installing the Python from the Microsoft Store?

[matteius]

Just remembering that I had to add to my path after installing from the Microsoft Store Python:

C:\Users\matte\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\pyenv-win\bin
C:\Users\matte\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\Scripts

I also added these environment variables for pyenv-win:

[mrufsvold]

Okay, so passing a complete path to a Python executable works. i.e. pipenv install --python c/path/to/python.exe

But after uninstalling/reinstalling, pyenv is not giving me output. It is recognized as a command (I have all of the path variables set up), but when I run

pyenv
pyenv install -l
pyenv --version

They all run without error but show no output.

Obviously, this is far outside the scope of a pipenv issue lol

Seems like it is something broken with my pyenv install to begin with.

Thank you SO much for working through this with me!

[matteius]

@mrufsvold How did you install pyenv? Are you using pyenv-win because of windows?

[mrufsvold]

pip install pyenv-win --target $HOME\\.pyenv
And then I tried installing with the .zip version just incase, still nothing!

[matteius]

@mrufsvold What shell are you using?

[matteius]

I had to do this in git shell environment, not the two alias at the top:

alias python='winpty python.exe'
alias pipenv='winpty pipenv.exe'
export PATH=$PATH:~/.pyenv/pyenv-win/bin/:~/.pyenv/pyenv-win/shims
export PYENV_HOME=~/.pyenv
export PYENV_ROOT=~/.pyenv
export PYENV=~/.pyenv

export PATH=$PATH:~/.local/bin

[mrufsvold]

I've tried Powershell and bash.

I've run those commands, but what's weird is that python, pipenv, and pyenv are all recognized commands. It's just that pyenv doesn't do anything when I try to run it.

[matteius]

Weird about pyenv.

matte@LAPTOP-N5VSGIBD MINGW64 ~/Projects/pipenv-triage/pipenv-5022e
$ which pyenv
/c/Users/matte/.pyenv/pyenv-win/bin/pyenv

matte@LAPTOP-N5VSGIBD MINGW64 ~/Projects/pipenv-triage/pipenv-5022e
$ pyenv
pyenv 2.64.11
Usage: pyenv <command> [<args>]

Some useful pyenv commands are:
   commands     List all available pyenv commands
   duplicate    Creates a duplicate python environment
   local        Set or show the local application-specific Python version
   global       Set or show the global Python version
   shell        Set or show the shell-specific Python version
   install      Install a Python version using python-build
   uninstall    Uninstall a specific Python version
   update       Update the cached version DB
   rehash       Rehash pyenv shims (run this after installing executables)
   vname        Show the current Python version
   version      Show the current Python version and its origin
   version-name Show the current Python version
   versions     List all Python versions available to pyenv
   exec         Runs an executable by first preparing PATH so that the selected Python
   which        Display the full path to an executable
   whence       List all Python versions that contain the given executable

See `pyenv help <command>' for information on a specific command.
For full documentation, see: https://github.com/pyenv-win/pyenv-win#readme

[mrufsvold]

Just circling back to this in case Google brings anyone here.

The source of the problem is definitely Pyenv, not pipenv. I cannot figure out the source of the problem and our IT department has not had time to help me yet.

My good-enough solution for now is to manually install a bunch of vanilla python versions at the user level with default settings. Pipenv was able to pick them up, so I can now run pipenv install --python x.x, and it will install properly.

Thanks again, @matteius !

[matteius]

Thanks @mrufsvold -- since we know your issue is a system configuration and pyenv issue at this point I am going to close this one out. I think it will still be useful to anyone that happens to find the closed issue, but will help keep our backlog clean. Good luck!