File Exist error on locking #4326

con-f-use · 2020-06-07T11:45:32Z

Issue description

After updating pipenv from 2018.11.26 to 2020.6.2 pipenv always crashes for one of my projects with this traceback:

┌─07-13:29[confus@confusion6:~/devel/doubleplusungood] e82d1:master(+3/-2)+* 1 ± 
└▪ pipenv lock --verbose --dev
Locking [dev-packages] dependencies…
Building requirements...
Traceback (most recent call last):
  File "/home/confus/.local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 374, in lock
    write=not state.quiet,
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/core.py", line 1114, in do_lock
    keep_outdated=keep_outdated
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1318, in venv_resolve_deps
    deps, project, r=False, include_index=True
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1434, in convert_deps_to_pip
    new_dep = Requirement.from_pipfile(dep_name, dep)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2743, in from_pipfile
    r = FileRequirement.from_pipfile(name, pipfile)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1843, in from_pipfile
    arg_dict["setup_info"] = arg_dict["parsed_line"].setup_info
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 813, in setup_info
    self.setup_info = self.get_setup_info()
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 802, in get_setup_info
    setup_info = SetupInfo.from_ireq(self.ireq, subdir=self.subdirectory)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1870, in from_ireq
    shutil.copytree(path, target, symlinks=True)
  File "/usr/lib/python3.6/shutil.py", line 321, in copytree
    os.makedirs(dst)
  File "/usr/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/tmp/reqlib-srcqkiyds5t/..'
┌─07-13:29[confus@confusion6:~/devel/doubleplusungood] e82d1:master(+3/-2)+* 1 ± 
└▪ pipenv lock --clear
Locking [dev-packages] dependencies…
Building requirements...
Traceback (most recent call last):
  File "/home/confus/.local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/cli/command.py", line 374, in lock
    write=not state.quiet,
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/core.py", line 1114, in do_lock
    keep_outdated=keep_outdated
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1318, in venv_resolve_deps
    deps, project, r=False, include_index=True
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/utils.py", line 1434, in convert_deps_to_pip
    new_dep = Requirement.from_pipfile(dep_name, dep)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2743, in from_pipfile
    r = FileRequirement.from_pipfile(name, pipfile)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 1843, in from_pipfile
    arg_dict["setup_info"] = arg_dict["parsed_line"].setup_info
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 813, in setup_info
    self.setup_info = self.get_setup_info()
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 802, in get_setup_info
    setup_info = SetupInfo.from_ireq(self.ireq, subdir=self.subdirectory)
  File "/home/confus/.local/lib/python3.6/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1870, in from_ireq
    shutil.copytree(path, target, symlinks=True)
  File "/usr/lib/python3.6/shutil.py", line 321, in copytree
    os.makedirs(dst)
  File "/usr/lib/python3.6/os.py", line 220, in makedirs
    mkdir(name, mode
FileExistsError: [Errno 17] File exists: '/tmp/reqlib-srcxj1os9tf/..'

$ pipenv --support

Pipenv version: '2020.6.2'

Pipenv location: '/home/jan/.local/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

... hangs indefinetly trying to find python installations....

The text was updated successfully, but these errors were encountered:

darrenmeehan · 2020-06-08T15:51:23Z

I'm facing a similar issue with a single project.

On this system I'm using python3.7 as system default, with python3.6 installed using pyenv.

I'll do some digging and see if there's anything I can find to help narrow down what is happening.

Edit:

It seems to be an issue with installing a local package - I can reproduce the issue by running pipenv install -e ../.

@con-f-use could you check your Pipfile to see if you're also installing a local package?

Edit 2:

I've noticed the path of the local package Pipfile seems to be invalid after trying to install the local package (Which is up a single directory from the Pipfile & Pipfile.lock).

This is what I'm running; which fails with similar output to above:

pipenv install -e ../

This is an extract from my Pipfile afterwards:

mypackagename = {editable = true, path = "./.."}

To clarify, when installing a local package the path is incorrectly added to the Pipfile. Are you able to reproduce this @con-f-use ?

munsu · 2020-06-17T07:26:21Z

Chipping in I'm getting the same thing. Tried rolling back to old pipenv versions and the latest (stable) one that works is 2018.11.26.

Re: the issue, I did indeed narrow it down to the local package as the 2020.6.2 version works properly if I remove the following line from my Pipfile:

packagename = {editable = true,path = "./../.."}

con-f-use · 2020-06-17T12:39:50Z

I am installing local packages in my dev-packages, too. So yes.

darrenmeehan · 2020-06-17T20:44:44Z

It seems to be related to the path being a parent directory of the directory with the Pipfile & Pipfile.lock. I haven't had more time to investigate this issue since my initial comment.

@con-f-use are you local packages also in a parent directory?

con-f-use · 2020-06-17T23:47:45Z

Basically this:

[dev-packages]
pipenv-to-requirements = ">=0.8"
# ...
framework-core = {path = ".",allow_prereleases = true,editable = true}
framework-api = {path = "../framework-api",allow_prereleases = true,editable = true}
framework = {path = "..",allow_prereleases = true,editable = true}

[packages]
azure = "==4.0.0"
# ...

I helped myself by hacking pipenv/vendor/requirementslib/models/setup_info.py:1870 via wrapping a try: shutil.copytree(path, target, symlinks=True) except OSError: pass around the copytree command for now. Pipenv then proceeds normally (from what I can tell).

The pipenv --support call hanging indefinitely seems to be different issue. @ pipenv team, is there a ticket for that? Should I create one?

hleb-albau · 2020-08-02T16:19:16Z

Same issue for local packages for package with own Pipfile.
Also works with 2018.11.26 version.

con-f-use · 2020-10-14T09:52:40Z

It's been a couple of month, can we get this fixed, please?

frostming · 2020-10-14T10:02:30Z

@con-f-use Did you try 2020.8.13 release? No longer copy tree for now.

con-f-use · 2020-10-14T11:26:29Z

I was on 2020.08.13, yes. The copytree is in one of the vendor libraries included with pipenv

┌─14102020 [(python-2.7.17) confus@confusion6 /tmp/tmp.4gYNCX9RMd]                                                                                                                                                     
└$ pipenv --version                                                                                                                                                                                                 
pipenv, version 2020.8.13

┌─14-13:25 [(python-2.7.17) confus@confusion6 /tmp/tmp.4gYNCX9RMd]                                                                                                                                                     
└$ which pipenv                                                                                                                                                                                                     
/tmp/tmp.4gYNCX9RMd/.direnv/python-2.7.17/bin/pipenv

although it is quite weird, when I look at the source:

┌─14-13:31 [(python-2.7.17) jan@confusion6 /tmp/tmp.4gYNCX9RMd]                                                                                                                                                     
└$ cd .direnv/python-2.7.17/lib/python2.7/site-packages/pipenv/  

┌─14-13:32 [(python-2.7.17) jan@confusion6 /tmp/tmp.4gYNCX9RMd/.direnv/python-2.7.17/lib/python2.7/site-packages/pipenv] [1]                                                                                        
└$ cat __version__.py                                                                                                                                                                                               
#     ___     ( )  ___      ___       __
#   //   ) ) / / //   ) ) //___) ) //   ) ) ||  / /
#  //___/ / / / //___/ / //       //   / /  || / /
# //       / / //       ((____   //   / /   ||/ /
__version__ = "2020.8.13"

┌─14-13:36 [(python-2.7.17) jan@confusion6 /tmp/tmp.4gYNCX9RMd/.direnv/python-2.7.17/lib/python2.7/site-packages/pipenv]                                                                                            
└$ grep -F copytree vendor/requirementslib/models/setup_info.py

There is indeed no copytree. Is pipenv in my virtual environment somehow importing an old version from somewhere else?!

matteius · 2022-04-22T04:11:50Z

I think this is closeable at this point, pipenv==2022.4.21 is released. Let me know if it needs to be re-opened.

triage-new-issues bot added the triage label Jun 7, 2020

darrenmeehan mentioned this issue Jun 12, 2020

Error installing the project in editable mode #4335

Closed

twitty-onica mentioned this issue Sep 8, 2020

fix YamlDirTestGenerator python 3 compatibility rackspace/runway#442

Merged

matteius closed this as completed Apr 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

File Exist error on locking #4326

File Exist error on locking #4326

con-f-use commented Jun 7, 2020 •

edited

Loading

... hangs indefinetly trying to find python installations....

darrenmeehan commented Jun 8, 2020 •

edited

Loading

munsu commented Jun 17, 2020

con-f-use commented Jun 17, 2020

darrenmeehan commented Jun 17, 2020

con-f-use commented Jun 17, 2020 •

edited

Loading

hleb-albau commented Aug 2, 2020 •

edited

Loading

con-f-use commented Oct 14, 2020

frostming commented Oct 14, 2020

con-f-use commented Oct 14, 2020 •

edited

Loading

matteius commented Apr 22, 2022

File Exist error on locking #4326

File Exist error on locking #4326

Comments

con-f-use commented Jun 7, 2020 • edited Loading

Issue description

... hangs indefinetly trying to find python installations....

darrenmeehan commented Jun 8, 2020 • edited Loading

munsu commented Jun 17, 2020

con-f-use commented Jun 17, 2020

darrenmeehan commented Jun 17, 2020

con-f-use commented Jun 17, 2020 • edited Loading

hleb-albau commented Aug 2, 2020 • edited Loading

con-f-use commented Oct 14, 2020

frostming commented Oct 14, 2020

con-f-use commented Oct 14, 2020 • edited Loading

matteius commented Apr 22, 2022

con-f-use commented Jun 7, 2020 •

edited

Loading

darrenmeehan commented Jun 8, 2020 •

edited

Loading

con-f-use commented Jun 17, 2020 •

edited

Loading

hleb-albau commented Aug 2, 2020 •

edited

Loading

con-f-use commented Oct 14, 2020 •

edited

Loading