Pipenv can't install black because black is only a pre-release

[erinxocon]

Environment data

• VS Code version: Version 1.34.0-insider
• Extension version (available under the Extensions sidebar): 2019.3.6558
• OS and version: macOS 10.14
• Python version (& distribution if applicable, e.g. Anaconda): project is using 3.7.2, pyenv with global python set to 3.7.3
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pipenv
• Relevant/affected Python packages and their versions: black 19.3b0

Expected behaviour

Black installs ok

Actual behaviour

Black fails to install because there are only beta releases and pipenv doesn't look for those by default

Steps to reproduce:

1. When the popup asks to select a linter, choose black and install it.
2. Pipenv will fire up and try and install black as a dev package.
3. Pipenv will fail to install since --pre isn't provided

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Formatting with black failed.
You could either install the 'black' formatter, turn it off or use another formatter.
Error: Module 'black' not installed.

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

Not Applicable

Output from terminal:

pipenv install black --dev      Mon Apr  8 21:53:53 2019
Installing black…
Adding black to Pipfile's [dev-packages]…
✔ Installation Succeeded 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
✘ Locking Failed! 
[pipenv.exceptions.ResolutionFailure]:   File "/Users/erin/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv/resolver.py", line 70, in resolve
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/Users/erin/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv/utils.py", line 746, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/Users/erin/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv/utils.py", line 500, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/Users/erin/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv/utils.py", line 415, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches black
[pipenv.exceptions.ResolutionFailure]:       Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches black
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
There are incompatible versions in the resolved dependencies.
[pipenv.exceptions.ResolutionFailure]:       req_dir=requirements_dir
[pipenv.exceptions.ResolutionFailure]:   File "/Users/erin/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv/utils.py", line 746, in resolve_deps
[pipenv.exceptions.ResolutionFailure]:       req_dir=req_dir,
[pipenv.exceptions.ResolutionFailure]:   File "/Users/erin/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv/utils.py", line 500, in actually_resolve_deps
[pipenv.exceptions.ResolutionFailure]:       resolved_tree = resolver.resolve()
[pipenv.exceptions.ResolutionFailure]:   File "/Users/erin/.pyenv/versions/3.7.2/lib/python3.7/site-packages/pipenv/utils.py", line 415, in resolve
[pipenv.exceptions.ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]:       pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches black
[pipenv.exceptions.ResolutionFailure]:       Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  First try clearing your dependency cache with $ pipenv lock --clear, then try the original command again.
 Alternatively, you can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ERROR: Could not find a version that matches black
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1, 18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
There are incompatible versions in the resolved dependencies.

Proposed solution:

1. Add the --pre flag to the pipenv command when black is chosen and installed.

[erinxocon]

I propose adding a very very simple clause to https://github.com/erinxocon/vscode-python/blob/e441ed614fe9ccda1818e19acbba5ab8d82d772c/src/client/common/installer/pipEnvInstaller.ts#L33

        const args = ['install', moduleName, '--dev'];
        if (moduleName === 'black') {
            args.push('--pre');
        }
        return {
            args: args,
            execPath: pipenvName
        };

I am concerned though, this places the burden on the VS Code development team when black comes out with a non-pre version. The other option could be to instruct the user to do this manually passing the --pre flag. The reason I bring this up is that I believe, and @techalchemy can correct me if I'm wrong, but when --pre is passed as an argument this applies to the whole locking process and any package that has a pre-releases would end up being resolved and installed when the user might not expect.

[ericsnowcurrently]

@erinxocon, thanks for letting us know about this. We'll look into it.

[erinxocon]

@ericsnowcurrently thanks! I'm willing to work on a PR once a decision is reached on how to handle this edge case. I helped develop pipenv and am very familiar with it's workings 🌈🍰🌈

[DonJayamanne]

this applies to the whole locking process and any package that has a pre-releases would end up being resolved and installed when the user might not expect.

Agreed, we will have to make an exception with black
Please feel free to submita PR with the pre flag just for black.
I.e. your suggestion is perfect