pipenv shell starts a CMD shell when run from powershell

[pfmoore]

I use powershell as my primary shell, but pipenv shell starts a CMD prompt.

Describe you environment

1. OS Type: Windows 7
2. Python version: $ python -V 3.6.2
3. Pipenv version: $ pipenv --version 8.2.6

Expected result

I'd expect the shell that was started to be the same as my current shell (i.e., powershell)

Actual result

The shell is always a CMD shell, as far as I can see there is no way to change this.

Steps to replicate

pipenv shell. It's possible to work around this by using pipenv run powershell but this doesn't set $env:VIRTUAL_ENV (which I believe is a known issue).

[erinxocon]

@pfmoore this is actually a bug in pew. I know because I introduced it :) It will be fixed soon! :)

[pfmoore]

lol thanks for the clarification :-)

[lllama]

@erinxocon Any idea when the fix will be released? (or if there's a work around for the time being?)

[erinxocon]

@lllama no ETA yet, it's on my to do list, just been busy with work! I'll try and expedite this as I'm starting to use windows more and more for work and this bothers me too.

[lllama]

@erinxocon thanks for the reply. If anyone else is experiencing this, you can just launch powershell from within cmd and you'll still have paths and deps setup and installed.

[lllama]

If anyone else is wondering about this, then it appears to have been fixed with version 11.1.8. (or possibly earlier)

[techalchemy]

Stealth-fixed in 2dc06d4

[hstefan]

I'm still seeing this same behaviour when running powershell on Cmder, so I was wondering if there's any interested in having that fix or if users are except to use only PS or CMD. I have no idea what is causing this particular issue, but I could take a stab at it would be a desirable change.

[uranusjr]

@hstefan Cmder works perfectly for me with either cmd and Powershell, so this could be environment-dependent. Please do help improve it if possible!

[techalchemy]

@hstefan you need to provide more details, such as which version of pipenv you are using. For context, taking a stab at this would involve reimplementing the ctypes process tree traversal I implemented which initially resolved this. You should let us know what you have set for environment variables, one of those might be causing issues (@uranusjr what is your SHELL set to)?

[hstefan]

@uranusjr, @techalchemy thanks for the comments!

I'm running python 3.6.5, with pipenv, version 11.10.1, on a powershell console on Cmder v1.3.5. The behaviour I'm observing seems to happen after doing pipenv shell in the directory containing this Pipfile; once I issue the command the shell that is started is a cmd, instead of the expected powershell session.

Please let me know if there's more info that I can provide

[techalchemy]

@hstefan the output of get-childitem ENV: would be helpful

[uranusjr]

Ah, I think I know what’s going on here. On certain code paths for Cmder you go through this function in Pew:

https://github.com/pypa/pipenv/blob/v11.10.3/pipenv/patched/pew/pew.py#L185

As the name indicates, this spawns a new shell when it detects you’re using Cmder—but it always spawns a cmd, never Powershell. @hstefan Can you add a few breakpoints on Pipenv to confirm if this is indeed the code path you go through when you run pipenv shell?

This could be a little tricky to fix (since we’d need to know what Cmder shell you’re in), but I think it’s possible.

[techalchemy]

@uranusjr @hstefan this is probably due to COMSPEC settings in cmder. On my cmder setup, COMSPEC is always 'cmd.exe' and I can't seem to change it

[techalchemy]

Hm that might not be relevant anymore on second thought

[uranusjr]

Yeah COMSPEC is supposed to be always set to cmd.exe. Its purpose is simply to point to cmd.exe afaik.

[techalchemy]

I bet we can just remove the special behavior in cmder now, I am sure that was there because of the old way pew used to handle the shell determination on windows using psutil

[techalchemy]

open for tracking the cmder specific issue

[uranusjr]

We can for Powershell, as long as we detect its profile file ($CmderUserProfilePath) and load it. ⌘ is more tricky because it does not have the notion of profile files. Skip detecting it would give the user a default cmd in the subshell.

I can probably come up with something…

[hstefan]

Hey @uranusjr! It seems like we do hit the path you asked about, here's the stack trace I got from running pipenv shell with an added raise ValueError at the start of the function (sys.exc_info() was a bit noisy when running under pdb)

Traceback (most recent call last):                                                                      
  File "C:\Users\hstefan\Workspace\Personal\pipenv\venv\Scripts\pipenv-script.py", line 11, in <module> 
    load_entry_point('pipenv', 'console_scripts', 'pipenv')()                                           
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\vendor\click\core.py", line 722, in __call__  
    return self.main(*args, **kwargs)                                                                   
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\vendor\click\core.py", line 697, in main      
    rv = self.invoke(ctx)                                                                               
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\vendor\click\core.py", line 1066, in invoke   
    return _process_result(sub_ctx.command.invoke(sub_ctx))                                             
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\vendor\click\core.py", line 895, in invoke    
    return ctx.invoke(self.callback, **ctx.params)                                                      
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\vendor\click\core.py", line 535, in invoke    
    return callback(*args, **kwargs)                                                                    
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\cli.py", line 607, in shell                   
    three=three, python=python, fancy=fancy, shell_args=shell_args                                      
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\core.py", line 2206, in do_shell              
    pew.workon_cmd([workon_name])                                                                       
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\patched\pew\pew.py", line 384, in workon_cmd  
    shell(env, cwd=project_dir)                                                                         
  File "c:\users\hstefan\workspace\personal\pipenv\pipenv\patched\pew\pew.py", line 224, in shell       
    fork_cmder(env, cwd)                                                                                

[hstefan]

Also, I've tested #2161 briefly and it seems like it does fix the problem, or at least the launched shell is actually powershell. I haven't tested it thoroughly, but please let me know if there's anything specific I can help with.

[techalchemy]

@hstefan if you get a chance, just test it on as many windows shells as you can (cmder/cmd.exe/powershell/etc) since this is going to require manual checks to make sure it works. That's the main thing. I'll slot this in for 11.11

[uranusjr]

Also if anyone’s interested this would be a good chance to get Msys Git Bash support since I’m rewriting everything anyway.

[techalchemy]

@uranusjr I use msys2 with fish and run all of that via cmder. I’ll test it out tomorrow ;)

[hstefan]

@uranusjr I've tested bash on msys2 and cmder, and the weirdest thing I've seen was PATH being converted to windows-style paths and basically screwing up every binary that I previously had in PATH. Do you have plans to fix msys bash support in that same PR? Otherwise I'd be interested in trying to help out with that during my extended weekend this week.

[uranusjr]

@hstefan Give this a shot! I don’t use the Msys2 Bash much, and does not have it set up in Cmder either (I deleted the default entry), Any work from me at this front would be mostly be done in the dark; I’d very much need someone to be able to test it more extensively. I plan to work on restructuring the _detect_shell function first, so please do work on it if you can. (I’m visiting my parents this weekend and might not be working on this much during the time.)

[hstefan]

@uranusjr I've digged a bit into what was going on with bash on windows: apparently we were generating a PATH format that was not well-suited for the platform, as it would contain some entries as posix-style paths and others windows-style. I've made two changes which seem to have made msys bash work fine now (at least the version packaged with git for windows), one was using Path.home to expand to current directory on Windows and another a simplification of the temp bashrc we generate before forking, which now just does a source on existing .bashrc and piggy-backs on bash to expand $PATH when we prepend the env bin directory to it.

So, should I open a PR to your fork including said changes? Or would it be better if I just sent you a patch?

[techalchemy]

I’m guessing it’s easier to pr to the relevant branch

[uranusjr]

Yup, my changes are here (in a branch, not forked repo), so you can simply open a PR to this repo.

[andb0t]

Hey there,
I have pipenv, version 2018.11.26 installed on powershell 5.1.17763.1007 and it still spawns the CMD instead of a powershell window. This puzzles me, since I thought it was fixed. Have I missed something?