Pex should support creating a venv directly

[zmanji]

With the lockfile and venv features it's very easy now to create reproducible venvs with pex. It's very handy for setting up development environments without Docker and you can ensure all of the dependencies are wheels and not sdists.

I have been doing this:

1. Create a lock file with pex3 lock create. This lockfile contains the resolves for all third party dependencies for all platforms I care about. I can use --no-build to ensure all of the dependencies are wheels.
2. With the above lockfile create a dependencies pex with pex --lock pex.lock --venv -o depencies.pex
3. Create the venv: PEX_TOOLS=1 ./dependencies.pex venv ./venv
4. Iterate on my first party code by running PEX_EXTRA_SYS_PATH='./srcs' ./out.pex.venv/pex -m 'mymodule'. In my case srcs contains multiple packages.

I install the virtualenv for two reasons:

1. Having a virtualenv allows for tools like PyCharm to pick up all of my dependencies.
2. On my machine, I have observed that executing the pex directly has about 30ms of overhead which is just enough for me to go through with step 3 above.

I think pex should be able to collapse steps 2 and 3 above into a single command invocation. I would envision something like:

pex3 venv create --lock pex.lock -o venv

[jsirois]

Although you can command line golf today:

$ pex3 lock create --no-build cowsay -o lock.json
$ PEX_TOOLS=1 pex --lock lock.json --include-tools -- venv cowsay.venv
$ ./cowsay.venv/bin/cowsay Obscure
  _______
| Obscure |
  =======
       \
        \
          ^__^
          (oo)\_______
          (__)\       )\/\
              ||----w |
              ||     ||

I agree this would be useful. Your suggestion of pex3 venv ... sounds about right as a place for this to live.

[jsirois]

FWIW @zmanji the new --sh-boot combined with --venv should drop the overhead of warm invocations over native venv to ~1ms. So FYI for those cases where you don't need to know the venv path to tell an IDE.

[jsirois]

I ended up providing pex3 lock {export,subset,venv} ... instead. The export sub-command already existed but now supports passing -r requirements files to use to subset the lock. The subset and venv commands are new and both support optional sub-setting. The subset sub-command creates a new lockfile with a transitive closure subset of the original lock file. The venv sub-command is just like the PEX_TOOLS venv tool, except that it is faster. I'm using the latter with nox which has both sped up session set up enough that running my full suite is now ~40% faster than using nox's session.install(...) and made all tooling and packaging reproducible. PR(s) coming shortly.