specifying an order of install with requirements.txt

[tisdall]

My issue is specifically with using Cython. Several packages I have make use of Cython to compile package components only if it's available. If the package isn't available it uses a pure-python approach. So, I'd like to make sure Cython is installed first before those other packages so it properly compiles C extensions.

I saw another issue related to this but it was specifically talking about utilizing order to fix dependency issues. This issue is when it's a package that's not a dependency but if it exists it's utilized.

I stuck Cython at the top of the list but it seems to actually have installed it dead last instead of first. I notice that the order of install was opposite of the order of the file, so I then stuck Cython at the bottom and it seems to be the second one to be installed. It seems like if packages have no interdependencies it should install them in the order they're found in the file (and not reverse order).

[risingmoon]

This has bothered me for some time. It would be nice if there was a pip freeze that wouldn't always necessary print them by alphabetical order but rather top down or bottom up.

I'll probably write in a command option and do a pull-request.

Stay tuned.

[tisdall]

I'm more concerned about going the other direction. Cases where a package
isn't a requirement but are made use of if available (like Cython).

However, if what I'm suggesting is in place, then it would be good if
"freeze" returned the packages in the order they were installed. But, I'd
that info even available? Could it be assumed from file timestamps?
On Jan 30, 2015 10:46 PM, "Justin Lee" notifications@github.com wrote:

This has bothered me for some time. It would be nice if there was a pip
freeze that wouldn't always necessary print them by alphabetical order but
rather top down or bottom.

I'll probably write in a command option and do a pull-request.

Stay tuned.

—
Reply to this email directly or view it on GitHub
#2362 (comment).

[risingmoon]

That's actually not a bad idea; pip freeze based on file timestamp. Currently it's not in there; pip freeze right now actively sorts alphabeticaly. It's definitely its doable.

So what you're saying is a way to specify certain packages as not required but possibly recommended? Like IPython? I don't need it, but it's definitely nifty at times to install it.

[tisdall]

Yes and no. There's no way here to specify a package will use Cython if
available, but of a particular order is maintained then you know it'll be
used. Also, with what you're saying, duplicating an environment will more
likely to be a duplicate. (otherwise, your current environment may have
used Cython, but a freeze and install later may mean the new environment
didn't make use of it)
On Jan 31, 2015 3:58 PM, "Justin Lee" notifications@github.com wrote:

That's actually not a bad idea; pip freeze based on file timestamp.
Currently it's not in there; pip freeze right now actively sorts
alphabeticaly. I definitely its doable.

So what you're saying is a way to specify certain packages as not required
but possibly recommended? Like IPython? I don't need it, but it's
definitely nifty at times to install it.

—
Reply to this email directly or view it on GitHub
#2362 (comment).

[dstufft]

The order of installation can not be depended on, in the future this is likely to get changed so it can build/install in parallel. I don't see us changing this at any point.

[tisdall]

As for me, I think I'd rather have a convenient way of duplicating a build environment then to have some speed increase from parallel installation. If the order the packages are installed affects the state of the environment (which even though it probably shouldn't does occur), wouldn't it make sense to have some way to enforce that order? Perhaps some method of grouping or giving a priority score?

[tisdall]

Has the policy changed? It seems like the latest release of pip installs in the order it finds them in the file.

[Demetrio92]

I just drop by to say it's horrible. I have a requirements.txt which just won't install automatically because package X depends on cython. Both of them are in the requirements.txt, but there is no way to tell pip to install cython first (even though cython is alphabetically above the package X). So the line with cython in the file is simply useful. Everyone trying to set up the dev enviroment will figure out they need it.

[tisdall]

I've switched to having a "requirements-buildenv.txt" and then a "requirements.txt". The first has such things as pip and cython versions and the latter is everything else that can be installed in any order.

[Demetrio92]

@tisdall I forgot to link this issue apparently. I did not just rant, I also created a feature request. Apparently it's been solved in PEP 518 (hooray!).

see #5761 for a discussion

[tisdall]

@Demetrio92 - thanks for linking that! I wasn't aware of that PEP. I'm not sure it helps when trying to pin all dependencies to particular versions to make sure your dev environment matches your production environment (which is what I use the requirements.txt for, as well as just collecting dependencies).

[Demetrio92]

@tisdall seems to be able to specify versions the same way requirements.txt would do

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.