What if poetry.lock and poetry-core, used by the plugin, are incompatible?

[fredrikaverpil]

This is a little bit of a hypothetical question. But what would happen if the lockfile (poetry.lock) was generated with e.g. poetry 1.0 and poetry-core used by this plugin is based on poetry 2.0, with a possibility that 2.0 is not backwards compatible?

I haven't dug into the code of poetry-core or this plugin so I am not entirely sure how the mechanism works when tox installs the dev dependencies. But I presume tox will delegate to poetry-core to read the resolved lock file so to sort out which dependencies (and their versions) to install?

I guess you could mitigate this (lockfile-poetry incompatibility) by possibly pinning the tox-poetry-dev-dependencies and the poetry-core, but I am not a great fan of this approach. Because then you would have to know which version of poetry you are actually on vs the corresponding poetry-core, which might not be something which is top of mind, so to speak.

[tox]
minversion = 3.20.0
isolated_build = true
requires =
    tox-poetry-dev-dependencies==0.0.1    <--- hard pinned
    poetry-core==1.0.0b1    <--- hard pinned
provision_tox_env = .provision

So, instead, is there anything this plugin can do so to perhaps identify this type of issue?
I guess not, and maybe it shouldn't do anything, since realistically it should delegate all that to poetry-core.

Well, enough rambling from me. These are just some thoughts I brought up in the discord chat, and we can see if this is worth discussing any further here.

[sinoroc]

Thanks for the report.

First things first. Right now we do not do anything with the lock file. Well, at least nothing explicitly. So I guess pip just installs whatever it feels like installing. Which I guess right now, is the highest valid version of each requirement (and might actually let some version conflicts pass through). Unless pip's new experimental dependency resolver is enabled, in which case it does something a bit smarter (it at least should avoid version conflicts). In any case, I am pretty sure the poetry.lock file is never read.

So we should tackle that first. Maybe add a new setting use_poetry_lock_file = True.

[fredrikaverpil]

Aha. Yes, by using the lock file you would get reproducible installations, identical to what a "poetry install" would give you.

I was unaware of that this plugin just got the latest (valid) version of each dev dependency. 🙂 In CI (where I use tox primarily) I need to make sure the installed packages use the resolved versions from the lock file. This adds predictability and coherency between the dev env and CI.

I understand the plugin is more of a POC but not reading the lock file can yield false positives in tests.

Honestly, I would think a future 'use_poetry_lock_file' should be True by default.

[sinoroc]

My intention is to keep the behaviour as close as possible to Tox's default behaviour... unless it is instructed explicitly to do something specific.

By default Tox doesn't try to be clever about the versions of the dependencies, it just lets pip do its thing. So I want to keep that, and add the settings that would allow to match Poetry's behaviour, in that case: installing the versions pinned in the lock file.

But if it happens that the list of settings grows too much, then we could come up with a setting that would gather them all: match_poetry_behaviour = True, and be done with it.

If there are other things that you expect from this plugin let me know.

[sinoroc]

We would need to figure out what should happen in a case like the following:

[testenv]
deps =
    foo20: Foo==2.0
use_poetry_lock_file = True

I guess one would expect all dependencies to be installed as pinned in the lock file, except Foo.

[fredrikaverpil]

You'll have to take the subdeps of Foo into account too.

I would say first abide the lock file entirely and install everything according to it, without trying to interfere.
Then pip install anything specified in the tox "deps" (such as Foo==2.0). Pip will here take care of also installing/updating any subdeps of Foo that might differ if Foo (and it's subdeps) were already specified in the lock file.

[sinoroc]

Looks like this poetry-core change would be good for us here:

python-poetry/poetry-core#83