Tests should be run in a clean environment

[astrofrog]

Currently tests are run in the same environment that BEFORE_BUILD was run in, but it would be good to try installing the wheel into a clean environment to make sure that all the correct dependencies are installed and don't rely on BEFORE_BUILD.

[joerick]

Good point! Did you get burned by this?

…

Sent from my phone

On 9 Jan 2019, at 16:18, Thomas Robitaille ***@***.***> wrote: Currently tests are run in the same environment that BEFORE_BUILD was run in, but it would be good to try installing the wheel into a clean environment to make sure that all the correct dependencies are installed and don't rely on BEFORE_BUILD. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[astrofrog]

Somewhat - I have to currently install a more recent version of numpy manually in the test phase (since I want to test with the most recent version, not the version used for the build). So there's a workaround, but ideally we should fix this in cibuildwheel.

[YannickJadoul]

Then a CIBW_BEFORE_TEST would also be nice to have? (Maybe even defaulting to CIBW_BEFORE_BUILD for backwards compatibility?)

[joerick]

Maybe CIBW_TEST_REQUIRES supports this already? Ask for a version that's newer than whatever your built with?

[astrofrog]

@joerick - yes I can use CIBW_TEST_REQUIRES in my case but it's more that conceptually the test and build phases should be separate to make sure that the dependencies in the package are correctly defined. For example if the fast-histogram package didn't define the numpy depdendency in install_requires, I wouldn't have noticed (so I'm just pre-empting for other packages in future).

[joerick]

Yep, I agree on the broader point about test environments. I suppose the solution is to create a venv, install our new wheel in there and run the tests using that. Aside: I've never been totally happy with the strategy of moving out of the project dir and running the tests from elsewhere to avoid picking up the module (and using the {project} keyword in the test command). If the isolated test env could improve that too somehow, that would be great.

…

Sent from my phone

On 9 Jan 2019, at 23:39, Thomas Robitaille ***@***.***> wrote: @joerick - yes I can use CIBW_TEST_REQUIRES in my case but it's more that conceptually the test and build phases should be separate to make sure that the dependencies in the package are correctly defined. For example if the fast-histogram package didn't define the numpy depdendency in install_requires, I wouldn't have noticed (so I'm just pre-empting for other packages in future). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[illume]

tox might be an option for creating different environments with different pythons and running tests separately.

[joerick]

Fixed in #164, release should be soon!