Calling pytest.skip no longer works at module level in pytest 3

[Nikratio]

Consider the following test:

$ cat test_pytest.py 
#!/usr/bin/env python3
import pytest
pytest.skip("This test module isn't working yet")

In pytest 2.9.1 it works just fine:

$ python3 -m pytest test_pytest.py 
================================= test session starts =================================
platform linux -- Python 3.4.2, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
rootdir: /home/nikratio/tmp, inifile: 
plugins: catchlog-1.2.2
collected 0 items / 1 skipped 

============================== 1 skipped in 0.00 seconds ==============================

but in pytest 3 it fails:

$ python3 -m pytest test_pytest.py 
================================= test session starts =================================
platform linux -- Python 3.4.2, pytest-3.0.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/nikratio/tmp, inifile: 
collected 0 items / 1 errors 

======================================= ERRORS ========================================
___________________________ ERROR collecting test_pytest.py ___________________________
Using @pytest.skip outside of a test (e.g. as a test function decorator) is not allowed. Use @pytest.mark.skip or @pytest.mark.skipif instead.
!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.08 seconds ===============================

Note, however, that calling pytest.skip inside a test continues to work with both pytest 2 and pytest 3:

$ cat test_pytest2.py 
#!/usr/bin/env python3
import pytest

def test():
    pytest.skip("This test module isn't working yet")

So I think even if calling pytest.skip at module level is deliberately no longer allowed, the error message is misleading. I'm not using it as a decorator, and using it imperatively inside a test function is apparently allowed too.

[RonnyPfannschmidt]

The Main Problem is decorator usage is a huge part oft misstakes that people do very regular

At module level you can always use a marker just as well

[nicoddemus]

Hi @Nikratio,

Just to add what @RonnyPfannschmidt said, you can use a marker instead:

if SOME_CONDITION:
    pytestmark = pytest.mark.skip('skipping entire module')

This will apply the pytest.skip mark to all tests in a module.

So I think even if calling pytest.skip at module level is deliberately no longer allowed, the error message is misleading. I'm not using it as a decorator, and using it imperatively inside a test function is apparently allowed too.

I disagree that the documentation is misleading: Using @pytest.skip outside of a test implies that using it inside a test is OK, and e.g. as a test function decorator is just an example.

That's not to say that it can be improved. A PR improving the wording would be welcome!

I think we can close this for now, feel free to re-open it if you still have questions or comments.

[Nikratio]

I'm mostly objecting to including the decorator-@ in the error message. To me that reads as if pytest thinks that I used pytest.skip as a decorator and is objecting to this specific use. I think a better message would be

Using pytest.skip outside of a test is not allowed. If you are trying to decorate a test function, use the @pytest.mark.skip or @pytest.mark.skipif decorators instead.

(note the use of pytest.skip instead of @pytest.skip).

[nicoddemus]

Oh I agree that's a nicer message. Would you mind submitting a PR? 😁