fail on zero tests found

[pytestbot]

Originally reported by: Boris Rybalkin (BitBucket: cyberb, GitHub: cyberb)

---

Sometimes it is critical to fail when py.test does not find any test. This may happen when you are accidentally running in the wrong directory on the build agent and may lead to the impression that everything is ok.

Something like:
py.test --fail-on-empy

EDIT: fail by default would be even better

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/662

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Not sure. Maybe post to the pytest-dev mailing list to discuss the case of "no tests run" or "no tests collected"? For me, it might be ok if a test run where no tests are run results in != 0 return code. Question is if there is a case where this would be unwanted. In general we try to minimize the number of command line options.

[pytestbot]

Original comment by Anatoly Bubenkov (BitBucket: bubenkoff, GitHub: bubenkoff):

---

2 hands up for this (making it fail when no tests were executed)

[pytestbot]

Original comment by Anatoly Bubenkov (BitBucket: bubenkoff, GitHub: bubenkoff):

---

and i would do this opt-out

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

in which case would it be helpful to not fail? And why can we not shift this responsibility to the executing shell (ignoring the return code)?

[pytestbot]

Original comment by Anatoly Bubenkov (BitBucket: bubenkoff, GitHub: bubenkoff):

---

i agree, ideally pytest should not bother with additional parameter for that, and just fail.
just reporter proposed opting in, and opting IN is NOT an option for me

[pytestbot]

Original comment by Boris Rybalkin (BitBucket: cyberb, GitHub: cyberb):

---

fail by default would be an ideal solution!

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

ok, so please post to pytest-dev and suggest that pytest-2.7 should change the outcome to !=0 when no tests are executed in a pytest run and see if anyone complains wrt this backward-incompatible change.

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

i still kind of agree that failing on an empty set of tests by default is probably ok but no pytest-2.7 is out. If you do a PR it should get into pytest-2.8 at least :)

[The-Compiler]

There's a pull request in #817 now.

[vladyslav-savchenko]

Hi People,

I think it seems to be a bug here: _pytest/main.py:614..620:
if self._notfound:
errors = []
for arg, exc in self._notfound:
line = "(no name %r in any of %r)" % (arg, exc.args[0])
errors.append("not found: %s\n%s" % (arg, line))
#XXX: test this
raise pytest.UsageError(*errors)
If no tests found, "python -m pytest <py_src>" returns exit_code 4(EXIT_USAGEERROR) instead of 5 (EXIT_NOTESTSCOLLECTED).

[RonnyPfannschmidt]

@vladyslav-savchenko that is intentional - passing missing files is a usage error, passing test filters that cause no tests to be is the notestscollected error

[vladyslav-savchenko]

No, files exist, I have checked that with os.path.exists.
But here (_pytest/main.py:714) "matching" and "names" are empty and "NoMatch" exception raises.

I have found the cause of this wrong behavior here (_pytest/python.py:154). I get an exit code 4 if a python source does not have an extension at all. I think it is not completely correct to detect if it is a python source just checking for its extension. Not every python script has such one, but command
> file <path>
returns nevertheless
<path>: Python script, ASCII text executable.

Unfortunately I can not suggest a better solution yet :- (

Thank you!

[nicoddemus]

Can you provide a reproducible minimal case? If so feel free to open a new issue to discuss your problem.