Add all previous tests as dependency? Or wildcard?

[JoeSc]

We have a use case where we have a very long test that we want to skip if ANY of the previous tests have failed since it will just waste time.

Would it be possible to have that added to this library?

I would envision it working like
@pytest.mark.dependenc(depends="all")

diff --git a/pytest_dependency.py b/pytest_dependency.py
index a1355c3..74b6b0d 100644
--- a/pytest_dependency.py
+++ b/pytest_dependency.py
@@ -72,6 +72,11 @@ class DependencyManager(object):
         status.addResult(rep)

     def checkDepend(self, depends, item):
+        if depends == "all":
+            for key in self.results:
+                if not self.results[key].isSuccess():
+                    pytest.skip("%s depends on all previous tests passing (%s failed)" % (item.name, key))
+
         for i in depends:
             if i in self.results:
                 if self.results[i].isSuccess():

Another option would be to support regex in the depends list. So if you wanted all previous tests you could just do "test_" if you wanted all tests that started with "test_foo_" you could add that too

[JoeSc]

See
JoeSc@ca6212e

[RKrahl]

I'm not sure if I like the idea. I see at least two issues:

1. The use of magic names. Note that "all" is a perfectly valid name for a test.

2. It changes the nature of the dependency relation in an inconsistent manner. Until now this relation is static. The dependencies are given as a fixed list of tests. It is a priorily determined whether any other test belongs to the dependencies of a given test. This is independent of the current invocation of pytest. This is by design and I have use cases that depend on this. What your are proposing here is something dynamic. The dependencies of a tests in your proposal are all the tests that have been run until now in the current invocation of pytest. This is not determined a priorily.

Some examples to illustrate the issues this:

Example for the first issue

The following is perfectly valid with the current pytest-dependency:

import pytest

@pytest.mark.dependency(name="whole")
@pytest.mark.xfail(reason="deliberate fail")
def test_a():
    assert False

@pytest.mark.dependency(name="all")
def test_b():
    pass

@pytest.mark.dependency(name="universe", depends=["all"])
def test_c():
    pass

test_c does not depend on test_a. It will be run and pass. Yes, I do recongize the difference between depends=["all"] and depends="all". With your proposal in place, we might have the following:

import pytest

@pytest.mark.dependency(name="whole")
@pytest.mark.xfail(reason="deliberate fail")
def test_a():
    assert False

@pytest.mark.dependency(name="all")
def test_b():
    pass

@pytest.mark.dependency(name="universe", depends="all")
def test_c():
    pass

Now, test_c would be skipped. I would find this confusing.

Example for the second issue

Assume your proposal to be in place. Consider:

import pytest

@pytest.mark.dependency(name="a")
@pytest.mark.xfail(reason="deliberate fail")
def test_a():
    assert False

@pytest.mark.dependency(name="b")
def test_b():
    pass

@pytest.mark.dependency(name="c", depends="all")
def test_c():
    pass

From reading the code, one would assume it to be equivalent with:

import pytest

@pytest.mark.dependency(name="a")
@pytest.mark.xfail(reason="deliberate fail")
def test_a():
    assert False

@pytest.mark.dependency(name="b")
def test_b():
    pass

@pytest.mark.dependency(name="c", depends=["a", "b"])
def test_c():
    pass

But it is not. Assume the example to be saved as test.py and running python -m pytest test.py::test_b test.py::test_c respectively, test_c would not be skipped in the first variant, but skipped in the second one. I'd call this inconsistent.

[JoeSc]

Issue 1 I could see being solved by adding a unique flag in the mark to avoid confusion @pytest.mark.dependency(all_previous=True)

Issue 2 I see no way around since all by definition is dynamic.

Do you see any way to bring something like this into pytest-dependency? Or is the static vs dynamic definition of dependencies too great of barrier.

I have other uses for pytest-dependency so it'd be nice to keep it all clean with dependency marks. The only alternative I can see is using the request fixture

def test_d(request):
   if request.session.testsfailed:
       pytest.skip()
   pass