Recwarn empty

[pytestbot]

Originally reported by: Danilo Bellini (BitBucket: danilobellini, GitHub: danilobellini)

---

Problem: Test if a function "creates" warnings or if it don't create any.

Solution:

1. For now, I'm not using "recwarn", since it didn't work in my code.
2. Call the warning creator routine inside a warnings.catch_warnings context manager.
3. Call warnings.simplefilter("always") before the context manager (somehow, it don't work when "inside" the context manager).

The code is in:

https://github.com/danilobellini/audiolazy/blob/0fe2586bb81d9bc266998ce478a267a2659d4076/audiolazy/test/test_stream.py#L235

However, I think I should be using the recwarn funcarg. I tryed this:

class TestThub(object):

    @p("copies", range(5))
    @p("used_copies", range(5))
    def test_stream_tee_hub_memory_leak_warning_and_index_error(
            self, copies, used_copies, recwarn):
        data = Stream(.5, 8, 7 + 2j)
        data = thub(data, copies)
        assert isinstance(data, StreamTeeHub)
        if copies < used_copies:
            with pytest.raises(IndexError):
                [data * n for n in xrange(used_copies)]
        else:
            [data * n for n in xrange(used_copies)]
            data.__del__()
            if copies != used_copies:
                w = recwarn.pop()
                assert issubclass(w.category, MemoryLeakWarning)
                assert str(copies - used_copies) in str(w.message)
            assert recwarn.list == []

That did't work: the test in the link works, the test above "misses"/filters some warn() calls, and an error is given when "recwarn.pop()" is called with an empty recwarn. But only when used_copies > 0 (i.e., it works when used_copies == 0 and copies > 0).

Also, the recwarn.list isn't documented anywhere. I wanted to assert "there's no warning" and found no documented way with recwarn. Perhaps I can use the pop() method in a try block, but I think the list comparison is simpler.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/228

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

It seems recwarn needs some work. I am not sure what is causing the problem you are describing, though. I am hardly using warnings anymore these days. Given you are using it, you might want to check if something obvious is wrong or could be improved in _pytest/recwarn.py.

[pytestbot]

Original comment by BitBucket: peritus, GitHub: peritus:

---

This seems related: http://stackoverflow.com/questions/2390766/how-do-i-disable-and-then-re-enable-a-warning

I had to do call

mymodule.__warningregistry__.clear()

for my tests to be properly isolated.

At first neither recwarn nor the warnings context manager worked as expected. Maybe that could be handled by pytest ? Also note that my tests only failed without "-k" selection. When I selected "-k warnings" all tests passed as expected.

[pytestbot]

Original comment by Danilo Bellini (BitBucket: danilobellini, GitHub: danilobellini):

---

The __warningregistry__ could be otherwhere. By default, it would be in the caller globals, which might be a function __global__ / func_globals. I already solved that for what I needed due to the "default" filtering action: https://github.com/danilobellini/audiolazy/blob/c7b4e19ada89dd76ef583526bff8b9ae7b07b9a5/audiolazy/tests/test_stream.py#L458

The "works only with -k" behavior isn't so weird if you know what's happening. That happened with me in the past, as well. The "-k" selects some tests, so it's deselecting a test that would raise a warning, and you should know that:

One thing to be aware of is that if a warning has already been raised because of a once/default rule, then no matter what filters are set the warning will not be seen again unless the warnings registry related to the warning has been cleared

Source: http://docs.python.org/library/warnings.html

So avoiding the "default", "module" and "once" actions are the only solution to that, as these would create/fill the __warningregistry__ dict. Otherwise, you need to clean this dict on every time you're testing a warning. This can be called a pre-filter behavior, not written in the PEP http://legacy.python.org/dev/peps/pep-0230/ but implemented in both CPython (2 and 3) and PyPy.

The recwarn fixture makes/restores a backup of your filters and sets the action to "default", but doesn't change the several possible registry dicts written. To avoid cleaning these you have to:

1. Ensure that all other tests that might raise a specific warning just do so in another action (not "default", nor "module", nor "once").
2. Ensure no warning happens in the specific "recwarned test" while the filters actions aren't properly configured (again, not that 3 ones).
3. Same to 1, but for the test collecting and pytest loading, these shouldn't raise the specific warning you're trying to test. That's not a problem for you since "-k" can switch the tests results, but a warning raised while collecting wouldn't let recwarn ever find the warning.

Still another solution (or perhaps a hack) is to give a different message for every single warning done (and that was how I've found this all).

As this warning behavior is innerently a collateral effect, it would be nice avoiding it to happen in other tests, but to make a more robust and self-contained test today, the __warningregistry__ should always be cleaned in a test that asserts whether a warning was raised.

If you don't like any of these solutions, you can also mock the warnings.warn instead of using the recwarn fixture, something that would bypass both the filter and pre-filter behaviors. The first link I've provided in this comment do both the recwarn and the mock/monkeypatch testing.

PS: Sorry, I'm not entering StackOverflow again after my posts and comments there started to be downvoted/deleted just because I've cited contents I've made, and I feel I can't talk there about projects I've opensourced or helped with code. Ok, a diamond moderator also protected the one who said would retaliate me, but let's get back to the recwarn issue. From your comment, I think I know what you wrote there. If I'm wrong, please copy and paste the link contents here.

[RonnyPfannschmidt]

@danilobellini is this still an issue?

[RonnyPfannschmidt]

closing this one as unfixable from python, #840 seems a good followup