Unroll the iterable for all/any calls to get better reports

[Tadaboody]

Sometime I need to assert some predicate on all of an iterable, and for that the builtin functions all/any are great - but the failure messages aren't useful at all!
For example - the same test written in three ways:

• A generator expression

    def test_all_even():
        even_stevens = list(range(1,100,2))
>       assert all(is_even(number) for number in even_stevens)
E       assert False
E        +  where False = all(<generator object test_all_even.<locals>.<genexpr> at 0x101f82ed0>)

• A list comprehension

    def test_all_even():
        even_stevens = list(range(1,100,2))
>       assert all([is_even(number) for number in even_stevens])
E       assert False
E        +  where False = all([False, False, False, False, False, False, ...])

• A for loop

    def test_all_even():
        even_stevens = list(range(1,100,2))
        for number in even_stevens:
>           assert is_even(number)
E           assert False
E            +  where False = is_even(1)

test_all_any.py:7: AssertionError

The only one that gives a meaningful report is the for loop - but it's way more wordy, and all asserts don't translate to a for loop nicely (I'll have to write a break or a helper function - yuck)
I propose the assertion re-writer "unrolls" the iterator to the third form, and then uses the already existing reports.

• Include a detailed description of the bug or suggestion
• pip list of the virtual environment you are using

Package        Version
-------------- -------
atomicwrites   1.3.0  
attrs          19.1.0 
more-itertools 7.0.0  
pip            19.0.3 
pluggy         0.9.0  
py             1.8.0  
pytest         4.4.0  
setuptools     40.8.0 
six            1.12.0 

• pytest and operating system versions
  platform darwin -- Python 3.7.3, pytest-4.4.0, py-1.8.0, pluggy-0.9.0
• Minimal example if possible

[danielx123]

Hello, I am new here and would be interested in working on this issue if that is possible.

[blueyed]

@danielx123
Sure! But I don't think this is an easy issue, since it involved the assertion rewriting - but if you're familar with Python's AST and pytest's internals feel free to pick this up.
We also have a tag "easy" for issues that are probably easier for starting contributors: https://github.com/pytest-dev/pytest/issues?q=is%3Aopen+is%3Aissue+label%3A%22status%3A+easy%22

[Tadaboody]

I was planning on starting a pr today, but probably won't be able to finish it until next week - @danielx123 maybe we could collaborate?

[danielx123]

Yes, I can work on this during the weekend.

[nicoddemus]

This was introduced in #5103, but reverted in #5373 due to a number of regressions:

• Assertions on nested comprehensions can give NameError #5370
• Unexpected behaviour for generator expressions in python 3.7 #5371
• all(...) rewrite produces incorrect results #5372
• pytest 4.6.0 breaks matplotlib test gathering with TypeError: cannot unpack non-iterable NoneType object #5358

If/when this is re-introduced, we should make sure all the cases reported on those issues are tested.

[RonnyPfannschmidt]

we should solve this via creating an api to have collections and collection tests be expressed in a general manner ,
assert x == sorted(x) should work just as well as assert all(x)

[Tadaboody]

Do you mean an api for internal tests or an api meant for users?

[RonnyPfannschmidt]

meant for users