Pytest 3.0.2 memory leak with pytest.raises

[MSeifert04]

• Description

I tried to identify a memory leak in one of my functions and while debugging it I realized that besides the memory leak in my function there is also a memory leak in pytest.raises.

• conda list

colorama 0.3.7
pip 8.1.2 py35_0
py 1.4.31
pytest 3.0.2
python 3.5.2 0
setuptools 27.2.0 py35_1
vs2015_runtime 14.0.25123 0
wheel 0.29.0 py35_0

Windows 10 (64 bit) but the memory leak is also visible in Travis CI ( Linux 64 bit) and AppVeyor (I think 64 bit) independant of python version. The memory leak is not present in pytest 2.9.2 and 2.6.4 (tested with Travis CI)

• Minimal example

It's not really minimal and there are probably better ways to test it:

from collections import Counter
from gc import get_objects
import pytest

def difference_tracked_objects(func, specific_object=None):
    # rather trivial attempt at finding memory leaks, this attempt does not always work! 
    # That's why I created a custom class and used the "specific_object".
    before = Counter()
    after = Counter()
    before.update(map(type, get_objects()))
    func()
    after.update(map(type, get_objects()))
    if specific_object is None:
        return after - before
    else:
        leftover = after[specific_object] - before[specific_object]
        if leftover:
            return Counter({specific_object: leftover})
        else:
            return Counter()

class Test(object):
    def __init__(self, value):
        self.value = value

def testfunc():
    func = lambda x: x.value + ''
    v = Test(10)
    with pytest.raises(TypeError):
        func(v)

difference_tracked_objects(testfunc, Test)
# Counter() (on pytest < 3.0.2)
# Counter({<class '__main__.Test'>: 1}) (on pytest == 3.0.2) <-- looks like memory leak

[nicoddemus]

Thanks for the report!

[MSeifert04]

Is there any way I can assist in fixing this issue?

[nicoddemus]

You might want to take a look at the code and perhaps submitting a PR? The relevant code for that is in python.py.

[MSeifert04]

@nicoddemus Thank you for the pointer. I submitted a PR but I think I might need some help creating a regression test. At least locally the changes seem to fix the issue.

However it's really weird that Python can't garbage the cyclic references - normally that's not a problem.

[RonnyPfannschmidt]

@MSeifert04 does happen after triggering a gc.collect?

i recall that at least on cpython full gc is only triggered on occasion because quite often recounting is more than sufficient

[MSeifert04]

i recall that at least on cpython full gc is only triggered on occasion because quite often recounting is more than sufficient

You're right gc.collect fixes the problem. 😅

[nicoddemus]

Strange, I could sworn I tried gc.collect()... oh well, glad it worked in the end. 😁