Memory leak with PyIterProtocol over PyErr #2059
Description
Bug Description
When implementing PyIterProtocol for PyErr based on these examples from the docs, I observe a memory leak.
Other objects (e.g. usize) don't seem to cause memory leaks (though, I didn't check with #[pyclass(extends=...)].
Steps to Reproduce
use pyo3::{prelude::*, PyIterProtocol, exceptions::PyValueError};
#[pyclass]
struct Errors {
iter: std::vec::IntoIter<PyErr>,
}
#[pyproto]
impl PyIterProtocol for Errors {
fn __iter__(slf: PyRef<Self>) -> PyRef<Self> {
slf
}
fn __next__(mut slf: PyRefMut<Self>) -> Option<PyErr> {
slf.iter.next()
}
}
#[pyfunction]
fn iter() -> PyResult<Errors> {
let error = PyValueError::new_err("Failed");
Ok(Errors {
iter: vec![error].into_iter(),
})
}
#[pymodule]
fn pyo3_memory_leak_example(_py: Python, module: &PyModule) -> PyResult<()> {
module.add_wrapped(wrap_pyfunction!(iter))?;
Ok(())
}Memory measurement:
import tracemalloc
import pyo3_memory_leak_example
tracemalloc.start()
for _ in range(100000):
list(pyo3_memory_leak_example.iter())
snapshot = tracemalloc.take_snapshot()
print(snapshot.statistics('lineno')[0])It outputs the following (and the number grows with the iteration number):
/tmp/pyo3-memory-leak-example-repo/run.py:7: size=17.5 MiB, count=300081, average=61 B
Run with:
pip install maturin
maturin develop
python run.py
My expectation is that those errors are garbage collected as usual.
See the full reproduction example in this repo and the actual use case in jsonschema
Backtrace
No response
Your operating system and version
Linux archlinux 5.15.10-arch1-1 #1 SMP PREEMPT Fri, 17 Dec 2021 11:17:37 +0000 x86_64 GNU/Linux
Your Python version (python --version)
Python 3.9.9
Your Rust version (rustc --version)
rustc 1.57.0 (f1edd0429 2021-11-29)
Your PyO3 version
0.15.1
How did you install python? Did you use a virtualenv?
yay python39
Yes, I used a virtualenv.
Additional Info
No response