Leaked file object with gzip compression

[ampanasiuk]

Problem description

I am getting a warning that a file object is not closed when doing something like

from smart_open import open


def fun():
    with open("my_file.gz", encoding="utf-8") as f:
        print(f.read())


fun()
gc.collect()

This is because closing a GzipFile instance does not close the underlying stream. I didn't check whether the same applies to bz2.BZ2File and the support for .bz2 in smart_open.

https://github.com/RaRe-Technologies/smart_open/blob/develop/smart_open/compression.py

def _handle_gzip(file_obj, mode):
    import gzip
    return gzip.GzipFile(fileobj=file_obj, mode=mode)

https://docs.python.org/3/library/gzip.html#gzip.GzipFile

Calling a GzipFile object’s close() method does not close fileobj, since you might wish to append more material after the compressed data. This also allows you to pass an io.BytesIO object opened for writing as fileobj, and retrieve the resulting memory buffer using the io.BytesIO object’s getvalue() method.

[mpenkov]

Good catch! Are you interested in making a PR to fix this?

[ampanasiuk]

Yea, I'll look at this next week.