bz2.BZ2File.__init__() cannot be called twice with non-existent file

[Level]

BPO	19878
Nosy	@vstinner, @ned-deily, @serhiy-storchaka, @vajrasky
Files	fix_segfault_in_bz2_init_non_existent_file.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-12-08.14:49:31.829>
created_at = <Date 2013-12-03.18:51:08.909>
labels = ['interpreter-core', 'type-crash']
title = 'bz2.BZ2File.__init__() cannot be called twice with non-existent file'
updated_at = <Date 2013-12-10.16:23:19.713>
user = 'https://bugs.python.org/Level'

bugs.python.org fields:

activity = <Date 2013-12-10.16:23:19.713>
actor = 'python-dev'
assignee = 'nadeem.vawda'
closed = True
closed_date = <Date 2013-12-08.14:49:31.829>
closer = 'python-dev'
components = ['Interpreter Core']
creation = <Date 2013-12-03.18:51:08.909>
creator = 'Level'
dependencies = []
files = ['32965']
hgrepos = []
issue_num = 19878
keywords = ['patch']
message_count = 8.0
messages = ['205137', '205143', '205149', '205151', '205159', '205194', '205568', '205834']
nosy_count = 7.0
nosy_names = ['vstinner', 'nadeem.vawda', 'ned.deily', 'python-dev', 'serhiy.storchaka', 'vajrasky', 'Level']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue19878'
versions = ['Python 2.7']

[Level]

[level@<removed> fuzz]# cat pyfile.py
import bz2
obj = bz2.BZ2File('/tmp/fileName')
obj.__init__("fileName")
obj.__reduce__
[level@<removed> fuzz]# gdb --args python pyfile.py
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/python...(no debugging symbols found)...done.
Missing separate debuginfos, use: debuginfo-install python-2.6.6-37.el6_4.i686 python-2.6.6-37.el6_4.x86_64
(gdb) r
Starting program: /usr/bin/python pyfile.py
[Thread debugging using libthread_db enabled]
Traceback (most recent call last):
  File "pyfile.py", line 3, in <module>
    obj.__init__("fileName")
IOError: [Errno 2] No such file or directory: 'fileName'

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7a98170 in PyFile_DecUseCount () from /usr/lib64/libpython2.6.so.1.0
(gdb)

[ned-deily]

Sorry, the Python 2.6 series is now officially retired. As of 2.6.9, "All official maintenance for Python 2.6, including security patches, has ended." If you can reproduce the problem with a currently supported version of Python, such as Python 2.7.6 or 3.3.3, please reopen with similar documentation.

http://www.python.org/download/releases/2.6.9/

[vstinner]

I can reproduce the issue with Python 2.7. The problem is that BZ2File.__init__() doesn't reset the object when __init__() is called twice.

For example, the following script fails with "too many open files" error, before the previous file is not called:
---

import bz2
obj = bz2.BZ2File('bla.bz2')
for loop in range(1024*10):
    obj.__init__('bla.bz2')

---

By the way, why do you call __init__() twice? Why not creating a new object?

BZ2File was rewritten in pure Python in Python 3.3. Python 3.3+ is not affected by this issue.

[Level]

I was fuzzing the interpreter otherwise it would init itself

[nadeemvawda]

It appears that this *does* affect 2.7 (though not 3.2, 3.3 or 3.4, fortunately):

    ~/src/cpython/2.7☿ gdb --ex run --args ./python -c 'import bz2; obj = bz2.BZ2File("/dev/null"); obj.__init__("")'
    «... snip banner ...»
    Starting program: /home.u/nadeem/src/cpython/2.7/./python -c import\ bz2\;\ obj\ =\ bz2.BZ2File\(\"/dev/null\"\)\;\ obj.__init__\(\"\"\)
    [Thread debugging using libthread_db enabled]
    Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    IOError: [Errno 2] No such file or directory: ''

Program received signal SIGSEGV, Segmentation fault.
0x0000000000431d3e in PyFile_DecUseCount (fobj=0x0) at [Objects/fileobject.c:89](https://github.com/python/cpython/blob/main/Objects/fileobject.c#L89)
89          fobj->unlocked_count--;

[vajrasky]

Here is the preliminary patch.

[python-dev]

New changeset 55a748f6e396 by Nadeem Vawda in branch '2.7':
Closes bpo-19878: Fix segfault in bz2 module.
http://hg.python.org/cpython/rev/55a748f6e396

[python-dev]

New changeset 3337298f5c75 by Nadeem Vawda in branch '2.7':
Skip test for bpo-19878 on Windows.
http://hg.python.org/cpython/rev/3337298f5c75