multidict 4.7.1 causes segfault when bundled with PyInstaller

[gmarull]

Describe the bug

After upgrading to multidict 4.7.1 we have found that the applications we bundle using PyInstaller crash. Actually, I discovered this problem because we use aiohttp which depends on multidict. Version 4.6.1 does not have this problem.

To Reproduce

1. Create a new virtual environment
2. pip install multidict==4.7.1 pyinstaller
3. Create a simple test.py script with:

print("Hello!")
import multidict

4. Bundle it pyinstaller test.py
5. Run it:

./dist/test/test
Hello!
[1]    22548 segmentation fault (core dumped)  ./dist/test/test

Expected behavior

Application prints "Hello!" and quits nicely.

Logs/tracebacks

Provided above.

Your version of the Python

Observed with 3.6.9 and 3.7.5

Your version of the multidict distribution

4.7.1

Additional context

N/A

[razaqq]

Im getting this with every version > 4.6.1 (4.7.0 and 4.7.1).
Took me like 6 hours to find out it was multidict causing this...

[asvetlov]

Thanks for the report!

[Varriount]

This also happens when multidict is used with nuitka.

@razaqq A temporary workaround is to set the MULTIDICT_NO_EXTENSIONS environment variable to something before installing via pip, and to then pip install --no-binary multidict multidict (you might need to add --upgrade --force-reinstall --ignore-installed too, if the package is already installed).

I encountered this problem, and changed the install lines in my CI/CD pipeline to:

export MULTIDICT_NO_EXTENSIONS=1
pip install --upgrade --force-reinstall --ignore-installed --no-binary :all: <package_dir>

[Varriount]

@asvetlov GDB states that the segfault occurs at https://github.com/aio-libs/multidict/blob/master/multidict/_multidict.c#L1595 , though I don't know if that's 100% accurate, as I wasn't working with a debug build (I don't think).

[asvetlov]

Decref of static type is definitely a mistake but it happens on import failing only as I see.
I wonder why the import multidict fails at all?

[iemelyanov]

Decref of static type is definitely a mistake but it happens on import failing only as I see.
I wonder why the import multidict fails at all?

The fail happens here - https://github.com/aio-libs/multidict/blob/master/multidict/_multidict.c#L1494. And only when using pyinstaller.

[asvetlov]

@iemelyanov thanks!
I'll have time to work on the fix in 2 hours.

[Oberon00]

A segfault with 4.7.0 also occurred in open-telemetry/opentelemetry-python#329. This is not using py installer or any other bundling tool.

[philpax]

Can confirm that we've been getting random segfaults with 4.7.1; downgrading to 4.6.1 fixed the issue for us. This is through regular pip installation.

[kkagill]

In my case, it crashed with slack-client 2.5.0. Downgrading to 4.6.1 fixed the problem.

[asvetlov]

Fixed by #438