Decorators on classes ignored

[csomh]

Hello Graham,

We met this issue when trying to upgrade our app to Fedora Linux 35 (which has mod_wsgi 4.9.0 and Python 3.10). After some research, it seems that decorators added to class definitions are somehow ignored, so any modification they would do to the class just wont happen.

For example, the following code raises a TypeError: Greeting() takes no arguments when served through mod_wsgi running on Fedora Linux 35, but works without errors on Fedora Linux 34 (mod_wsgi 4.7.1, Python 3.9).

Serving it with flask's own server or through gunicorn on Fedora Linux 35 also works fine.

Running the code in an interactive interpreter or standalone script also works fine.

from flask import Flask

def tweak_init(cls):
    def new_init(inst, name):
        inst.text = f"Hello, {name}\n"

    cls.__init__ = new_init
    return cls

@tweak_init
class Greeting:
    pass

app = Flask(__name__)

@app.route("/")
def main():
    return Greeting("Pete").text

I've prepared a reproducer to build the app above both on Fedora Linux 34 and 35.

ATM I have run out of ideas how to continue to debug this. If you could give me some pointers, I would gladly spend the time to research this more.

[GrahamDumpleton]

Ignoring mod_wsgi, what are you trying to do with this decorator in the first place?

There are better ways to monkey patch existing code, but in that very simple example you would be better off using derivation. So can you step back and explain why you are trying to override the constructor method of the class and perhaps can explain a better way of doing that for a start.

[GrahamDumpleton]

BTW, you know your Apache config is wrong. You have:

WSGIDaemonProcess app threads=4

but are lacking corresponding WSGIProcessGroup directory or process-group option on WSGIScriptAlias.

[GrahamDumpleton]

Your repository which replicates the issue also works perfectly fine for me. Try rebuilding it and use --no-cache with docker build to ensure not using some strange cached version. If you aren't using docker but podman or something else, I have no idea.

BTW, I would also recommend you ignore the system packages for mod_wsgi and pip install the latest mod_wsgi package and use mod_wsgi-express instead. It provides a much better default configuration if running mod_wsgi inside of a container.

ARG fedora_version=34
FROM fedora:${fedora_version}

RUN dnf -y install \
    httpd \
    httpd-devel \
    python3-devel \
    python3-flask \
    python3-pip \
    gcc \
    && dnf clean all

COPY app /src/app
COPY setup.py /src/
WORKDIR /src
RUN python3 setup.py install
RUN pip3 install mod_wsgi
COPY app.wsgi /usr/share/app/app.wsgi

EXPOSE 8080

USER 48

CMD ["mod_wsgi-express", "start-server", "--port", "8080", "--log-to-terminal", "/usr/share/app/app.wsgi"]

That still isn't a properly setup container for doing Python web stuff as is missing standard language locales and other stuff required when using Python in a container.

[csomh]

Ignoring mod_wsgi, what are you trying to do with this decorator in the first place?

The error we see is raised from jsonschema._types, which uses attrs to decorate the TypeChecker class. From what I could tell, attrs does similar monkey-patching as done in the reproducer.

BTW, I would also recommend you ignore the system packages for mod_wsgi and pip install the latest mod_wsgi package and use mod_wsgi-express instead. It provides a much better default configuration if running mod_wsgi inside of a container.

Using mod_wsgi-express makes the issue go away, regardless if mod_wsgi is installed from PyPI or as a system package. Using the PyPI version of mod_wsgi but running Apache directly still triggers the error on Fedora 35 (but not on Fedora 34).

On the other hand I found that the same error raised from jsonschema was seen in other projects too (poljar/weechat-matrix#293), so probably it's safe to say that the issue is not mod_wsgi related.

Thank you for your help and time spent on this!

[GrahamDumpleton]

Smells a bit like some C extension is handling an internal error from some C API call but not clearing the error state using:

PyErr_Clear();

What can occur is that if the error state is not clear but then a successful response is still returned, that existing error state can affect some operation later on and appear as an error there when it isn't related but is from something that occurred prior.

[hroncok]

FWIW, this triggers the same error:

class Greeting:
    pass

def new_init(inst, name):
    inst.text = f"Hello, {name}\n"
 
Greeting.__init__ = new_init

But this works:

class Greeting():
    __init__ = None  # <- define whatever __init__ here and it works

def new_init(inst, name):
    inst.text = f"Hello, {name}\n"

Greeting.__init__ = new_init

[hroncok]

I believe this is a bug in Python subinterpreters: https://bugs.python.org/issue46034

[srittau]

Bug was closed in favor of https://bugs.python.org/issue46006 (which has open PR python/cpython#30131).

Just for posterity, two fairly simple examples where this fails:

from dataclasses import dataclass

@dataclass
class Foo:
    tag: str
NULL_FOO = Foo("")  # Foo() takes no arguments

Because pytz does some trickery with dynamically created lazing loading list classes:

from pytz import timezone

DEFAULT_TIMEZONE = pytz.timezone("Europe/Berlin")  # pytz.exceptions.UnknownTimeZoneError

[srittau]

And as an update: The problem was fixed with Python 3.10.2.