Creating thread in superset_config.py freezes app

[philipuvarov]

Bug description

Hey! This is probably on weirder side of things, but I wanted to create a job in superset_config.py to update some things and i have noticed that even though I am doing thing in a separate thread, it still hangs the whole app.

So basically if I add something like this in superset_config.py

In superset_config.py add:

import time
from threading import Thread

def _updates():
    time.sleep(1800)


data_updates = Thread(target=_updates)
data_updates.start()

the app freezes until sleep finishes. Is this an expected behavior?

How to reproduce the bug

In superset_config.py add:

import time
from threading import Thread

def _updates():
    time.sleep(1800)


data_updates = Thread(target=_updates)
data_updates.start()

App becomes unresponsive until sleep finishes even though it is executed on another thread.

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

Hey there, @philipuvarov! I'm here to help you out with any issues you're facing. I can assist with bug fixes, answer any questions you have, and guide you on becoming a contributor. Let's work together to resolve any issues you're experiencing while we wait for a human maintainer.

Creating a thread in superset_config.py causes the entire Superset app to freeze until the thread completes because the configuration file is executed in the main thread during the initialization of the application. This means that any blocking operation, such as creating and running a thread, will block the main thread and prevent the application from continuing its initialization process.

To prevent this behavior, you should avoid running blocking operations directly in the configuration file. Instead, you can defer the execution of such operations to a later point in the application's lifecycle, such as after the application has fully initialized.

Here is an example of how you can defer the execution of a blocking operation using a Flask application context:

# superset_config.py

from threading import Thread
from flask import current_app

def my_background_task():
    # Your long-running task here
    pass

def start_background_task():
    thread = Thread(target=my_background_task)
    thread.start()

def init_app(app):
    with app.app_context():
        start_background_task()

# Add the init_app function to the list of functions to be called when the app is initialized
APP_INITIALIZATION = [init_app]

In this example, the init_app function is added to the APP_INITIALIZATION list, which ensures that it will be called after the application has been initialized. The start_background_task function creates and starts a new thread for the long-running task, preventing it from blocking the main thread during the application's initialization ^[1].

To continue talking to Dosu, mention @dosu.

[philipuvarov]

^ this does not fly unfortunately.