pn.state.onload doesn't work with threads

[armaaar]

ALL software version info

Python 3.10.13
Panel: 1.3.1
Bokeh: 3.3.0
Param: 2.0.0

Description of expected behavior and the observed behavior

By enabling multi-threading (using pn.config.nthreads), I expect differnet callbacks registered with pn.state.onload to run in parallel on different threads. But currently they all run on the main process only.

Complete, minimal, self-contained example code that reproduces the issue

import os
import threading
import time

import panel as pn
from panel.template import MaterialTemplate
from panel.viewable import Viewer


def slow_task():
    pid = os.getpid()
    tid = threading.get_ident()

    print(time.time(), f"{pid=}, {tid=} working slowly ...")
    time.sleep(5)
    print(time.time(), f"{pid=}, {tid=} done.")


class Page(Viewer):
    sidebar_content = pn.Column("Loading...")
    main_content = pn.Column("Loading...")

    def __init__(self, **params):
        super().__init__(**params)

        print("curdoc:", pn.state.curdoc)

        def sidebar_cb():
            print("Sidebar CB")
            slow_task()
            self.sidebar_content.clear()
            self.sidebar_content.append("# Sidebar")

        pn.state.onload(sidebar_cb)

        def main_cb():
            print("Main CB")
            slow_task()
            self.main_content.clear()
            self.main_content.append("# Main")

        pn.state.onload(main_cb)

    def __panel__(self):
        template = MaterialTemplate()

        template.sidebar.append(self.sidebar_content)
        template.main.append(self.main_content)

        return template


pn.config.nthreads = os.cpu_count()

print("nthreads:", pn.config.nthreads)
pn.serve(Page)

Stack traceback and/or browser JavaScript console output

> python issue.py
nthreads: 10
Launching server at http://localhost:53940
curdoc: <bokeh.document.document.Document object at 0x142cc9270>
Sidebar CB
1699268606.199587 pid=33641, tid=6225473536 working slowly ...
1699268611.2046762 pid=33641, tid=6225473536 done.
Main CB
1699268611.22032 pid=33641, tid=6225473536 working slowly ...
1699268616.2236311 pid=33641, tid=6225473536 done.

Screenshots or screencasts of the bug in action

• I may be interested in making a pull request to address this

[philippjfr]

Thanks, I was confused. It looks like the onload callbacks all run on the same separate thread but aren't scheduled concurrently.

[philippjfr]

Note that I did not enable this by default in #5862 because people might have relied on consecutive execution. So to enable this you will have to do pn.state.onload(callback, threaded=True).

[ratanasov]

Thanks, @philippjfr !

Would it be easy to add a similar option to pn.state.schedule_task, to be able to also run background tasks on the Panel thread pool, e.g. to (refresh a) cache.

[philippjfr]

Seems like a good feature request, mind filing a separate issue for that? Doesn't have to be super descriptive or anything.

[MarcSkovMadsen]

Did you consider setting making a deprecation warning for threaded=False? And then in for example 1.5 use True as default value.