Dash Plotly long callback error

[GeorgeAzaru]

<>

I get this error when I try to use a dash plotly app with long callback functionality:

Traceback (most recent call last):
File "cmt_appnge_copy.py", line 605, in
File "dash_callback.py", line 310, in wrap_func
File "dash\long_callback\managers_init_.py", line 83, in register_func
File "dash\long_callback\managers_init_.py", line 103, in hash_function
File "inspect.py", line 1024, in getsource
File "inspect.py", line 1006, in getsourcelines
File "inspect.py", line 835, in findsource
OSError: could not get source code

Also, after initiating the start of the executable there are two cache folders created:

• cache-directory
• cache

I do not know what the error could be.

[rokm]

Traceback (most recent call last):
File "cmt_appnge_copy.py", line 605, in
File "dash_callback.py", line 310, in wrap_func
File "dash\long_callback\managers__init__.py", line 83, in register_func
File "dash\long_callback\managers__init__.py", line 103, in hash_function
File "inspect.py", line 1024, in getsource
File "inspect.py", line 1006, in getsourcelines
File "inspect.py", line 835, in findsource
OSError: could not get source code

The long callback manager requires the source code of the python script/module where long_callback is used, and PyInstaller does not collect source code, but rather byte-compiled modules. Hence the "could not get source code" error.

If you are using long_callback in the entry-point script, then you will need to collect the entry-point script as a data file, using --add-data program_name.py;..

If the code that uses long_callback is organized into module/package, you can pass module_collection_mode dictionary to Analysis in the spec file (e.g., module_collection_mode={'myprogram_pkg': 'py'} (requires PyInstaller >= 5.3).

To illustrate on an example, suppose we have an entry point `program.py˙ (code taken from here):

program.py

# program.py
import time
import dash
from dash import html
from dash.long_callback import DiskcacheLongCallbackManager
from dash.dependencies import Input, Output

## Diskcache
import diskcache
cache = diskcache.Cache("./cache")
long_callback_manager = DiskcacheLongCallbackManager(cache)

app = dash.Dash(__name__)

app.layout = html.Div(
    [
        html.Div([html.P(id="paragraph_id", children=["Button not clicked"])]),
        html.Button(id="button_id", children="Run Job!"),
    ]
)

@app.long_callback(
    output=Output("paragraph_id", "children"),
    inputs=Input("button_id", "n_clicks"),
    manager=long_callback_manager,
)
def callback(n_clicks):
    time.sleep(2.0)
    return [f"Clicked {n_clicks} times"]


if __name__ == "__main__":
    app.run_server(debug=True)

Building with pyinstaller --clean --noconfirm program.py and running program gives us:

Traceback (most recent call last):
  File "program.py", line 27, in <module>
  File "dash\_callback.py", line 310, in wrap_func
  File "dash\long_callback\managers\__init__.py", line 83, in register_func
  File "dash\long_callback\managers\__init__.py", line 103, in hash_function
  File "inspect.py", line 1147, in getsource
  File "inspect.py", line 1129, in getsourcelines
  File "inspect.py", line 958, in findsource
OSError: could not get source code
[1508] Failed to execute script 'program' due to unhandled exception!

Collecting the entry-point script as a data file, pyinstaller --clean --noconfirm program.py --add-data program.py;. gets us past that error, but raises a new one:

Traceback (most recent call last):
  File "program.py", line 33, in <module>
    app.run_server(debug=True)
  File "dash\dash.py", line 2134, in run_server
  File "dash\dash.py", line 1896, in run
  File "dash\dash.py", line 1653, in enable_dev_tools
  File "dash\dash.py", line 1662, in <listcomp>
AttributeError: 'FrozenImporter' object has no attribute 'filename'

which looks like incompatibility between PyInstaller's FrozenImporter and attributes that dash expects to find on the loader/importer object.

This can be worked around by disabling the debug mode on dash_server, i.e., changing the last line of entry-point script to app.run_server(debug=False) and rebuilding the program.

This gets the application running and allows us to connect to its web server, but the button doesn't work. That's because behind the scenes, multiprocessing module is used, and to use multiprocessingin a PyInstaller-frozen application, you need to call multiprocessing.freeze_support at the start of the entry-point script. So the block at the end of the entry-point script needs to be changed into:

if __name__ == "__main__":
    import multiprocessing
    multiprocessing.freeze_support()
    
    app.run_server(debug=False)

After rebuilding again, the example seems to work as expected

---

If you need the debug mode (app.run_server(debug=True)), then the work-around for the second error is to collect dash and its submodules in source-only form, which bypasses the PyInstaller's FrozenImporter and uses built-in file loader. Take the .spec file that PyInstaller generated in the previous steps, and modify it to look as follows:

program.spec

# -*- mode: python ; coding: utf-8 -*-


block_cipher = None


a = Analysis(
    ['program.py'],
    pathex=[],
    binaries=[],
    datas=[('program.py', '.')],
    hiddenimports=[],
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=False,
    module_collection_mode={'dash': 'py'},  # <--- add this line; requires PyInstaller 5.3 or later
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)

exe = EXE(
    pyz,
    a.scripts,
    [],
    exclude_binaries=True,
    name='program',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)
coll = COLLECT(
    exe,
    a.binaries,
    a.zipfiles,
    a.datas,
    strip=False,
    upx=True,
    upx_exclude=[],
    name='program',
)

Then build by running PyInstaller against the spec file instead of the entry-point:

pyinstaller --clean --noconfirm program.spec

(NOTE: the above example is for onedir builds; you seem to be using onefile, so the spec will look a bit different).

---

Also, after initiating the start of the executable there are two cache folders created:

cache-directory
cache

That's probably due to how you set up your cache directories. If you did it like in the above example, i.e.,

## Diskcache
import diskcache
cache = diskcache.Cache("./cache")
long_callback_manager = DiskcacheLongCallbackManager(cache)

then cache directory will be created in the current working directory. You should use absolute paths, either anchored to __file__ (but not if you're using onefile builds, since that would place it in application's temporary directory and delete it every time after application exits) or in application-specific directory in user's home directory (e.g., somewhere in %LOCALAPPDATA%).