Watch mode to rebuild all files & restart everything on file changes

[wilzbach]

Hi,

So I am a big fan of tooling that auto-reloads everything on a change.
Thus I'm currently using this simple setup to automatically rebuild the JS and Python bundles + restart the web server.

1) Running webpack in watch mode

npx webpack --mode development -w

2) Rebuilding the meta information on every change

while true; do
    npm run build:py;
    inotifywait --event modify,create,delete,delete_self,close_write,move,move_self -q **/*.js ;
done

3. Reloading the server on external changes

---

gunicorn supports this partially out of the box, e.g.

gunicorn --reload --reload-extra-file my_dash_component/ExampleComponent.py usage:server 

Though of course this won't refresh your browser.

Ideas

While for many users the webpack-serve (as part of npm run start) will do just fine, I think the user experience could be improved with:

1. The metajson generation could be hooked into Webpack's lifecycle as a plugin (see e.g. https://webpack.js.org/concepts/plugins/) to achieve a single npm run app in an easy way.

2a) build:py could touch usage.py, s.t. even an out of the box Flask does a reload when a new meta extraction has been run
2b) Alternatively, one could pass the generated Python modules in my_dash_component to Flask as extra_files, s.t. its built-in reloader takes notice of them (see e.g. http://werkzeug.pocoo.org/docs/0.14/serving and http://flask.pocoo.org/docs/1.0/api/#flask.Flask.run)

3. Not sure what your plans on supporting hot reloading in the browser with Dash, but that would make it awesome to develop (though I can see it's on your list: Dash Dev Tools dash#292)

What are your thoughts on this? (1) and (2) are easy and I would be happy to send quick PRs for this.

[ned2]

Hot reloading is more than on the list of TODOs, there's a work in progress PR: plotly/dash#362 that has seen substantial development. Read through the discussion to find the latest release candidate that you can pip install to try out.

[T4rk1n]

If the py files are regenerated with changes and used by the running code, it will be picked up by the reloader in plotly/dash#362

2a: Why would it touch usage.py ?
2b I thought about adding the components packages dir to the watch of the hot-reload for the js bundles, don't know if it will work with eggs.

Also, this repo is being transformed in a cookiecutter #14.

[wilzbach]

2a: Why would it touch usage.py ?

Well werkzeug's default auto-reloader stat just iterates over the list of all imported module files sys.modules.values() (see https://github.com/pallets/werkzeug/blob/bd0e25ac543363184a0bfb6d074b0a8cb8f10cdc/werkzeug/_reloader.py#L19). And the custom import of dash's component loader in my_dash_component/__init__.py doesn't add the imported components to Python's globals sys.modules and the init script just adds the components as attributes:

dash-component-boilerplate/my_dash_component/__init__.py

Lines 46 to 47 in eaceef1

	for _component in _components:
	setattr(_this_module, _component.__name__, _component)

Which is why werkzeug's reloader will never listen to them. Of course, touching usage.py or adding them with extra_files are just workarounds and I think the proper solution would be to add the imported modules to sys.modules, s.t. Flask's reloader works out of the box with imported components.

One solution would be to add the imported module to sys.modules in the Dash's component_loader.
Alternatively, to show what I'm trying to go here's a PoC hack that adds a fake module to Python's global module dict, s.t. werkzeug will be able to find it and thus watch it for changes:

--- a/my_dash_component/__init__.py
+++ b/my_dash_component/__init__.py
@@ -6,6 +6,9 @@ import json
 
 import dash as _dash
 
+import types
+from os import path
+
 if not hasattr(_dash, 'development'):
     print('Dash was not successfully imported. '
           'Make sure you don\'t have a file '
@@ -47,3 +50,7 @@ for _component in _components:
     setattr(_this_module, _component.__name__, _component)
     setattr(_component, '_js_dist', _js_dist)
     setattr(_component, '_css_dist', _css_dist)
+    module_name = "%s.%s" % (__name__, _component.__name__)
+    fake_module = types.ModuleType(module_name)
+    setattr(fake_module, "__file__", path.join(path.dirname(path.abspath(_this_module.__file__)), "%s.py" % _component.__name__))
+    _sys.modules[module_name] = fake_module

there's a work in progress PR: plotly/dash#362 that has seen substantial development. Read through the discussion to find the latest release candidate that you can pip install to try out.

That's pretty cool and works nicely (awesome work!), but as long as the generated components aren't imported to Python's global sys.modules, werkzeug's default watcher won't listen to them. So changed metadata like e.g. new attributes still require a manual server restart.

[T4rk1n]

You looking at the outdated __init__, in #14 it is changed to match the one in dash-core-components. The py files are all imported in the __init__ and I get hot reloading when running build:py in a local install of dcc.

It doesn't need to touch usage.py, if you running usage.py with debug=True and the dash hot-reload rc versions, it will reload it just fine.

[T4rk1n]

Hot reload now reloads when the bundle is rebuilt.