PyFunc/DrawFunc dynamic reloading

[bwsw]

A developer needs instruments to develop interactively. However, GStreamer/DeepStream takes time to initialize, which makes the development less productive. To overcome that, we must introduce a new feature allowing the pipeline to reload Python functions specified in YAML and the draw_function to be reloaded when the changes are detected.

The function is optional and configured with a global parameter defined in YAML: dynamic_reload: bool. If it is set to true the feature is enabled; otherwise, the code doesn't reload on change. By default, the flag must be set to false which is overridable with an environment variable DYNAMIC_RELOAD.

Dynamic reloading works on a per-message basis (that is why it is important to make it in a way to avoid extra checks when dynanic_reload is set to false). The framework, when forwards a message to a particular pyfunc checks if the corresponding module is changed. If there is a change, it reloads the module and recreates the Object according to the manifest (kwargs, yada-yada). The state is not copied from the old object. Finally, it replaces the old object instance with a new one and forwards the traffic.

It is important to implement an ultimate exception-handling solution covering all unhandled exceptions and pretty stack trace presentation (e.g., with https://pypi.org/project/pretty-traceback/).

When reloading happens, it must inform the user about it:

INFO Dynamic reloading is enabled and changes in `module.name.Cls` detected; reloading.
INFO The module `module.name.Cls` reloading complete: OK
OR
ERROR The module `module.name.Cls` reloading failed.
ERROR ------------------------ BEGIN ------------------------------------
ERROR stacktrace ... (pretty stacktrace)
ERROR ------------------------- END -------------------------------------

Keep in mind there could be multiple classes declared in a single module, so the reloading can be suboptimal from the module perspective (multiple reloads).

Please investigate if GST output messages also can be colored to conform the whole situation.