Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
use dill to store the spawn function as bytes
when creating a worker, we pass in a spawn function. Dill is an improved version of pickle so dill can serialize many more functions than pickle can. Thus we use dill to serialize the spawn function to bytes, which can then be serialized/deserialized by pickle when spawning subprocesses. Then when the worker needs to run the spawn function, we can use dill to deserialize the bytes into the desired function.
- Loading branch information