Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse common data #154

Closed
mschubert opened this issue Jul 2, 2019 · 1 comment
Closed

Reuse common data #154

mschubert opened this issue Jul 2, 2019 · 1 comment
Milestone

Comments

@mschubert
Copy link
Owner

If I set up workers with multiple subsequent computations, it would be nice to only re-send constant/exported data if it actually changes.

Something like:

Running 21,010 calculations (5 objs/348.8 Mb common - 56% reuse; 11 calls/chunk) ...

Could potentially store this for more computations as well, but that will get tricky with memory management (if we're reaching the limit, we'd want to delete unused objects rather than failing the computation - but how to track this?)

@wlandau
Copy link
Contributor

wlandau commented Jul 2, 2019

Awesome! Seems like data reusage could power #147 and ropensci/drake#672 quite nicely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants