add reduceResultsList.fun arg to reduceResultsBatchmark #29
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi! in my use case, I want to do model interpretation, so I am using store_models=TRUE.
But that makes my cluster system kill my R process (too much RAM used) when I do reduceResultsBatchmark, which gets killed when it calls reduceResultsList, #18 (comment) because the total of all
learner_state
is too large.So in this PR I add a new argument, reduceResultsList.fun, which gets passed on, so then I can process the
learner_state
. For example the code below works for me using this PR. (uses reasonable amount of memory / R is not killed)(but without the proposed reduceResultsList.fun, the above code results in my cluster system killing R)
Thanks for your consideration.