Performance: added option to move matplotlib figure generation to separate process #12
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thisistheplace
changed the title
andrewlyden
approved these changes
Jul 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi @andrewlyden,
The matplotlib
savefigmethod is causing a bottleneck in the overall runtime. It takes ~ 1/3 to 1/2 of the overall time when running the fixed order solver. There doesn't seem to be anything that can be done about this within the matplotlib library.I originally planned on moving the figure writing to a separate thread but matplotlib isn't threadsafe.
Instead I've moved the figure writing to a separate process (compute core) so that it doesn't block the solver from moving onto the next parametric simulation. This can make debugging more tricky but single core processing can be forced by specifying the new
--singlecoreoption.I've also changed the progress bars to use the tqdm library which is simpler to implement. Log messages mostly get written to the log file with just progress (and warnings/errors!) written to the screen which hopefully makes monitoring the overall run progress a bit clearer.
Thanks,
Ben