Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Significant performance concerns were raised when attempting to use the
partitionGenerator
on a CONUS hydrofabric. After a cursory look at the code, some significant optimizations are possible by simply reducing the amount of copying that happens, especially when functions are called inside of large loops. This PR addresses some low hanging fruit for optimizing thepartitionGenerator
by simply using references instead of copies/values.Preliminary analysis shows these changes decrease runtime by nearly 50% on VPU 6, and now most all the run time in that run is in reading the initial catchment data. (20 seconds of the total 23.6 seconds).
@mattw-nws this should help a bit with your CONUS partitioning.
EDIT:
Was able to run the conus partitioning on my macbook in ~15 minutes (after doing a little more memory efficiency in the geojson reader keep from crashing when I ran out of swap and RAM...I'll push those edits in another PR).
I will also note that to get to that speed, I was using a RELEASE build, not a DEBUG build. The boost ptree reading is highly sensitive to that!
cmake -DCMAKE_BUILD_TYPE=Release
makes reading the geojson significantly faster!Additions
Removals
Changes
const
referenceO(n^2)
toO(n log n)
implementation.Testing
Screenshots
Notes
Network
could be passed by reference, but notconst
reference due to many nested functions that were not const qualified.Todos
Checklist
Testing checklist (automated report can be put here)
Target Environment support