-
Notifications
You must be signed in to change notification settings - Fork 63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: hybrid incremental solver and performance improvements #349
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tdejager
reviewed
Sep 25, 2023
Co-authored-by: Tim de Jager <tdejager89@gmail.com>
tdejager
approved these changes
Sep 25, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a big PR that does two things:
Incremental solver
The solver is now incremental by default. This means that the solver will only request information from the
DependencyProvider
when it is relatively sure that that information is needed to come up with a solution. Previously the solver would greedily request all information about the problem space up front. This made the solver very unsuitable for cases where requesting information about packages is expensive.Optionally a
DependencyProvider
has the ability to hint to the solver that it already has dependency information available for some candidates. The solver will use this information to increase its knowledge of the problem space without requiring network operations. The latter is used for the conda solver.In general, it's better to have all the information available upfront because if the solver has "all" the information available it is able to reach a solution quicker because it doesn't have to guess about information it doesn't know.
Performance improvements
This PR also implements a performance improvement by being more thoughtful about making decisions about the next steps to take in the algorithm. Instead of picking the first available decision it can make it checks all possible choices to make and picks the decision that involves the least amount of versions to pick.
This is similar to what pubgrub does:
https://github.com/dart-lang/pub/blob/master/doc/solver.md#decision-making
In the future, we might take a look at different heuristics but this implementation is good enough for now.
Some results:
Note that for "simple" cases this PR is slower but just slightly and for "complex" cases this PR is much faster.