-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Store failure cause and provide to _get_preference #84
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is smart and I like it.
Saving the failure causes directly on the resolver feels wrong to me, since the cases are only specific to a backtracking session and not the entire resolution. I feel it makes more sense to put them on the State objects instead (or another stack that only manages backtracking, but I want to avoid that if possible). |
I'd not thought about this approach. Taking a quick look at the code pushing new state seems to happen Pip doesn't have new failure causes, so I'm not quite sure how this would be achieved. I'll try and study the code more carefully and think on it. |
Expanding on the thought, |
Some more context on why I feel the attribute should be on A resolver is fundamentally traversing deeper and deeper into a graph to find a working solution. This can be most naively implemented by recursion, but you know, Python does not do that particularly well, so the resolver is, at the highest level, unrolling the recursion into a loop, and With recuision, the easiest way to implement backtracking is with exceptions and re-raising, which bubbles up through the call chain until we find a working call site. Mapping this to iteration, we’d keep popping states, modifying the previous state a bit (the |
This was very helpful thanks! Any solution that involved pinning the failure causes to the state would end up having the same effect as option 1 as I mentioned here: #84 (comment) . So first I am going to test if this approach still works in the test cases I have. It seems also that the simplest way to do this is to have |
Yeah this would be the easiest approach, but maybe we can redesign |
Okay, I took a look at this and I need a better understanding of how Criterion.incompatibilities work, I'll do some further research. In the mean time I was able to move failure_causes off being a global variable and on to the state, this slightly changes the behavior of when the failure_causes preferences are applied so I am running over the requirements files I have now to make sure it still solves excessive backtracking. I'll update the PR once done. After that, I think, migrating to using Criterion.incompatibilities should just be a refactoring and not affect the behavior of backtracking. |
I've not updated the causes to be stored on the state instead of the Resolver instance, and I am calling them "backtrack_causes" instead of "failure_causes". This is because they are not actually the failures of the current state, but instead failures of a state which needed to be backtracked out of. I personally feel it makes everything read better, especially in the Provider, But let me know if anyone was particularly tied to "failure_causes" |
FYI, I consider this PR ready, if maintainers agree I'll start fixing tests. I took a look at merging the logic with incompatibilities but I think there's a timing issue. The failure causes are generated from a state which is deleted by backtracking. Incompatibilities are generated from the previous state. I have to save the failure causes before that state gets deleted then pin them to the latest state that exists after backtracking. But let me know if you're still unhappy with this approach. |
Please fix the tests then. You can do this in a separate commit, so if we want to switch direction the tests can easily be un-changed with |
I pushed a commit that fixes the tests, turned out to be fairly trivial fix. But the tests fail on Python 2.7 because |
Yes 2.7 is still supported. You can use |
Yup, already submitted a commit with that and and all tests pass. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
I do think we should add tests for the new behaviour though, to ensure that we’re passing valid things with a valid structure in the new argument.
I took a look at the existing tests yesterday and I couldn't find anything that tests any of the behavior here. This PR doesn't actually change the behavior of The main breaking change here is passing a new argument to It also adds an extra property to |
Yup yup, this is what I'm saying we should add a test for. None the less, I also thought we had more extensive tests for this library than we actually do, so I'm happy to kick the can down the road for this. :) |
Hurrah. @uranusjr do you wanna cut the release, or do you think it might make sense for me to learn how to do that on this project? :) |
I had a think about this and thought maybe provider should provide a Base or Abstract class so this can be checked / tested. Then I looked at the code and realized there is an How do I proceed fixing this PR post-merge? Make a new PR? Also would it be possible to give |
Created new Pull Request #86 due to missing |
@pradyunsg Feel free try your hands, I can always take over if things go south. There's a tox task to automate release, but to be honest I never remember its exact arguments and always ended up reading the source and manually invoked all the steps instead, it's actually easier than worrying about whether automation is doing the right thing. |
What are people's thoughts about making it for the pip 21.3 release? Obviously I would of preferred that, but if it can't it can't. |
Cool, that worked out. 0.8.0 is up. Here's the release process, which I'll note down while it's still fresh in my head:
|
That's a nice clean release process 🙂 Well done to whoever implemented the automation around that! |
I added the bullets to DEVELOPMENT.rst, thanks for verifying the process works :p |
As per pypa/pip#10479
If pip prefers failure causes then resolvelib must provide them to get_preference
Required for pypa/pip#10481