Skip to content
This repository has been archived by the owner on Apr 14, 2021. It is now read-only.

Infinite "Resolving dependencies" using 1.16 with fastlane and atlassian-stash #6181

Closed
jshier opened this issue Nov 21, 2017 · 12 comments · Fixed by #6186
Closed

Infinite "Resolving dependencies" using 1.16 with fastlane and atlassian-stash #6181

jshier opened this issue Nov 21, 2017 · 12 comments · Fixed by #6186

Comments

@jshier
Copy link

jshier commented Nov 21, 2017

Bundler 1.16 spins forever in the "Resolving dependencies" step, outputting periods, with Gemfile containing only fastlane and atlassian-stash gems. 100% reproducible.

Environment

Bundler             1.16.0
  Platforms         ruby, x86_64-darwin-17
Ruby                2.4.2p198 (2017-09-14 revision 59899) [x86_64-darwin17]
  Full Path         /Users/jshier/.rvm/rubies/ruby-2.4.2/bin/ruby
  Config Dir        /Users/jshier/.rvm/rubies/ruby-2.4.2/etc
RubyGems            2.6.14
  Gem Home          /Users/jshier/.rvm/gems/ruby-2.4.2@resolutiontest
  Gem Path          /Users/jshier/.rvm/gems/ruby-2.4.2@resolutiontest:/Users/jshier/.rvm/gems/ruby-2.4.2@global
  User Path         /Users/jshier/.gem/ruby/2.4.0
  Bin Dir           /Users/jshier/.rvm/gems/ruby-2.4.2@resolutiontest/bin
Tools               
  Git               2.13.6 (Apple Git-96)
  RVM               1.29.3 (latest)
  rbenv             not installed
  chruby            not installed
  rubygems-bundler  (1.4.4)

Bundler Build Metadata

Built At          2017-10-31
Git SHA           10f20fa33
Released Version  true

Gemfile

Gemfile

source "https://rubygems.org"

gem "fastlane"
gem "atlassian-stash"

Gemfile.lock

<No /Users/jshier/Desktop/ResolutionTest/Gemfile.lock found>
@arbonap
Copy link
Contributor

arbonap commented Nov 22, 2017

Interesting...I also got the same thing to happen as @jshier when I didn't tie any of the gems to a version with Bundler 1.16. I let Bundler 1.16 try to resolve dependencies for at least a good 10 minutes.

@jshier not sure if this is useful to you, but when I tied the fastlane gem to the latest version, ie like gem "fastlane", "= 2.66.2" my dependencies resolved quickly with Bundler 1.16.

@colby-swandale
Copy link
Member

@arbonap is correct, Bundler is trying to resolve a very wide set of possible solutions. Put a version constraint on both gems in the Gemfile and Bundler should resolve a lot quicker.

@jshier
Copy link
Author

jshier commented Nov 22, 2017

Sure, but that’s a severe regression over 1.15.4. Sorry, I forgot to mention that. I’ll play around with the versions without actually locking.

@segiddins
Copy link
Member

@greysteil resolver regression

@greysteil
Copy link
Contributor

Uh oh - I'll take a look this evening

@greysteil
Copy link
Contributor

greysteil commented Nov 23, 2017

@jshier - thanks masses for reporting this. Can you let me know how long it takes Bundler to resolve this dependency set on v1.15.4? Running locally (v1.15.4) and I still don't have a resolution after a few minutes. Scratch that, my dev environment was screwed and claiming v1.16.0 was v1.15.4...

@greysteil
Copy link
Contributor

This would be fixed by the tweak to the dependency sort order in #6186. I don't love that, but I don't have any major optimisations to the resolver up my sleeve right now, which would be the other way to address it.

@jshier
Copy link
Author

jshier commented Nov 23, 2017

@greysteil: It takes less than a second usually. So fast I’ve never bothered to time it. I can do more tests after Thanksgiving.

@greysteil
Copy link
Contributor

greysteil commented Nov 23, 2017

@jshier - thanks. No need for more tests - the updated resolver eats this for breakfast after #6186. Enjoy Thanksgiving!

@lparry
Copy link

lparry commented Nov 27, 2017

I've also noticed our weekly bundle update has gone from taking seconds to taking 10-15 minutes, we run a fairly large Gemfile that is mostly unconstrained and review the changes each week.

Just in case #6186 doesn't solve our (similiar/probably related) issue, is there some flag I can pass to bundle update to see which gems it's spending so long trying to resolve are? Not super keen to add constraints to work around bundler suddenly having major performance regressions, but if we have to I'd like to narrow it down as much as possible

@segiddins
Copy link
Member

@lparry setting the env var DEBUG_RESOLVER will cause a whole load of debug output to be printed

@indirect
Copy link
Member

@lparry thanks for the additional data point. so that you're aware, putting no constraints in your Gemfile explicitly says to Bundler that you are willing to accept any version of that gem in existence, all the way back down to 0.0.0. As you can probably imagine, this forces Bundler to try every version of every gem every time you run update, even if you aren't actually willing to accept many of those versions. Setting minimum versions, or better yet using pessimizer to set pessimistic versions constraints and using something like Dependabot to handle upgrades will give you a better experience all around, even if we fix this exact performance issue.

bundlerbot added a commit that referenced this issue Nov 29, 2017
…t, r=indirect

Resolver: treat dependencies with prerelease versions as slightly constrained

Fixes #6181.

In Bundler 1.15.4, the way we checked for pre-releases meant that dependencies with a lot of them sorted as more constrained than those with few/none. When we moved to our updated resolution strategy in 1.16.0 we accidentally lost that behaviour.

I'm not wild about this PR, but it fixes the resolver regression. It's unsatisfying because it would be nice to believe the resolver will always resolve performantly, regardless of the sort order thrown at it, but clearly that isn't the case.

Conceptually, this can be justified as "when two resolutions are possible, we prefer the one that does not require one dependency to be at a pre-release version".

If others are happy with this I can write a test for it.
hsbt pushed a commit to hsbt/bundler that referenced this issue Dec 11, 2017
…onstraint, r=indirect

Resolver: treat dependencies with prerelease versions as slightly constrained

Fixes rubygems#6181.

In Bundler 1.15.4, the way we checked for pre-releases meant that dependencies with a lot of them sorted as more constrained than those with few/none. When we moved to our updated resolution strategy in 1.16.0 we accidentally lost that behaviour.

I'm not wild about this PR, but it fixes the resolver regression. It's unsatisfying because it would be nice to believe the resolver will always resolve performantly, regardless of the sort order thrown at it, but clearly that isn't the case.

Conceptually, this can be justified as "when two resolutions are possible, we prefer the one that does not require one dependency to be at a pre-release version".

If others are happy with this I can write a test for it.
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Mar 14, 2018
## 1.16.1 (2017-12-12)

Bugfixes:

  - avoid hanging on complex resolver errors ([#6114](rubygems/bundler#6114), @halfbyte)
  - avoid an error when running `bundle update --group` ([#6156](rubygems/bundler#6156), @mattbrictson)
  - ensure the resolver prefers non-pre-release gems when possible ([#6181](rubygems/bundler#6181), @greysteil)
  - include bundler's gemspec in the built gem ([#6165](rubygems/bundler#6165), @dr-itz)
  - ensure locally installed specs are not overriden by those in remote sources during dependency resolution ([#6072](rubygems/bundler#6072), @indirect)
  - ensure custom gemfiles are respected in generated binstubs (@pftg)
  - fail gracefully when loading a bundler-generated binstub when `bin/bundle` was not generated by bundler ([#6149](rubygems/bundler#6149), @hsbt)
  - allow `bundle init` to be run even when a parent directory contains a gemfile ([#6205](rubygems/bundler#6205), @colby-swandale)

## 1.16.0 (2017-10-31)

Bugfixes:

  - avoid new RubyGems warning about unsafe YAML loading (to keep output consistent) (@segiddins)
  - load digest subclasses in a thread-safe manner (@segiddins, @colby-swandale)
  - avoid unusued variable warnings under ruby 2.5 (@amatsuda)
  - fix printing the same message twice in verbose mode ([#6028](rubygems/bundler#6028), @akhramov)
  - allow `SignalException`s to bubble up to the interpreter during `bundle exec` ([#6090](rubygems/bundler#6090), @dekellum)
  - avoid activating stdlib digest under Ruby 2.5 (@segiddins)
  - prioritise explicitly requested gems in dependency resolution sort order (@segiddins)
  - reduce memory usage during dependency resolution ([#6114](rubygems/bundler#6114), @greysteil)
  - ensure that the default bundler gem is not accidentally activated on ruby 2.5 when using local git overrides (@segiddins)

## 1.16.0.pre.3 (2017-10-04)

Features:

  - the output from `bundle env` includes more information, particularly both the compiled & loaded versions of OpenSSL (@indirect)

Bugfixes:

  - fix a bug where installing on FreeBSD would accidentally raise an error (#6013, @olleolleolle)
  - fix a regression in 1.16 where pre-release gems could accidentally be resolved even when the gemfile contained no pre-release requirements (@greysteil)
  - bundler will avoid making unnecessary network requests to fetch dependency data, fixing a regression introduced in 1.16 (@segiddins)
  - the outdated bundler version message is disabled by default until the message has been fine-tuned (#6004, @segiddins)

## 1.16.0.pre.2 (2017-09-06)

Bugfixes:

  - handle when a connection is missing a socket when warning about OpenSSL version (@greysteil)
  - the description for the `rake release` task now reflects `$RUBYGEMS_HOST` (@wadetandy)
  - fix a bug where `bundle update` would regress transitive dependencies (@greysteil)

## 1.16.0.pre.1 (2017-09-04)

Features:

  - allow using non-branch symbolic refs in a git source (#4845, @segiddins)
  - allow absolute paths in the `cache path` setting (#5627, @mal)
  - gems created via `bundle gem` with rspec have `--require spec_helper` in their `.rspec` file (@koic)
  - `bundle env` includes `Gem.ruby` and the `bundle` binstub shebang when they don't match (#5616, @segiddins)
  - allow passing gem names to `bundle pristine` (@segiddins)
  - `bundle version` and `bundle env` include the commit and build date for the bundler gem (#5049, @segiddins)
  - add the `--shebang` option to `bundle binstubs` (#4070, @segiddins, @zorbash)
  - gemfiles are `eval`ed one fewer time when running `bundle install` (#4952, #3096, #4417, @segiddins)
  - the `fileutils` gem is now vendored so different versions of the gem can be activated (@segiddins)
  - speed up no-op installations (#5842, @segiddins)
  - default to keeping the lockfile in the default gem template (@deivid-rodriguez)
  - add a special bundler binstub that ensures the correct version of bundler is activated (#5876, @segiddins)
  - speed up dependency resolution and ensure that all resolvable gemfiles can be installed (@segiddins, @greysteil)
  - add a `bundle list` command that prints the gems in use (#4754, @colby-swandale)
  - allow adding credentials to a gem source during deployment when `allow_deployment_source_credential_changes` is set (@adrian-gomez)
  - making an outdated (and insecure) TLS connection to rubygems.org will print a warning (@segiddins)

Bugfixes:

  - allow configuring a mirror fallback timeout without a trailing slash (#4830, @segiddins)
  - fix handling of mirrors for file: urls that contain upper-case characters (@segiddins)
  - list the correct gem host for `rake release` when `allowed_push_host` has been set (@mdeering)
  - ensure `Bundler.original_env` preserves all env keys that bundler sets (#5700, @segiddins)
  - ensure `bundle pristine` removes files added to a git gem (@segiddins)
  - load plugin files from path gems before gem installation (#5429, @segiddins)
  - ensure gems containing manpages are properly set up (#5730, @segiddins)
  - avoid fetching remote specs when all effected gems are in groups that are not being installed (@segiddins)
  - allow `BUNDLE_GEMFILE` to be a relative path (#5712, @gxespino)
  - show a more helpful error message when a gem fails to install due to a corrupted lockfile (#5846, @segiddins)
  - add a process lock to allow multiple concurrent `bundle install`s (#5851, @stefansedich)
  - ensure that specifications always return an array for `#extensions` (@greysteil)
  - print a helpful error message when using a gem in the Gemfile with an empty name (@colby-swandale)
  - ensure that all gemfiles are included in `bundle env` (@segiddins)
  - use ssl client cert and ca cert settings from gem configuration as fallbacks (@stan3)
  - avoid global namespace pollution when loading gems (#5958, @shyouhei)
  - avoid running a complete re-resolve on `bundle update --bundler` (@segiddins)
  - allow `bundle binstubs --standalone` to work without `path` being set (@colby-swandale)
  - fix support for bundle paths that include jars or wars on jruby (#5975, @torcido)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants