bundler error #5797
Comments
|
What did you do? Problem is intermittent. |
|
Here is output of 'bundle env' $ bundle env EnvironmentBundler settingsGemfileGemfilesource 'https://artifactory.XXXXX.com/artifactory/api/gems/rubygems-remote/'
source 'https://rubygems.org'
gem 'aws-sdk', '~>2.6.33'
gem 'json'
gem 'slack-ruby-client', '~>0.7.7'
gem 'sparkle_formation', '0.3.0'
gem 'random-word', '~>1.3.0'
gem 'json-schema', '~>2.6.2'
gem 'octokit', '~>4.6.0'
gem 'jenkins_api_client', '~>1.4.5'
gem 'berkshelf', '5.6.3'
gem 'splunk-sdk-ruby'
gem 'mixlib-cli'Gemfile.lock |
|
I'm unable to reproduce -- when I attempt to install the given gemfile, I get the following: |
|
that's my organization's artifactory repo which I've blocked out for privacy. |
|
Can you please share a gemfile we can use to reproduce the error? Otherwise it's unlikely we'll be able to figure out what's going wrong |
|
Happens for us here too at Yammer, but we also have an internal geminabox server that you won't be able to access. Anything we can do to help you troubleshoot it otherwise? We see it sporadically. We'll run bundler once and it'll fail and then run it again the exact same way and it'll work. |
|
Looks like this same gem comes up in a few different issues when looking back at past issues: https://github.com/bundler/bundler/issues?utf8=✓&q=ruby-ajp @indirect mentioned this in an old comment to something similar: "This seems to be an error that only surfaces when some (but not all) of the requests to rubygems.org fail. They're currently having load balancer issues that intermittently return 500 errors." Could that be happening here? |
|
@bmorton can you please share the output of |
|
Doing some more investigation, I'd love to confirm if that fallback to pulling all the gemspecs is happening again. If that does happen, then it looks like the gemspec for Using this Gemfile: source 'https://rubygems.org'
gem "ajp-rails", "0.0.0"#2615 mentions this file: http://aws-eu-cache01.rubygems.org/quick/Marshal.4.8/ajp-rails-0.0.0.gemspec.rz |
|
@colby-swandale, I can but our security team will probably require that I redact some things in it and we have an internal gemserver that you won't be able to access. Is it still helpful without that stuff? |
|
Also, I've added |
|
I've gotten it to trigger now with verbose. I am going to talk to our security team to see if I can send you the whole verbose log, but here are the relevant bits for us at least. We end up getting rate limited (probably because of how our outbound egress is routed and the number of webs that we deploy to at a time?) Once we get rate limited, bundler starts pulling back every gemspec, but these requests don't seem to be rate limited. Once it gets to a gemspec that's bad, the whole thing crashes. |
|
This looks like it might be a bug with rubygems.org, as it's serving an invalid gemspec for https://rubygems.org/gems/ajp-rails/versions/0.0.0. The only thing Bundler can do here is to error out more gracefully, but I recommend opening up an issue on rg.org for the invalid gem spec. |
|
See #5819 |
|
Cool, thanks. I'll open an issue with them. As for Bundler, is it expected behavior that when Bundler gets rate limited with a 429, it instead rapidly makes ~33k requests for every version of every gem? |
|
@bmorton it's a known edge case bug that we have had trouble easily or consistently reproducing in tests, but that we would like to fix. Hopefully the information from this ticket will help! |
|
Sweet. I can actually reproduce this locally with a proxy now that injects 429s at the right place. Is that helpful to post how I did that here? |
|
The trick is reproducing it inside a test in our test suite, but if you could gist a writeup of your repro steps, that would probably be helpful. Thanks! |
|
Here are the steps. Its a bit tricky to repro, but this should lay it out pretty well. Basically its a mix of getting the 429 at the right time and using an additional geminabox source. https://github.com/bmorton/rubygems-rate-limiter Let me know if you need help getting this working. I ran it through with a coworker (thanks @matthewshafer!) real quick and it seemed to work (as long as you have golang setup already). I think we'll have a stab at trying to get a test case going for this later, but are gonna leave it alone for a little bit. |
[RemoteSpecification] Fail gracefully when deps is an array of array of string Instead of containing Gem::Dependency objects ### What was the end-user problem that led to this PR? The problem was some gems have invalid gemspecs served by RubyGems.org. See #5797. ### Was was your diagnosis of the problem? My diagnosis was (very old) some gemspecs can have `s.dependencies = [["name", "req"]]` instead of `s.dependencies = [Gem::Dependency.new("name", "req")]`. ### What is your fix for the problem, implemented in this PR? My fix coerces the invalid dependencies to an array of dependency objects so we can fail more gracefully during installation, without spitting out the error template. Closes #5797. ### Why did you choose this fix out of the possible options? I chose this fix because it allows resolution to finish, and falls back upon existing error messages.
[RemoteSpecification] Fail gracefully when deps is an array of array of string Instead of containing Gem::Dependency objects ### What was the end-user problem that led to this PR? The problem was some gems have invalid gemspecs served by RubyGems.org. See #5797. ### Was was your diagnosis of the problem? My diagnosis was (very old) some gemspecs can have `s.dependencies = [["name", "req"]]` instead of `s.dependencies = [Gem::Dependency.new("name", "req")]`. ### What is your fix for the problem, implemented in this PR? My fix coerces the invalid dependencies to an array of dependency objects so we can fail more gracefully during installation, without spitting out the error template. Closes #5797. ### Why did you choose this fix out of the possible options? I chose this fix because it allows resolution to finish, and falls back upon existing error messages. (cherry picked from commit 5c62240)
Running bundler
--- ERROR REPORT TEMPLATE -------------------------------------------------------
Error Report
Questions
Please fill out answers to these questions, it'll help us figure out
why things are going wrong.
What did you do?
I ran the command
/var/lib/jenkins/.rvm/gems/ruby-2.3.3/bin/bundle install --quietWhat did you expect to happen?
I expected Bundler to...
What happened instead?
Instead, what happened was...
Have you tried any solutions posted on similar issues in our issue tracker, stack overflow, or google?
I tried...
Have you read our issues document, https://github.com/bundler/bundler/blob/master/doc/contributing/ISSUES.md?
...
Backtrace
Environment
Bundler settings
--- TEMPLATE END ----------------------------------------------------------------
The text was updated successfully, but these errors were encountered: