URI constants & core dump. Problem with Bundler, Ruby, RubyGems, or ruby-install?

[esotericpig]

Describe the problem as clearly as you can

So I noticed that Bundler is screwed up with URI reporting tons of re-initialized constant issues:

...
/home/user/.rubies/ruby-3.0.2/lib/ruby/3.0.0/uri/mailto.rb:24: warning: already initialized constant URI::MailTo::COMPONENT
/home/user/.gem/ruby/3.0.2/gems/uri-0.11.0/lib/uri/mailto.rb:24: warning: previous definition of COMPONENT was here
/home/user/.rubies/ruby-3.0.2/lib/ruby/3.0.0/uri/mailto.rb:52: warning: already initialized constant URI::MailTo::HEADER_REGEXP
/home/user/.gem/ruby/3.0.2/gems/uri-0.11.0/lib/uri/mailto.rb:52: warning: previous definition of HEADER_REGEXP was here
/home/user/.rubies/ruby-3.0.2/lib/ruby/3.0.0/uri/mailto.rb:55: warning: already initialized constant URI::MailTo::EMAIL_REGEXP
/home/user/.gem/ruby/3.0.2/gems/uri-0.11.0/lib/uri/mailto.rb:55: warning: previous definition of EMAIL_REGEXP was here

I also get a core dump sometimes: ruby_coredump.txt.

Did you try upgrading rubygems & bundler?

First, I tried just re-installing all of the infected Gems (URI, etc.), as well as Bundler.

Finally, I just trashed everything and re-installed Ruby completely.

Post steps to reproduce the problem

First, here's the version of everything I'm using:

• ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
• Bundler version 2.2.29
• gem -v => 3.2.22
• ruby-install: 0.8.3
• chruby: 0.3.9

So I realize that ruby-install/chruby might be the issue, I deleted everything and re-installed these tools. Then I deleted all ~/.rubies and ~/.gem and did a fresh Ruby install of the latest version (v3.0.2):

• ruby-install --latest ruby
• gem update

Now, if you just use IRB by itself, there is no issue:

$ irb
irb(main):001:0> require 'uri'

But now go to any project, even a simple project, and try that. I made a new project using bundle gem testme:

$ bundle gem testme
$ cd testme

# Fix .gemspec file (remove TODOs).

$ bundle install
$ bundle update
$ bundle exec irb
irb(main):001:0> require 'uri'
/home/user/.rubies/ruby-3.0.2/lib/ruby/3.0.0/uri/version.rb:3: warning: already initialized constant URI::VERSION_CODE
/home/user/.gem/ruby/3.0.2/gems/uri-0.11.0/lib/uri/version.rb:3: warning: previous definition of VERSION_CODE was here
/home/user/.rubies/ruby-3.0.2/lib/ruby/3.0.0/uri/version.rb:4: warning: already initialized constant URI::VERSION
/home/user/.gem/ruby/3.0.2/gems/uri-0.11.0/lib/uri/version.rb:4: warning: previous definition of VERSION was here
...

Now you can see the warnings.

To get the core dump is a bit more difficult, it requires using URI and parsing it, and maybe also using nokogiri?

If you want to get the core dump, you can download my project (quite huge) to see it:

$ git clone https://github.com/esotericpig/nhkore.git
$ cd nhkore
$ bundle install
$ bundle exec rake update_core
# Just type "n" for each option, and then has core dump.

Now, if I remove require 'bundler/setup' from my main nhkore.rb and run rake update_core then I never get a core dump. Note here that if I remove require 'bundler/setup' and then run bundle exec rake update_core then I still get a core dump.

So I only get issues if I use bundle/bundler. I never get issues when just using Ruby.

Because the core dump points to .so files, it seems easy to say a bad Ruby or a bad system, but it works completely fine without Bundler. It just doesn't work with Bundler. And just for good measure, I completely updated my system and a did a full system restart.

So I'm not sure if this is where Bundler just doesn't work with the newest version of Ruby or what's going on. I apologize also because wasn't sure where to post this issue. Thanks.

If not included with the output of your command, run bundle env and paste the output below

Bundler       2.2.29
  Platforms   ruby, x86_64-linux
Ruby          3.0.2p107 (2021-07-07 revision 0db68f023372b634603c74fca94588b457be084c) [x86_64-linux]
  Full Path   /home/user/.rubies/ruby-3.0.2/bin/ruby
  Config Dir  /home/user/.rubies/ruby-3.0.2/etc
RubyGems      3.2.22
  Gem Home    /home/user/.gem/ruby/3.0.2
  Gem Path    /home/user/.gem/ruby/3.0.2:/home/user/.rubies/ruby-3.0.2/lib/ruby/gems/3.0.0
  User Home   /home/user
  User Path   /home/user/.gem/ruby/3.0.0
  Bin Dir     /home/user/.gem/ruby/3.0.2/bin
Tools         
  Git         2.31.1
  RVM         not installed
  rbenv       not installed
  chruby      0.3.9

## Bundler Build Metadata

Built At          2021-10-08
Git SHA           4440b035ef
Released Version  true

### Gemfile

# encoding: UTF-8
# frozen_string_literal: true

source 'https://rubygems.org'

gemspec

[deivid-rodriguez]

Thanks!

I managed to reproduce the warnings. In particular, the following usage of uri inside rubygems is triggered from bundler in presence of the gemspec DSL:

https://github.com/rubygems/rubygems/blob/27c308f0edd714bce039624079d26c9fbaa6c04f/lib/rubygems/specification_policy.rb#L408

We should vendor uri inside rubygems and use the vendored implementation instead of possibly activating a version of a gem that might not match the one bundler activates later.

Regarding the segfaults, I couldn't reproduce them :(

[deivid-rodriguez]

You can workaround the warnings for the moment by adding the same version of uri that rubygems is requiring (0.11.0) to your Gemfile/Gemfile.lock.

[esotericpig]

You can workaround the warnings for the moment by adding the same version of uri that rubygems is requiring (0.11.0) to your Gemfile/Gemfile.lock.

Awesome, I had to do this for uri and pathname and no problem now! Doing it for pathname fixed it so no segfaults/coredump.

So I added this to my .gemspec file for now:

  spec.add_development_dependency 'uri','0.11.0'
  spec.add_development_dependency 'pathname','0.2.0'

Note that pathname can't be another version for whatever reason.

Thanks!

[esotericpig]

Note that for pathname it seems to be coming from require "pathname" in shared_helpers.rb.

So I guess for speed, C Ruby uses /home/user/.rubies/ruby-3.0.2/lib/ruby/3.0.0/x86_64-linux/pathname.so for pathname and because the versions are off like you said, then the functions don't match up and causes segfaults.

This can be recreated in the same way:

$ bundle exec irb
irb(main):001:0> require 'pathname'
# warnings...
irb(main):002:0> exit
# segfaults/coredump...

You don't get the segfaults until you exit.