Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge from upstream #32

Merged
merged 105 commits into from
Nov 13, 2022
Merged

Merge from upstream #32

merged 105 commits into from
Nov 13, 2022

Conversation

chrisseaton
Copy link
Collaborator

Fixes a merge conflict in gc.h.

nobu and others added 30 commits November 6, 2022 10:16
As the target-list of `rustc` is different from `config.guess` and
`config.sub`, `$target` cannot be used directly.
Even if `rustc` is available, it should not be an error unless
`--enable-yjit` is explicitly given.
The bitrig OS is no longer maintained with the last release being 7
years ago.

rubygems/rubygems@85ed90ddd0
The IRIX OS is no longer maintained with the last release being 16 years
ago.

rubygems/rubygems@5381c6a871
There are a few downsides of the current approach:

1. Because gem specs are lazily retrieved, this computation happens in
   every irb completion test case, which is not necessary. (In tests we
   don't cache the result of `retrieve_files_to_require_from_load_path`)
2. Gem::Specification.latest_specs is sensible to the content of
   LOAD_PATH. And when combined with 1, tests fail "randomly" if they
   try to mutate LOAD_PATH, even though the test subject it's something
   else.

So by pre-computing and storing the gem paths in a constant, it guarantees
that the computation only happens once and it doesn't get affected by test
cases.

One argument could be made against the change is that, it'll store
unnecessary data for users that disable autocompletion. But the
counter-arguments are:

1. Since autocompletion is enabled by default, this should not be the
   case for most users.
2. For users with autocompletion enabled, IRB already caches the
   result of `retrieve_files_to_require_from_load_path` in memory, which
   should have a similar size of GEM_SPECS. And we currently haven't
   received any report about problems caused by such memory consumption.

ruby/irb@c671d39020
(ruby/irb#430)

* Support non-string input in show_source

* Test show_source as a method
Before:

```
Reducing stack by rule 639 (line 5062):
   $1 = token "integer literal" (1.0-1.1: 1)
-> $$ = nterm simple_numeric (1.0-1.1: )
```

After:

```
Reducing stack by rule 641 (line 5078):
   $1 = token "integer literal" (1.0-1.1: 1)
-> $$ = nterm simple_numeric (1.0-1.1: NODE_LIT)
```

`"<*>"` is supported by Bison 2.3b (2008-05-27) or later.
https://git.savannah.gnu.org/cgit/bison.git/commit/?id=12e3584054c16ab255672c07af0ffc7bb220e8bc

Therefore developers need to install Bison 2.3b+ to build ruby from
source codes if their Bison is older.

Minimum version requirement for Bison is changed to 3.0.

See: https://bugs.ruby-lang.org/issues/19068 [Feature #19068]
`throw TAG_BREAK` instruction makes a jump only if the continuation of
catch of TAG_BREAK exactly matches the instruction immediately following
the "send" instruction that is currently being executed. Otherwise, it
seems to determine break from proc-closure.

Branch coverage may insert some recording instructions after "send"
instruction, which broke the conditions for TAG_BREAK to work properly.

This change forces to set the continuation of catch of TAG_BREAK
immediately after "send" (or "invokesuper") instruction.

[Bug #18991]
GCC [Bug 99578] seems triggered by calling `rb_reg_last_match` before
`match_check(match)`, probably by `NIL_P(match)` in `rb_reg_nth_match`.

[Bug 99578]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99578
[Bug #19105]

If no fiber scheduler is registered and the fiber that
owns the lock and the one that try to acquire it
both belong to the same thread, we're in a deadlock case.

Co-authored-by: Jean Boussier <byroot@ruby-lang.org>
…by#6684)

* YJIT: improve/fix code to automatically build YJIT when available

* Set YJIT_SUPPORT=no

* Fix rustc => $RUSTC
k0kubun and others added 26 commits November 10, 2022 17:31
* Enable --yjit-stats for release builds

In order for people in the real world to report information about how their application runs with YJIT, we want to expose stats without requiring rebuilding ruby. We can do this without overhead, with the exception of count ratio in yjit, since this relies on the interpreter also counting instructions.

This change exposes those stats, while not showing ratio in yjit if we are not in a stats build.

* Update yjit.rb

Co-authored-by: Takashi Kokubun <takashikkbn@gmail.com>
Co-authored-by: Maxime Chevalier-Boisvert <maximechevalierb@gmail.com>
I meant they should be also fixed in
ruby#6694 (comment)
Extract an `rb_shape_get_parent` method instead of continually calling
`rb_shape_get_shape_by_id(shape->parent_id)`
JRuby uses these same files for testing racc. The existing logic will not
find 'racc' in a JRuby project checkout. This change allows it to work by
just assuming 'ruby -S racc' when running tests. This will not change C
Ruby's detection when setting up tests (since earlier checks will find
racc).

ruby/racc@b74ce5e018
It does not work well in assert_separately
Ruby interpreter loads some special gems at startup: did_you_mean,
error_highlight, and syntax_suggest. These gems are loaded before
`bundler/setup` is loaded by `RUBYOPT=-rbundler/setup`.
So, the versions of the gems are not controllable by Gemfile.

This change will `require "bundler/setup"` in rubygems.rb (i.e., before
the special gems are loaded). Now `bundle exec` sets an environment
variable `BUNDLER_SETUP`, and rubygems requires the variable if defined.

See also: https://bugs.ruby-lang.org/issues/19089

rubygems/rubygems@963cb65a2d
(ruby/irb#437)

* Transform ls's --grep/-G option to keyword args

* Make --grep less flexible

* Support -g instead of --grep

* Suppress warnings from symbol aliases
The Ruby built for wasm cannot be execute without a WebAssembly runtime.

```
$ ruby-wasm32-wasi/usr/local/bin/ruby -e 'puts "a"'
bash: ruby-wasm32-wasi/usr/local/bin/ruby: cannot execute binary file: Exec format error
```

Because the Ruby's file type is different from the one built normally, that is
the `/usr/local/ruby-3.2.0-preview2/bin/ruby` below.

```
$ file ruby-wasm32-wasi/usr/local/bin/ruby
ruby-wasm32-wasi/usr/local/bin/ruby: WebAssembly (wasm) binary module version 0x1 (MVP)

$ file /usr/local/ruby-3.2.0-preview2/bin/ruby
/usr/local/ruby-3.2.0-preview2/bin/ruby: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=a37822085e285c0971159982e7642dda88cea606, for GNU/Linux 3.2.0, with debug_info, not stripped
```
The tests failed on windows

https://github.com/ruby/ruby/actions/runs/3440997073/jobs/5740085169#step:18:62
```
    1) Failure:
  TestRegexp#test_s_timeout [D:/a/ruby/ruby/src/test/ruby/test_regexp.rb:1586]:
  <0.30000000000000004> expected but was
  <0.3>.

    2) Failure:
  TestRegexp#test_timeout_shorter_than_global [D:/a/ruby/ruby/src/test/ruby/test_regexp.rb:1631]:
  <0.30000000000000004> expected but was
  <0.3>.
```
Bundler 2 switched to secure https here rubygems/rubygems@c2e81f8

Insecure protocols should be avoided to prevent MITM attacks.

rubygems/rubygems@758413364a
Previously, there is no instruction boundary patch point after
the call to a non-leaf C function we generate for
OPTIMIZED_METHOD_TYPE_CALL. This meant that if code GC is triggered
while inside the C function, we would keep running invalidated code when
we return from the C function. This had the effect of running
stale branch stubs, jumping to bad code, etc.

Use jit_prepare_routine_call() to make sure we exit from the invalidated
region as soon as possible after the C call in case of invalidation.
ruby/net-http@16d042fad6

Co-authored-by: Peter Zhu <peter@peterzhu.ca>
@chrisseaton chrisseaton requested a review from wks November 12, 2022 17:09
@wks wks merged commit d829fe4 into mmtk Nov 13, 2022
wks pushed a commit that referenced this pull request Dec 2, 2022
wks pushed a commit that referenced this pull request Aug 4, 2023
[Bug #19793]

Dummy frames are created at the top level when requiring another file.
While requiring a file, it will try to convert using encodings. Some of
these encodings will not respond to to_str. If method_missing is
redefined on Object, then it will call method_missing and attempt raise
an error. However, the iseq is invalid as it's a dummy frame so it will
write an invalid iseq to the created NoMethodError.

The following script crashes:

```
GC.stress = true

class Object
  public :method_missing
end

File.write("/tmp/empty.rb", "")
require "/tmp/empty.rb"
```

With the following backtrace:

```
frame #0: 0x00000001000fa8b8 miniruby`RVALUE_MARKED(obj=4308637824) at gc.c:1638:12
frame #1: 0x00000001000fb440 miniruby`RVALUE_BLACK_P(obj=4308637824) at gc.c:1763:12
frame #2: 0x00000001000facdc miniruby`gc_writebarrier_incremental(a=4308637824, b=4308332208, objspace=0x000000010180b000) at gc.c:8822:9
frame #3: 0x00000001000faad8 miniruby`rb_gc_writebarrier(a=4308637824, b=4308332208) at gc.c:8864:17
frame #4: 0x000000010016aff0 miniruby`rb_obj_written(a=4308637824, oldv=36, b=4308332208, filename="../iseq.c", line=1279) at gc.h:804:9
frame #5: 0x0000000100162a60 miniruby`rb_obj_write(a=4308637824, slot=0x0000000100d09888, b=4308332208, filename="../iseq.c", line=1279) at gc.h:837:5
frame #6: 0x0000000100165b0c miniruby`iseqw_new(iseq=0x0000000100d09880) at iseq.c:1279:9
frame #7: 0x0000000100165a64 miniruby`rb_iseqw_new(iseq=0x0000000100d09880) at iseq.c:1289:12
frame #8: 0x00000001000d8324 miniruby`name_err_init_attr(exc=4309777920, recv=4304780496, method=827660) at error.c:1830:35
frame #9: 0x00000001000d1b80 miniruby`name_err_init(exc=4309777920, mesg=4308332496, recv=4304780496, method=827660) at error.c:1869:12
frame #10: 0x00000001000d1bd4 miniruby`rb_nomethod_err_new(mesg=4308332496, recv=4304780496, method=827660, args=4308332448, priv=0) at error.c:1957:5
frame #11: 0x000000010039049c miniruby`rb_make_no_method_exception(exc=4304914512, format=4308332496, obj=4304780496, argc=1, argv=0x000000016fdfab00, priv=0) at vm_eval.c:959:16
frame #12: 0x00000001003b3274 miniruby`raise_method_missing(ec=0x0000000100b06f40, argc=1, argv=0x000000016fdfab00, obj=4304780496, last_call_status=MISSING_NOENTRY) at vm_eval.c:999:15
frame #13: 0x00000001003945d4 miniruby`rb_method_missing(argc=1, argv=0x000000016fdfab00, obj=4304780496) at vm_eval.c:944:5
...
frame #23: 0x000000010038f5e4 miniruby`rb_vm_call_kw(ec=0x0000000100b06f40, recv=4304780496, id=2865, argc=1, argv=0x000000016fdfab00, me=0x0000000100cbfcf0, kw_splat=0) at vm_eval.c:326:12
frame #24: 0x00000001003c18e4 miniruby`call_method_entry(ec=0x0000000100b06f40, defined_class=4304927952, obj=4304780496, id=2865, cme=0x0000000100cbfcf0, argc=1, argv=0x000000016fdfab00, kw_splat=0) at vm_method.c:2720:20
frame #25: 0x00000001003c440c miniruby`check_funcall_exec(v=6171896792) at vm_eval.c:589:12
frame #26: 0x00000001000dec00 miniruby`rb_vrescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792, args="Pȗ") at eval.c:919:18
frame #27: 0x00000001000deab0 miniruby`rb_rescue2(b_proc=(miniruby`check_funcall_exec at vm_eval.c:587), data1=6171896792, r_proc=(miniruby`check_funcall_failed at vm_eval.c:596), data2=6171896792) at eval.c:900:17
frame #28: 0x000000010039008c miniruby`check_funcall_missing(ec=0x0000000100b06f40, klass=4304923536, recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, respond=-1, def=36, kw_splat=0) at vm_eval.c:666:15
frame #29: 0x000000010038fa60 miniruby`rb_check_funcall_default_kw(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000, def=36, kw_splat=0) at vm_eval.c:703:21
frame #30: 0x000000010038fb04 miniruby`rb_check_funcall(recv=4304780496, mid=3233, argc=0, argv=0x0000000000000000) at vm_eval.c:685:12
frame #31: 0x00000001001c469c miniruby`convert_type_with_id(val=4304780496, tname="String", method=3233, raise=0, index=-1) at object.c:3061:15
frame #32: 0x00000001001c4a4c miniruby`rb_check_convert_type_with_id(val=4304780496, type=5, tname="String", method=3233) at object.c:3153:9
frame #33: 0x00000001002d59f8 miniruby`rb_check_string_type(str=4304780496) at string.c:2571:11
frame #34: 0x000000010014b7b0 miniruby`io_encoding_set(fptr=0x0000000100d09ca0, v1=4304780496, v2=4, opt=4) at io.c:11655:19
frame #35: 0x0000000100139a58 miniruby`rb_io_set_encoding(argc=1, argv=0x000000016fdfb450, io=4308334032) at io.c:13497:5
frame #36: 0x00000001003c0004 miniruby`ractor_safe_call_cfunc_m1(recv=4308334032, argc=1, argv=0x000000016fdfb450, func=(miniruby`rb_io_set_encoding at io.c:13487)) at vm_insnhelper.c:3271:12
...
frame #43: 0x0000000100390b08 miniruby`rb_funcall(recv=4308334032, mid=16593, n=1) at vm_eval.c:1137:12
frame #44: 0x00000001002a43d8 miniruby`load_file_internal(argp_v=6171899936) at ruby.c:2500:5
...
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.