[1.7-rc2] Base.promote_type inference regression

[OlivierHnt]

There seems to be the following regression in Julia v1.7.0-rc2 (compared to Julia v1.6.3):

(on Julia v1.7.0-rc2)

julia> foo() = Base.promote_type(Bool, Irrational{:π})
foo (generic function with 1 method)

julia> @code_warntype foo()
MethodInstance for foo()
  from foo() in Main at REPL[30]:1
Arguments
  #self#::Core.Const(foo)
Body::Any
1 ─ %1 = Base.promote_type::Core.Const(promote_type)
│   %2 = Core.apply_type(Main.Irrational, :π)::Core.Const(Irrational{:π})
│   %3 = (%1)(Main.Bool, %2)::Any
└──      return %3

On the other hand, Base.promote_type(Bool, Irrational{:π}) seems to predict the type properly on both versions of Julia.

(on Julia v1.7.0-rc2)

julia> @code_warntype Base.promote_type(Bool, Irrational{:π})
MethodInstance for promote_type(::Type{Bool}, ::Type{Irrational{:π}})
  from promote_type(::Type{T}, ::Type{S}) where {T, S} in Base at promotion.jl:282
Static Parameters
  T = Bool
  S = Irrational{:π}
Arguments
  #self#::Core.Const(promote_type)
  _::Core.Const(Bool)
  _::Core.Const(Irrational{:π})
Body::Type{Float64}
1 ─      nothing
│   %2 = $(Expr(:static_parameter, 1))::Core.Const(Bool)
│   %3 = $(Expr(:static_parameter, 2))::Core.Const(Irrational{:π})
│   %4 = Base.promote_rule($(Expr(:static_parameter, 1)), $(Expr(:static_parameter, 2)))::Core.Const(Irrational{:π})
│   %5 = Base.promote_rule($(Expr(:static_parameter, 2)), $(Expr(:static_parameter, 1)))::Core.Const(Float64)
│   %6 = Base.promote_result(%2, %3, %4, %5)::Core.Const(Float64)
└──      return %6

[N5N3]

Replacing

function promote_type(::Type{T}, ::Type{S}) where {T,S}
    @_inline_meta
    promote_result(T, S, promote_rule(T,S), promote_rule(S,T))
end

with

function promote_type(::Type{T}, ::Type{S}) where {T,S}
    @_inline_meta
    rule1 = promote_rule(T,S)
    rule2 = promote_rule(S,T)
    rule1 === rule2 === Bottom && return typejoin(T, S)
    promote_type(rule1, rule2)
end

will fix the above example. But I think this might not be the correct way.

[martinholters]

Bisected:

c2ec70cd1842b53f45ac72ea74c94f008adbc7b3 is the first bad commit
commit c2ec70cd1842b53f45ac72ea74c94f008adbc7b3
Author: Jameson Nash <vtjnash@gmail.com>
Date:   Thu Apr 8 23:10:57 2021 -0400

    inference: limit single-level nested `Type` signature (#40379)

CC @vtjnash

The problem here is the promote_type -> promote_result -> promote_type recursion. In many cases, one of promote_rule(T,S) and promote_rule(S, T) returns Union{} which hits a dedicated promote_type method in the recursion. But

julia> Base.promote_rule(Bool, Irrational{:π})
Irrational{:π}

julia> Base.promote_rule(Irrational{:π}, Bool)
Float64

The easiest fix would be to add a specialized promote_rule method to avoid this. But I wonder how many other combinations are equally problematic. Making inference's recursion detection more liberal (like it accidentally was before) seems not great, as this is actually a case where limiting is required to ensure termination.