extraneous type parameter induces segfault

[jrevels]

Found while working on #27736:

julia> # removing the `S` here fixes the segfault
       function g(x::T) where {T,S}
           cf = @cfunction identity Ref{T} (Ref{T},)
           GC.@preserve cf begin
               fptr = Base.unsafe_convert(Ptr{Cvoid}, cf)
               ccall(fptr, Ref{T}, (Ref{T},), x)
           end
       end
g (generic function with 1 method)

julia> g(1)
[1]    35506 segmentation fault  julia

julia> versioninfo()
Julia Version 0.7.0-beta.9
Commit d7bf89fc1c (2018-06-25 13:53 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin17.5.0)
  CPU: Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

[pablosanjose]

A milder form of this (assuming it's related) also happens without segfaulting.

julia> @noinline function foo(x::T) where {T}
           zero(T)
       end
foo (generic function with 1 method)

julia> @noinline function bar(x::T) where {T,S}
           zero(T)
       end
bar (generic function with 1 method)

julia> @btime foo(0)
  0.023 ns (0 allocations: 0 bytes)
0

julia> @btime bar(0)
  11.385 ns (0 allocations: 0 bytes)
0

[JeffBezanson]

I poked around a bit and I think this is a bug in cfunction inside functions that use jl_fptr_sparam. Stacktrace:

#0  0x00007ffff74ceb65 in jl_new_bits (dt=0x0, data=0x7fffe51486d0)
    at /home/jeff/src/julia/src/datatype.c:535
#1  0x00007fffe2b8e338 in jlcapi_cfunction_32704 ()
#2  0x00007fffe2b8e257 in macro expansion; () at gcutils.jl:87
#3  japi3_g_32703 (x=<optimized out>) at REPL[1]:3
#4  0x00007ffff7488f93 in jl_fptr_sparam (m=0x7fffe6503810, args=0x7fffffffd520, nargs=2)
    at /home/jeff/src/julia/src/gf.c:1828
#5  0x00007ffff7488e95 in jl_fptr_trampoline (m=0x7fffe6503810, args=0x7fffffffd520, nargs=2)
    at /home/jeff/src/julia/src/gf.c:1813

@pablosanjose Good observation. What happens is that S cannot be determined from the arguments. Whenever that is the case, the compiler assumes it doesn't have enough type information to determine S, and so it arranges to be able to pass S at run time in case more type information is available then. This kind of call takes a bit longer. Of course for this function, it will never be possible to determine S, but we don't make that distinction since we just assume this case isn't important (you can just delete S from your code!).

[Drvi]

@JeffBezanson I'll note that the solution is not always as easy to implement as removing the redundant type param, take these examples:

julia> using Test

julia> module Unbound
          f(args::T...) where {T} = 0

          struct Foo{T}
              f::Union{Nothing,Vector{T}}
          end

          struct Bar end
          const Grok{T} = Union{Foo{T}, Bar} where {T}
          g(x::Grok{T}) where {T} = 0

          struct X{A,B} end
          z(x::Union{X{A,B},Foo{A}}) where {A,B} = 0
       end
Main.Unbound

julia> Test.detect_unbound_args(Unbound)
[1] g(x::Union{Main.Unbound.Bar, Main.Unbound.Foo{T}}) where T in Main.Unbound at REPL[15]:10
[2] z(x::Union{Main.Unbound.Foo{A}, Main.Unbound.X{A, B}}) where {A, B} in Main.Unbound at REPL[15]:13
[3] f(args::T...) where T in Main.Unbound at REPL[15]:2
[4] Main.Unbound.Foo(f::Union{Nothing, Vector{T}}) where T in Main.Unbound at REPL[15]:5

IIUC, the vararg function encounters a case where f() method is defined but has no input argument to be bound to T. Foo generates a constructor which associates a type param with a union, where only some of its members are parametrized (the constructor most likely fails, but is still picked up by the tooling). g is similar, adding it as it took longer for me to figure out. z is a variation on this theme, with a different number of provided and expected type params.

I've seen all of these examples in production code. Hopefully, these examples are helpful.

[vtjnash]

Those are different from this (which doesn't use it at all). For those, the solution is usually to move the where clause inside the function declaration (for example: z(x::Union{X{A,B},Foo{A}} where {A,B})), so that it does not appear to be expecting a value in the Test.detect_unbound_args test (or just ignore it in that test).