Static evaluation of method_exists (when true)

[timholy]

Given that I encouraged @nalimilan to shoot for #16401, I felt I should chip in by trying to figure out how to evaluate method_exists at compile time. Also of apparent interest to @stevengj (#7088).

I tried to simply follow #7088, but there are clearly some differences. Here's my test script:

function foo(a, b)
    if ccall(:jl_method_exists, Cint, (Any, Any), typeof(-).name.mt, Tuple{typeof(-), typeof(a), typeof(b)}) != 0
        return a - b
    end
    b
end

function foo2(a, b)
    if my_method_exists(-, (typeof(a), typeof(b)))
        return a - b
    end
    b
end

@inline function my_method_exists(f, t)
    return ccall(:jl_method_exists, Cint, (Any, Any), typeof(f).name.mt, makett(f, t)) != 0
end

using Base.@pure

@pure function makett(f, t)
    t1 = Base.to_tuple_type(t)
    Tuple{isa(f,Type) ? Type{f} : typeof(f), t1.parameters...}
end

Following with gdb, it's clear that foo works (confirmed by @code_native, though in contrast with isleaftype this happy outcome is not revealed with @code_warntype). However, with foo2 it doesn't. It also doesn't work if I evaluate makett and store it in a variable before issuing the ccall. Do I have to statically evaluate the call to makett, or am I barking up the wrong tree here?

[yuyichao]

codegen is too late for this to help type inference.

[yuyichao]

Can you just make method_exists @pure? It should be apart from #265 and that's effectively what you are assuming here anyway?

[timholy]

codegen is too late for this to help type inference.

It's not to help type inference (that's already done), it's to avoid the run-time method lookup.

Can you just make method_exists @pure?

If I put @pure in front of my_method_exists, then @code_native says no.

[yuyichao]

#!/usr/bin/julia -f

using Base.@pure

function foo(a, b)
    if ccall(:jl_method_exists, Cint, (Any, Any), typeof(-).name.mt, Tuple{typeof(-), typeof(a), typeof(b)}) != 0
        return a - b
    end
    b
end

function foo2(a, b)
    if my_method_exists(-, Tuple{typeof(a), typeof(b)})
        return a - b
    end
    b
end

@pure function my_method_exists(f, t)
    return ccall(:jl_method_exists, Cint, (Any, Any),
                 typeof(f).name.mt, makett(f, t)) != 0
end

@pure function makett(f, t)
    t1 = Base.to_tuple_type(t)
    Tuple{isa(f,Type) ? Type{f} : typeof(f), t1.parameters...}
end

@code_warntype foo2(1, 2)
@code_llvm foo2(1, 2)

Variables:
  #self#::#foo2
  a::Int64
  b::Int64

Body:
  begin  # /home/yuyichao/tmp/julia/a.jl, line 13:
      $(QuoteNode(true)) # /home/yuyichao/tmp/julia/a.jl, line 14:
      return (Base.box)(Int64,(Base.sub_int)(a::Int64,b::Int64))
  end::Int64

define i64 @julia_foo2_49878(i64, i64) #0 !dbg !6 {
top:
  %2 = sub i64 %0, %1
  ret i64 %2
}

Tuple of types can't be inferred as good as tuple types (const prop should in principle work though.)

[yuyichao]

It's not to help type inference (that's already done), it's to avoid the run-time method lookup.

My help type inference I mean help const prop (and dead code elimination) in type inference.

[timholy]

That's sweet! I should have tested Tuple{} vs ().

Presumably one only wants to "inline" the answer when it evaluates as true. I'll try combining your solution and mine.

[timholy]

OK, this proved to be a little subtle. The fact that you only want to evaluate this at compile time when it evaluates to true means that you can't use @pure. The fact that a ccall wants expressions for all of its arguments makes this difficult/impossible to encapsulate in a function. So, I introduced a macro @method_exists to be used when you want this to evaluate at compile time, when true.

Now including docs and tests. I believe this is ready to merge.

[yuyichao]

The fact that you only want to evaluate this at compile time when it evaluates to true

Why is that?

[timholy]

Because I'm hoping to pay the taxidermy fees for whoever wants to stuff and mount #265 on their wall. I didn't want to introduce another mechanism by which behavior depends on the order in which functions are defined and run.

[yuyichao]

I don't think this will make #265 much worse and this is also the behavior we want in the long term...

[timholy]

Are you saying we'll also start compiling in throw(MethodError(...)) if the function doesn't exist yet?

[yuyichao]

Yes and only recompile the method when you hit the global scope again.

[tkelman]

would be better to move these inline rather than adding to this file

[timholy]

Wow, that's a huge change. This is happening soon enough that I/you can go ahead and just do the @pure implemenation? (Feel free to do it yourself, since you suggested it, or I will if you'd prefer.)

[vtjnash]

I don't think this is @pure, which makes me hesitate to add this right now.

[mschauer]

Currently this can only have a one-sided solution. There might be a place for language to state that a function is "intentionally left blank", something like size(x::Number) = nomethod() which then would produce a method error on call, a warning if a method is (re-)defined and would return method_exists(...)==false

[StefanKarpinski]

If we had that it would probably be better spelled as @nomethod size(x::Number) but it feels weirdly redundant. No kidding there's no method – we didn't define one!

[timholy]

During my travels I missed @vtjnash's point. Sounds like I should rebase this and merge? (EDIT: in case it's not clear, this solution doesn't use @pure; it's @yuyichao's that does.)

[mschauer]

Not really redundant per se, rather an alternative to defining traits like HasThatfunction() giving the same information. I read it as promise (to be broken at one's own risk) not to monkeypatch a certain method which then allows full static evaluation of method_exists.

[timholy]

Rebased. Since this implementation doesn't use @pure, I'm going to merge this soon in the absence of other objections.

[vtjnash]

not all of the fields in typename are constant

[timholy]

This only needs the mt field, is that safe? (It also only needs the name field of a datatype, so if there's any danger there I could restrict it.) EDIT: of course the method table changes as you add new methods, but presumably the table itself is constant?

[vtjnash]

No, the table itself is not constant

[timholy]

Ah. To me that sounds like "we should close this without merging." Or, if the overall functionality is desirable, "find a way to do it that doesn't involve jl_static_eval, so it doesn't imply this is safe generally."

[vtjnash]

Now implemented (though possibly documented wrongly now)