Add user-documentation for `isprecompilable`

[charleskawczynski]

Over at Clima we're running into some latency issues, and a lot of our code seems to be, according to the docs, not "naively precompilable". We generated flame graphs using @snoop_inference, but most of them are red. There isn't really any mention of what conditions are needed to make code precompilable.

Our code satisfies a lot of other important properties:

• it's type stable as I can see (we pass JET tests)
• not allocating
• No type piracy
• we've reduced loads of method invalidations

Any tips are greatly appreciated.

We have experiments that, as far as I can tell, result in the compiler allocating ~100 GB of memory and takes ~7 minutes to compile for the first time (99% of the time is compilation). I can't easily make a "simple reproducer" for this, but we are definitely experiencing development pains as this is difficult to develop with.

[timholy]

"Naively precompilable" means something you don't need PrecompileTools for: Julia's own mechanisms will (as of 1.9) cache that code. So red bars just mean you need to ensure they are compiled with PrecompileTools.

Assuming you're already doing this, something is going wrong. What happens if you create a Startup package with @recompile_invalidations? I'm not saying this is a good final solution, but if it doesn't work then there's something more fundamentally broken.

[charleskawczynski]

"Naively precompilable" means something you don't need PrecompileTools for: Julia's own mechanisms will (as of 1.9) cache that code. So red bars just mean you need to ensure they are compiled with PrecompileTools.

Ok, thanks for the clarification on that. But I'm still not totally clear on something. When, (and what) will julia decide to "naively" precompile? For example, I could imagine Julia precompiling

struct Foo{A <: Float64}
  a::A
end
bar(::Foo) = 1

for Float64, but what would it naively precompile in the case where

struct Foo{A}
  a::A
end
bar(::Foo) = 1

?

Assuming you're already doing this, something is going wrong.

We're using PrecompileTools a bit, but it's really not helping much. I think it's because we're in a tough spot because we use our low level data structures, called DataLayouts, with Base.Broadcast. Our DataLayouts contain the names of variables and their array sizes for specialization (to improve runtime performance). This results in a pretty large number of specialized functions, as we have a pretty wide variety of kernels.

This means that every low level kernel that we want to optimize/specialize is a function of:

• variable names used
• sizes of the arrays
• broadcasted objects, and the functions called in those broadcasted objects

This combination of things has given us reasonably good runtime performance (although we're working on fusing kernels, which will reduce the number of kernels but increase the complexity of them), but we're getting kind of hammered by latency.

Furthermore, compiling "vanilla" or "typical" workloads is not trivial due to the large combination of specializations. In other words, only need to miss one specialization, like spatial resolution (as this impacts array sizes), in order to miss the whole workload. I think https://github.com/JuliaArrays/FixedSizeArrays.jl could really help on this front, so I'm hoping that people are working on this, but I don't think we can bank on it. If the array sizes are out of the equation, then there might be a path to precompiling some of our kernels in our atmosphere code.

What happens if you create a Startup package with @recompile_invalidations? I'm not saying this is a good final solution, but if it doesn't work then there's something more fundamentally broken.

I haven't tried @recompile_invalidations, but I think that we have mitigated the egregious invalidations. That said, it's worth a shot trying this out given the state that we're in.

[charleskawczynski]

Actually, I can't guarantee that everything is actually inferred, I've been finding JET to give false negatives, and while I've seen Cthulhu identify those false negatives, JuliaDebug/Cthulhu.jl#566 is really a usability barrier for us. Navigating through each part of the menu takes minutes to print, and it's zillions of lines of type information.

[timholy]

what would it naively precompile in the case where

Nothing. You have to execute the code to get something to compile. Precompilation just stores already-compiled code, but if you've never executed it (or used a manual precompile directive), then there's nothing to cache. PrecompileTools does not try to go around compiling methods you've not executed during module building. That's why it's centered around a "workload"---you do work to trigger compilation, and that's what gets cached.

[timholy]

I've been finding JET to give false negatives

Make sure you're using it with concrete signatures. If you're using report_file, don't expect much useful to come out. You really can't say much about inference without a concrete signature entry point.

[charleskawczynski]

I want to say that we’re pretty good with type concreteness

[timholy]

What does that mean? There's absolutely nothing wrong with writing

f(x::Real) = ...    # abstract method signature

But when you go to precompile it you want it to be

@compile_workload f(3.0)   # concrete specialization during compilation

and tested with

JET.@report_opt f(3.0)

rather than

JET.report_file("definition_of_f.jl"; analyzer = OptAnalyzer)

The latter will miss lots of problems, the former will not.

[charleskawczynski]

What does that mean? There's absolutely nothing wrong with writing

I mean that we're pretty good with avoiding https://docs.julialang.org/en/v1/manual/performance-tips/#Avoid-fields-with-abstract-type. And yes, I understand that writing signatures for abstract types is fine.

@compile_workload f(3.0) # concrete specialization during compilation

Yes, we do this.

JET.@report_opt f(3.0)

We do this, and no errors are reported-- although Cthulhu doesn't seem happy.

JET.report_file("definition_of_f.jl"; analyzer = OptAnalyzer)

I don't use report_file..

[timholy]

Gotcha. I couldn't tell for certain if we were on the same page. Weird that JET is giving you false negatives.

You're not doing this with coverage on, by any chance? We just worked out a zinger, #413 (comment). On Julia 1.11.3 and 1.11.4 that can lead to code-deletion.

But I'd guess you're not. So in that flame graph you linked in the OP, was that exact workload precompiled? Or did you mean in #411 (comment) that you have so much specialization that almost nothing is precompilable anyway? If so, I don't really know if there's much to do, other than enhance the docs as you suggest at the top.

[charleskawczynski]

Gotcha. I couldn't tell for certain if we were on the same page.

👍🏻

Weird that JET is giving you false negatives.

Yes, it's unfortunate

You're not doing this with coverage on, by any chance? We just worked out a zinger, #413 (comment). On Julia 1.11.3 and 1.11.4 that can lead to code-deletion.

No, I typically only do this interactively without codecov. But thanks for letting me know.

But I'd guess you're not. So in that flame graph you linked in the OP, was that exact workload precompiled? Or did you mean in #411 (comment) that you have so much specialization that almost nothing is precompilable anyway?

No, none of that was precompiled, mostly due to the specialization issues that we're facing.

If so, I don't really know if there's much to do, other than enhance the docs as you suggest at the top.

I agree, and I think a bit more clarification on the docs would be great.

[timholy]

If you can review #416 for clarity, that would be much appreciated.