Async GL shader compilation

[mosra]

The goal is figuring out a way how to use KHR_parallel_shader_compile without having to give up on the current way shaders are compiled, linked and initialized in a single constructor expression:

Shaders::FooGL shader{Shaders::FooGL::UniformBuffers, 5, 3, 128};

Right now, shader compiler parallelization is only partially exploited in list-taking GL::Shader::compile() and GL::AbstractShaderProgram::link() which first submit compile/link command for all shaders and then check for errors. Since those are always called from the constructor, the max amount of parallelization is only 2 (or 3 when GS is used), which really isn't much. One option to improve this would be to call compile() and link() outside of the constructor, on multiple shader/program instances, but that means we lose the convenient single-expression RAII setup, and because the status is still checked right after all compilation jobs is submitted it won't help much on certain backends such as the shitty old D3D compiler that can sometimes take several seconds to compile a single shader.

The requirements are thus:

• keep RAII constructor initialization, because otherwise all shader interfaces would have to do additional checks for completeness
• avoid having to implement two separate code paths for direct and async shader compilation, because that'd soon become a maintenance nightmare
• remove existing limitations to the parallelism like implicit error checks or an inability to parallelize more than 2 or 3 compilations at the same time
• don't complicate the existing shader creation APIs any further, as such simplicity is still desirable for tools and quick demos where startup time isn't the topmost priority

A proposal that could check out all of the above is:

• Add void Shader::submitCompile() and void AbstractShaderProgram::submitLink() (better naming?) that only submit the compilation/link and don't check for the status
• Add bool Shader::checkCompile() and bool AbstractShaderProgram::checkLink() (naming??) that returns false and prints an error message
  • or maybe returns it in a string instead?
  • do we actually need to have a way to check compilation status at all, or only when link fails? the WebGL extension spec says // Never check compile status unless subsequent linking fails. so maybe the desired workflow is
    1. check link status
    2. if it fails, check compile status
    3. if compile failed, print compile status message
    4. if compile didn't fail, print link status message
• Make the original Shader::compile(...) and AbstractShaderProgram::link(...) just wrappers that call submitCompile() / submitLink() on all instances and then checkCompile() / checkLink() on all of them
• Add bool Shader::isCompileFinished() / bool AbstractShaderProgram::isLinkFinished() (naming??) that exposes the new query of KHR_parallel_shader_compile
• Create a Shaders::FooGL::CompileState that contains the intermediate state of the compilation as private fields (vertex, fragment GL::Shader instances, construction parameters like flags, light count)
  • Probably have it derived from a AbstractShaderProgram::CompileState base, which is able to hold the program object and allows us to call submitLink() / checkLink() / isLinkFinished() without having any AbstractShaderProgram instance yet
• Add a Shaders::FooGL::submitCompile(...) -> Shaders::FooGL::CompileState (or just compile()?) that populates the output structure and calls submitCompile() / submitLink() on each, but without checking status -- basically the "first half" of the existing constructor
  • Or have submitLink() done separately as well, to be able to submit 200 shader compilations first, 100 shader links second and then 100 error checks last? The WebGL extension spec suggests that, but does it actually matter whether I interleave compilations and linkings or not if I don't check for any status until the end?
• Add a Shaders::FooGL::FooGL(Shaders::FooGL::CompileState&&) that then creates the real shader instance from the intermediate state by checking for errors, querying uniform locations and performing initialization -- the "second half" of the existing constructor
• Make the original Shaders::FooGL::FooGL(...) constructor simply delegate to FooGL{compile(...)}

This would then allow workflows such as the following where one populates intermediates for desired shader variants, does independent work in the meantime and then (either after checking for completion or forcibly) populates real instances:

Containers::Array<Shaders::FooGL::CompileState> intermediates{InPlaceInit, {
    Shaders::FooGL::submitCompile(Shaders::FooGL::UniformBuffers, 1, ...),
    Shaders::FooGL::submitCompile(Shaders::FooGL::UniformBuffers, 5, ...),
    Shaders::FooGL::submitCompile(Shaders::FooGL::UniformBuffers|Shaders::FooGL::AlphaMask, 1, ...),
    ...
}};

// Load meshes, textures, ...

Containers::Array<Shaders::FooGL> shaders;
for(Shaders::FooGL::CompileState& intermediate: intermediates) {
    // optionally check for intermediate.isLinkFinished() and skip / postpone
    Shaders::FooGL shader{std::move(intermediate)};
    arrayAppend(shaders, std::move(shader));
}

Further work:

• Create a Containers::Either<T, U> which would be like an Optional but providing an actual value for the "disengaged" state, with the usual -> accessing the T and asserts if not there
  • probably needs a better-fitting name, it's more like ValueExceptIfNotThenItsThisOtherThing
• Containers::Either<Shaders::FooGL, Shaders::FooGL::CompileState> could then hold the intermediate for as long as the compilation is happening in the background, and then when it's finished it gets flipped to the final type
• And ultimately AbstractShaderProgram could provide some wrapper around (AsyncShader<T>: Containers::Either<T, T::CompileState>), which automatically switches the underlying type when compilation is done or forces a stall if the user actually accesses the to-be-populated instance with ->

Example:

GL::AsyncShader<Shaders::FooGL> shader{Shaders::FooGL::UniformBuffers, 5, ...};

// Scheduling other work in the meantime
while(!shader.isLinkFinished()) { ... }

// Or accessing directly and forcing a completion
shader->draw(mesh);

Cc: @Squareys

[pezcode]

Seeing all these isFinished etc., would std::future<CompileState> be useful here? I don't know a lot about the underlying implementation, but it'd provide a useful interface for querying and waiting for results.

[mosra]

(My STL skepticism immediately kicked in, sorry 😆) If I understand correctly, std::future deals mainly with cross-thread synchronization, so mutexes, atomics, allocating shared state, virtual calls and whatnot, so I think that's a bit of an overkill for GL which is everything but usable in a multithreaded context :)

[pezcode]

std::future is not thread-safe so I don't think you'll find any mutexes in there. std::shared_future would be the thread-safe version of it. Not to say there's no reason for skepticism but it might be worth investigating.

[dranikpg]

Hi! I've taken this up, I hope a draft will be ready in a few days

[mosra]

Oh? That's very cool :)

Since this would be I think your very first contribution to Magnum, feel free to post a WIP state as soon as you have something so we can agree on the direction and workflow early on.

[mosra]

Implemented with #576.