Fabric Rendering API

[grondag]

This PR is a successor to #45 and #55 made in response to feedback. It specifies interfaces and creates hooks for the implementation of a ModelRenderer that provides the following features:

• Enhanced static block rendering: emissive rendering, overlay textures, multiple blend modes (solid, cutout, translucent) in the same model.
• Dynamic block models, with or without block entities: some or all of a block model can be generated during chunk rebuild.
• Dynamic Block Entity Rendering: some or all of a block model can be re-buffered every tick or every frame, with the option to redraw prior buffers if model state is unchanged.
• Enhanced item model rendering: emissive, etc. but limited to variation provided by ModelItemPropertyOverrideList.
• Freedom for ModelRenderer implementations to employ novel lighting, rendering and optimization techniques with minimal technical constraints and maximum compatibility for mod content using these interfaces.

This freedom is achieved by delegating most of the functionality to the ModelRenderer implementation. Some implementations may focus on aesthetics (shaders) and some may focus on performance, and others still may try to balance both.

To enable compatible variation, the standard hides vertex formats, vertex data structures, and other implementation details from mod authors, instead requiring the implementation to provide builders for creating materials and enhanced/dynamic models. Mod authors who use these interfaces can be assured their content will render well across a diverse range of implementation approaches.

A reference implementation to prove viability and offer baseline functionality until more advanced options emerge is in progress here.

[grondag]

Working on a basic implementation is showing me the need for some adjustment that I will be making:

Some model implementations could benefit from serializing materials at primitives vs having to hold references. It would be a trivial thing for ModelMaterial to expose an integer index and to make it retrievable by same.

Hiding vertex formats from models does nice things for the renderer implementation, but forcing the baked model implementation into the renderer is too limiting for model makers and too burdensome for the renderer. It makes it harder to do something like WeightedBakedModel with enhancements, for example. (Could be done, but would have to be a fully dynamic model, which is wasteful.)

I plan to solve this by changing the ModelBuilder into a VertexBuilder and revert to letting model implementations decide how to organize the resulting vertex data in their implementations. Models still won't know vertex formats, only the material and per-quad stride. I'll then add methods for the ModelVertexConsumer to receive the earlier-baked vertex data in bulk, similar to the methods currently available for block and item formats.

This will allow models to organize vertex content in ways that enable multi-part or otherwise variable but still pre-baked models - and still allow the model renderer to control all vertex formats and receive vertex data via direct transfer from an array, ideally batched by material. Model renderers will not be responsible for nor constrain model implementations.

[grondag]

Should we have a standard material which emulates the vanilla pipeline, or four standard materials which correspond to each render layer?

Definitely the first. There is no 1:1 mapping from render layers to material, because material also defines shading, which is normally inferred from block brightness, model ambient occlusion and quad color index.

Making models use render-layer specific material would actually make implementations harder, because it would be something like 16 materials in combination and models would somehow need to replicate the determination that is normally made on the fly by the vanilla renderer.

The "MATERIAL_STANDARD" material addresses this. BlockRenderLayer (blend mode) and ShadingMode are both null on the standard material, meaning the renderer should apply the vanilla logic to infer them from block and model attributes. It is identical to the "blank" material you'd get from a new material builder.

It is important to remember the ModelRenderer doesn't accept quads - there are no quad instances to be found anywhere - it's vertex data all the time. So attributes that used to live on quads and attributes we want to add, like emissive lighting, texture layers or shaders, needed a new home: Model Material.

Best practice for model creators will be to specify the rendering characteristics you want via the material builder and let the renderer figure out which material instance that actually equates to depending on the pipeline.

edit: strike derp

[grondag]

I don't like hardcoding the two liquids here.

In general, I hope to add a hook which lets you register "fluid rendering info", containing the necessary textures in particular - I would add the material to that.

Materials don't know anything about texture sprites. All UV coordinates going into the pipeline should be pre-baked. They only define rendering characteristics, which typically wouldn't change.

The primary use case for these two is when the renderer includes fancy shaders for water/lava rendering and model authors want to have those same effects applied to their own textures or to a surface that is emulating water/lava.

Edit: Maybe I misunderstood - if you want to define all the fluid stuff in one place that makes sense. Would imply that renderers who do implement fancy fluids would need to use that facility to do so, but I don't see a problem with that.

[asiekierka]

if you want to define all the fluid stuff in one place that makes sense

I don't want to special-case water and lava as fluids or surfaces. And that too.

[grondag]

Will remove. Think I misunderstood some of your comments in Discord regarding fluid renders. Was trying to make you happy. :-)

[asiekierka]

I think, once we add fluid rendering hooks (the ability to specify custom textures for fluid rendering, to be exact), it's not a far stretch to possibly add a way to control what material fluids render with, with defaults.

[grondag]

Input Requested

The renderer no longer owns model creation and it doesn't accept baked quads - only materials and associated vertex data. But this leaves three properties of BakedQuad with no home: useAmbientOcclusion, colorIndex and face. Where are they stored and how does the renderer receive them?

The first two are easy:

• useAmbientOcclusion goes away - material shadeMode does the job better
• colorIndex can be tracked by the model however it wants and passed to the renderer as another parameter alongside material

But face is a problem. It is NOT cull face, which the model already knows. It's used for lighting, determined during quad bake based on quad geometry and should never be null. (For face-aligned quads it will usually be the same as cull face, but lighting needs it every time.)

The renderer handles baking for enhanced models, so the model will only know the lighting face if we make a way for the renderer to pass it back to the model as an output of vertex baking. But I think that's an unnecessary complication - the model itself doesn't need it, and it's one more thing that can go wrong in the the back-and-forth.

I don't want to bring back BakedQuad instances or turn Material into BakedQuad by another name, so I propose to have the renderer store the light face with the vertex data itself. Essentially, it would report a quad stride that is 1 more than what it needs for the vertex data and store it there and retrieve it for world-dependent lighting when the quad is about to be buffered.

It may seem unorthodox to have metadata tacked on to the vertex data, but there's nothing stopping us because the renderer fully owns (and hides) vertex formats. It also means the lighter can cache additional geometric characteristics (is the quad square, for example) to avoid re-analyzing geometry during lighting.

Implementations wouldn't have to store lighting face or other metadata with the vertex data. They could instead re-analyze the geometry at world-lighting time. It can be expensive for non-cubic quads but simpler implementations might use that approach. However I would not want to leave it as the only way.

One minor downside I see is that quads can't be continuously packed - you couldn't copy multiple quads as-is to a vertex buffer, for example, because metadata is interleaved. However, if a renderer is doing world lighting on the near side of the buffer it will probably be going quad-by-quad in any case, so this doesn't seem like a significant penalty.

Any objections to this proposed approach? Any other ideas on how to handle?

An alternate option I don't like much is to have VertexBuilder populate two arrays - one for vertex data and one for meta data. Too complicated, IMO, and exposes render-specific messiness to the models that will only cause problems.

[asiekierka]

Material properties more useful to models than I expected... especially during texture bake. See no harm in exposing them.

In what scenario?

[grondag]

The model would had to have known all the attributes anyway at some earlier point because the model needed them to specify the material and pass the material to the renderer. And the model already has to track material per quad. Exposing the attributes means the model doesn't have to persist them anywhere else if it later has need of them - it can look them up from the material.

The main use case is textureDepth(). All light baking and color blending belong to the renderer but models are still responsible for baking texture coordinates, so models need a way to know when a quad has more than one texture layer.

The other material attributes are less useful but models may find some utility in analyzing the material blending/lighting properties in retexturing use cases.

Models still won't know the renderer's vertex formats and the renderer will need the material implementation to expose the attributes anyway for its own purposes, so it did not seem like a significant burden or constraint on renderer implementation.

[asiekierka]

Making this required is, IMO, an idea worth considering. (adding a default endBegin method not unlike Profilers would also be worth considering then) What's not a good idea - from the perspective of analysis/transformation/processing - would be variable-length strides per material.

[LemmaEOF]

This has an approval and hasn't had any updates for a while. Anything else this needs before it can be merged?

[grondag]

Asie wanted jar-in-jar before this went live so the default renderer could be easily distributed with Fabric. And there may need to be a special-case hook added (Asie wasn't specific about where/how) to disable the default renderer when another one is present.

I'll also need to update this PR and Indigo to the latest API/mappings before release. I usually do that each snapshot but may skip this last one - doesn't seem like it will get merged until loader is done, and I have no idea when that will be.

Lastly, I think the api package is awkward (net.fabricmc.fabric.api.client.model.fabric) but haven't thought of something better. Would be open to suggestions.

Anyone in a hurry can use FREX and a temporary distribution of Indigo for development. They will have to change their imports when this is finally merged.

[grondag]

Packages and mixins are now organized for new API split. Let me know if this looks OK and I'll update Indigo to match.

[asiekierka]

Looks good, albeit I'm not sure if we're gonna keep calling it fabric-renderer, per se, as we have a fair amount of rendering-related modules that don't belong with it.

[grondag]

Yeah, I actually agree but what else to call it?

[modmuss50]

Looks good, not sure if v0 is correct, my understanding is v1 is used for new apis. Check with @asiekierka on this.

[asiekierka]

Mother of all PRs, someone finally got to it. And it was me... #189