WebGLRenderer: Add .setLookCDL()

[donmccurdy]

• Alternative to WebGLRenderer: Add 'AgXPunchyToneMapping' #27618

Adds a basic primary color grading API to THREE.WebGLRenderer, based on ASC CDL v1.2. Ultimately, I think full-featured color grading APIs belong in post-processing effects and/or TSL nodes, not THREE.WebGLRenderer... but our tone mapping implementations are very limited today by not having anything comparable to Blender's “Looks”. Without that, we need the hard-coded brightness boost for ACES Filmic, and AgX is stuck at a base contrast never intended as a one-size-fits-all look.

This PR offers a more flexible API than just a list of preset Looks, and is meant to be fully compatible with AgX and ACES Filmic tone mapping. Basic support in our other tone mappers is possible, but I'm not confident the other tone mapping methods were designed to handle color grading robustly, so issues like hue shifting may occur.

For our purposes, “primary color grading” means:

1. The first step in a color correction workflow
2. Affects the entire image¹, no masks or power windows
3. Applied before tone mapping, minimizing information loss
4. Supports open domain [0,∞] input and output, compatible with “HDR” workflows
5. Efficient enough to run on any WebGL-compatible device

The CDL, chosen to keep the implementation lightweight, is represented as a single vec4 uniform, representing: slope, offset, power, and saturation. In a full CDL implementation, the slope+offset+power (SOP) parameters would each be of type vec3 (a value for each RGB channel). I did not include per-channel SOP values, as they are not required to match Blender's default Looks.

Usage:

renderer.toneMapping = THREE.AgXToneMapping;
renderer.setLookCDL( 1.0, 0.0, 1.35, 1.4 ); // AgX Punchy

---

Parameters

Admittedly, CDL parameters are not the most “artist-friendly” color grading API, but their advantages are important. The CDL's implementation is precisely defined, and consistent across implementations, requiring only that it be applied to inputs with the same color space on both sides. The CDL is valid for open domain [0, ∞] inputs, and can be safely applied before tone mapping. The CDL can be applied to either linear or log inputs: the results differ, but are useful for different things in each case. From an excellent answer on Blender Stack Exchange ...

Many formulas passed down through Stack Exchange and other sites with information all too frequently assume things about RGB that shouldn't be assumed. While Lift, Gamma, Gain is one such display referred assumed formula, there are many other traps like this. The AdobePDF specification lists many of them, and many of those formulas have sadly ended up in scene referred image manipulation applications.
...
The CDL has no such limitation. It can work on RGB values of any range.

While the SOP parameters will need some explanation, they're capable of representing more artist-friendly concepts, if placed in an appropriate UI. For example, 'contrast' control might be implemented as:

const slope = contrast;
const offset = ( 1 - contrast ) x pivot;
const power = 1;

A pivot of 0.2 will match the contrast changes found in Blender's looks.

Tone mapping

Currently, these changes are enabled only with AgX tone mapping.

Of our current tone mapping implementations, I'm confident that AgX and ACES Filmic can be combined with color grading and wide gamut color management correctly. Both provide a corresponding log space (AgX Log and ACEScc respectively) in which to apply the CDL. I have some remaining questions about ACES Filmic, so that step is commented out for now.

Other tone mappers (or even no tone mapping) could be optionally supported by the CDL, but we'd want to choose some log space in which to apply the CDL. My inclination is that for fully-supported color management, color grading, and future wide gamut support, efforts should be focused on AgX and ACES Filmic.

Looks

With these four CDL parameters we can support Blender's default color management looks. Derived parameters for each look are included below.

look	slope	offset	power	saturation
Punchy	1.00	0.000	1.35	1.40
Very High Contrast	1.57	-0.114	1.00	0.90
High Contrast	1.40	-0.080	1.00	0.95
Medium High Contrast	1.20	-0.040	1.00	1.00
Base Contrast	1.00	0.000	1.00	1.00
Medium Low Contrast	0.90	0.020	1.00	1.05
Low Contrast	0.80	0.040	1.00	1.10
Very Low Contrast	0.70	0.060	1.00	1.15

These looks were designed for AgX. While they could be used with ACES Filmic, looks like "High Contrast" cannot guarantee consistent contrast from one tone mapper to another.

Base Contrast — the AgX default — is a good choice for shooting flat, offering an excellent baseline for additional color grading. Starting from highly-saturated colors can make later color grading more difficult. However, users who just want a satisfying image “out of the box” will probably want to choose a look with higher contrast.

Punchy — AgX preset with greater saturation and contrast, and more likely to match the expectations of users switching over from ACES Filmic or Khronos PBR Neutral to AgX. See below, a comparison between Khronos PBR Neutral (left) and AgX Punchy (right).

Golden — included in some AgX implementations, but not in Blender — requires per-channel controls over the SOP parameters, and so is omitted here.

Greyscale — no exact match with the CDL, but simply reducing saturation to 0 will create a greyscale image, and the remaining parameters work as before, e.g. for contrast adjustments.

---

The CDL is not intend to be the complete color grading workflow, but to complement tone mapping in getting close to the desired look, before making other (potentially lossy) color grading changes. A more flexible color grading workflow would look something like Filament's ColorGrading, or OCIO's GradingPrimaryTransform. These might be good goals for us to support at some point, but I wasn't confident the added complexity would be welcome or necessary in WebGLRenderer. More high-level APIs could remain in post-processing effects, or be expressed as a set of color correction nodes with TSL.

I'll open a new PR, separately, for a usage example.

/cc @gkjohnson @WestLangley

Footnotes

1. One notable exception, materials excluded from tone mapping are not affected by primary color grading. Un-tonemapped materials could be included in the future, but this requires some complex decisions. ↩

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

Filesize dev	Filesize PR	Diff
684 kB (169.4 kB)	684.6 kB (169.6 kB)	+586 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

Filesize dev	Filesize PR	Diff
460.9 kB (111.2 kB)	461.5 kB (111.4 kB)	+586 B

[WestLangley]

A few thoughts...

applyPrimaryGradingCDL() does not belong in the tone mapping file, IMO.

Color correction and grading is output-device-independent. But when we support HDR monitors, our tone mapping will be output-device-specific.

Perhaps create LookModification.glsl.js, ColorGrading.glsl.js, or ColorCorrection.glsl.js and add it there, instead.

In addition to ASC CDL, we could also support a 3D LUT approach.

//

three.js implements tone mapping "inline" and optionally via "post processing". Do you intend ASC CDL to be implemented in both approaches?

[donmccurdy]

applyPrimaryGradingCDL() does not belong in the tone mapping file, IMO.

I'm happy to place the applyPrimaryGradingCDL function elsewhere. I do think it needs to be invoked from the active tone mapping function, after toneMappingExposure, and in the appropriate log space for each tone mapper, is that OK?

In addition to ASC CDL, we could also support a 3D LUT approach.

I'm not sure about putting a 3D LUT before tone mapping, won't it clip? We do have THREE.LUTPass available. Certainly valid after, or in place of, tone mapping. In pre-AgX versions of Blender, each of the Looks was implemented with a 1D LUT applied in a log space. Now, most use OCIO::GradingPrimaryTransform. I've derived the CDL parameters above from the GradingPrimaryTransform inputs.

Do you intend ASC CDL to be implemented in both approaches?

Yes, I intend to support both, as we do with .toneMappingExposure. Though the CDL is less essential for users of post-processing, since they can easily add existing color grading passes or create custom ones. We could add a separate CDLShader, for that matter, as I did while prototyping for this PR:

https://github.com/donmccurdy/three-color-grading/tree/main/src/shaders

[WestLangley]

My understanding from the ACES Imaging Pipeline is the Look Modification Transform, which includes color correction/grading, is applied in Scene Referred Linear Space prior to exposure and tone mapping.

[donmccurdy]

My understanding from the ACES Imaging Pipeline is the Look Modification Transform, which includes color correction/grading, is applied ...

... in scene referred space: Agreed!

... in linear space: I think it depends. Perhaps parts of an LMT are done in linear space. But the CDL's 'slope' parameter behaves as exposure in linear and more like contrast in log, and without the latter I imagine it would be difficult to create these looks. Also see CG Cinematography, and references to using ACEScc or ACEScct for color grading. The ACES Filmic tone mapper already passes through Linear AP1, with the same primaries as ACEScc, so I believe that's a practical place to convert to ACEScc and apply the CDL.

... prior to exposure: I'm not sure, but just as it wouldn't make sense to apply a LUT prior to exposure, I worry that applying the CDL prior to exposure may create a more difficult workflow. It also wouldn't match existing AgX implementations.

... prior to tone mapping: Agreed!

---

If we want to support a LUT as well, that could look like ...

const lut = new PrimaryGradingLUT()
  .setContrast( ... )
  .setLift( ... )
  .setGain( ... );

renderer.setPrimaryGradingCDL( 1, 0, 1, 1 );
renderer.setPrimaryGradingLUT( lut.build() );

... where the LUT would be applied after the primary CDL and before tone mapping.

Rebuilding a LUT for each change is not completely ideal, so I prefer the post-processing or TSL-based approaches in the long run. But if we want to support a more complete primary color grading API sooner, this would avoid needing a larger API in WebGLRenderer.

[WestLangley]

@donmccurdy Many thanks for your work on this!

so I believe that's a practical place to convert to ACEScc and apply the CDL

Perhaps the working color space is converted to log space such as ACEScc, CDL is applied, and then the color space is converted back to the prior working color space. To me, that makes CDL a stand-alone operation.

//

It is not clear to me yet if exposure via CDL would be in lieu of three.js tone mapping exposure, or in addition to it. Currently, I am leaning toward the latter.

[donmccurdy]

Perhaps the working color space is converted to log space such as ACEScc, CDL is applied, and then the color space is converted back to the prior working color space...

Conceptually I agree! As implementation, I would prefer not to bake ACES concepts into our color pipeline, and would instead perhaps call this the "primary grading color space", which might be ACEScc or AgX Log, dependent on choice of tone mapping for now. Perhaps user-configurable someday, if needed.

Pipeline:

1. render to working color space
2. apply exposure control
3. convert to primary grading color space
4. apply primary grading CDL
5. convert to working color space
6. apply tone mapping
7. convert to output color space

There is an extra matrix multiply or two when breaking it out this way, but let's assume that's OK.

I'd prefer to apply the CDL in addition to exposure, as well. In my mind "tone mapping exposure" is just "exposure", and there is no reason the CDL can't come between exposure and tone mapping. It is easier to color correct if the image is already properly exposed.

[donmccurdy]

Perhaps we should call this the "Look CDL", with API renderer.setLookCDL(...). Clearer about the purpose and how we intend it to fit into the pipeline?

[donmccurdy]

Moving this to 'draft' status for now. Open issues:

• Naming (setLookCDL, setPrimaryGradingCDL, ...)
• Listed CDLs for contrast-centric looks (e.g. “Medium High Contrast”) are incorrect, and will need to be updated. I'd missed an important difference in Blender's AgX implementation and the IOLITE minimal AgX implementation.
• Example and documentation

EDIT: A user in Discord mentioned needing to modify white balance in WebXR, and post-processing not being an option (both because we don't currently support it, and for performance cost). This PR would enable white balance control if we do per-channel parameters – so that slope, offset, and power each become vec3 instead of scalar. I'm inclined to switch to the full per-channel CDL implementation and support that.

[WestLangley]

I'm inclined to switch to the full per-channel CDL implementation and support that.

Agreed!

[Mugen87]

With #29510 merged do we still want to add a CDL API in WebGLRenderer?

In order to reduce the maintenance effort, we provide certain newer features only in WebGPURenderer. Thanks to the node material and TSL the implementations become easier and produce less side effects. Regarding CDL, I vote to leave this feature out of WebGLRenderer.

[donmccurdy]

With #29510 now in WebGPURenderer I think it would make sense to put a demo together for WebGPU, try it out there, and see how we feel about using CDLs for color grading when post-processing isn't available. The CDL is not the most artist-friendly, maybe it's not the right design for WebGLRenderer anyway. With WebGPU+TSL we can of course have any number of color operations available as nodes ranging from simple to advanced.

I do think we need something more for WebGLRenderer, though... AgX is (IMHO) the most useful tone mapping option we have, but it was never meant to be used without some kind of control over the 'look', as every other implementation I'm aware of offers, it looks too low-contrast without that. #27618 would also be an option.