-
-
Notifications
You must be signed in to change notification settings - Fork 10.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sRGB and linear color spaces #578
Comments
Sorry for not answering this earlier, this is partly overwhelming partly not a priority. If you wish to elaborate with a list of concrete things to change I'd be interested. ImGui doesn't do any computation on colors, those 4 components sliders are completely abstract values from the point of view of the library. Isn't it just a problem that you can work out by:
EDIT I am clueless about sRGB / linear space vs gamma space so will need some assistance here. |
Adjusting colors by pow(1/2.2) in the shader sounds like it would work for use with an sRGB rendertarget. My main confusion right now is stuff like ImGui::ColorButton taking float colors but interpreting it as sRGB. I have to think more about this to come up with a thorough analysis. Thanks for your patience Omar! |
As a color expert, I can understand the desire for proper color handling. However, one of the key features of ImGui, as I see it, is speed, to draw live data plots quickly. We use ImGui for developers' tools which analyze real-time performance. I'll take speed over correct color, if there's any tradeoff. We can always adjust our RGB values to look good. |
fair point. |
@ocornut Would you accept PR with new color mode ImGuiColorEditMode_Float? |
Themes can cover most of the issues like a correct shade of blue on a button. To Nicholas' original post, things like color pickers should for sure allow a color space to be set, or at least a gamma. Proper color space conversion is a big job that consumes a lot of cycles. |
@nem0 yes probably, but probably start from the (idle) ColorPicker branch. |
@nlguillemot I'm assuming the images you posted weren't taken on a High DPI display? correct? I'm seeing text become blurrier when switching to sRGB, came here wondering what color space is the text --> texture --> screen pipeline working under. |
Hi, just adding my 2 cents here. We also use a sRGB back buffer, and of course I came across the same issue mentionned by Nicolas :) So, I wanted to check the status of that issue... What is the best way of handling this right now, simply by modifying the pixel shader to cancel the linear->sRGB applied on the GPU, i.e.:
For me that option is working correctly and I can afford the cost for that additional instruction... |
@gjaegy The vertex should already have a color attribute in linear color space instead of using a color attribute in sRGB color space + applying an approximate sRGB-to-linear conversion in the Pixel/Fragment Shader. The barycentric interpolation of the color attribute for the fragment generation will be wrong, since the color isn't linear. To be correct, one should change the following
to
and apply a color conversion from sRGB-to-linear space when writing the color value. This also means that the shaders will stay the same. Only the vertex generation on the CPU will be affected. Or the vertex shader needs to perform the conversion. This will probably result in the smallest change. I didn't thought of this shader, though it is probably the most appropriate and convenient converter. :) |
Hi Matthias, good point, I didn't think about the vertex color actually. |
A short explanation/summary of the relevant color spaces: The actual color of a pixel, outputted on a monitor, does not linearly depend on the applied voltage signal for that pixel of the monitor. For CRT monitors, the actual color is approximately proportional to the applied voltage raised to the power of a so-called gamma value, which depends on the monitor. This gamma value typically lies between 2.2 or 2.5 for CRT monitors. To bypass this gamma value, you need to gamma correct the computed colors of each pixel before presenting. By raising the computed color to a power of the reciprocal of that same gamma value, the computed color becomes proportional to the actual color. Formally:
So in general gamma correction is a technique that adapts the computed colors to the transfer function of the monitor used for outputting. Non-CRT monitors each have their own transfer function. This means that to obtain correct actual colors, the final rendering pass should adapt the computed color depending on the used monitor. These linear-to-gamma color space conversions seem like unnecessary overhead from a monitor construction point of view. It is physically perfectly possible to construct a CRT monitor with a gamma value of exactly one, ensuring that the computed color is already proportional to the actual color, and eliminating the need for gamma correction. From a perceptual point of view, removing the need for gamma correction and using a monitor where computed colors are proportional to actual colors is actually a bad idea. Typically, actual colors are represented with 8 (or 10) bits for each of the red, green or blue channel. This quantization only supports 256 (or 1024) different colors. Here, a 0 value represents completely black and a 255 (or a 1023) value represents completely white. But what about the intermediate values? If a linear encoding is used (i.e. a gamma value of 1), the majority of values would be perceptually very close to white and a very small minority would be perceptually close to black. By using a gamma encoding (e.g. a gamma value of 2.2), the distribution is perceptually more linear (i.e. equidistant intervals between black and white). The sRGB color space has similar goals, assigning a perceptual linear range of color values to the available 256 different color values. A rough approximation is transforming linear color values to sRGB color values by raising to a power of 2.2. A more accurate approximation would distinguish between a more linear correspondence near black and a gamma (2.4) encoding near white. And finally, you can just use the exact transformation between linear and sRGB color spaces. The more accurate, the more expensive the calculation will be. The sRGB color space is obviously used for sRGB monitors which are primarily used for content creation (e.g. textures, etc.). Images with a R8G8B8A8 format are in most cases represented in sRGB color space and definitely not in linear color space (as a user, you need of course to know which color space is used for the encoding). RGB color pickers typically operate in sRGB color space as well. Colors represented in linear color space can be mutually added and multiplied. This above implies that colors represented in a gamma color space or in sRGB color space cannot be used as vertex attributes during the barycentric interpolation for fragment generation. The color attribute of a vertex needs to be expressed in linear space before passing to the rasterizer stage. The pixel/fragment shader will thus always obtain a fragment color expressed in linear space. The above applies to vertex attributes, so single color coefficients. But what about textures? For textures there isn't a choice. Most textures will contain colors expressed in sRGB color space, and thus these textures should be explicitly encoded as sRGB colors (e.g. DXGI_FORMAT...SRGB). By using an explicit sRGB texture format, the hardware will perform the sRGB-to-linear conversion when appropriate. So with regard to texture sampling/filtering:
Similarly for blending. If you use blending (i.e. all blending except opaque blending), the hardware will perform the linear-to-sRGB conversion when appropriate if the render target has an explicit sRGB texture format.
Alternatively, a halffloat render target can and should be used, if linear color values need to be stored instead. This, however, is not the responsibility of ImGui. ImGui should output linear colors, and the user of ImGui should provide an appropriate render target (i.e. R8G8B8A8_SRGB or R16G16B16A16). So to summarize the changes required to ImGui:
Without knowing all the details of ImGui, I assume only the last aspect should be dealt with. Application to the D3D11 demo: main.cpp
imgui_impl_dx11.cpp
|
The colors for the classic style were incorrectly transferred into linear-space without a gamma transform when they were authored. I imagine a mock style was created in PhotoShop or Gimp, and the sRGB values were sampled from there and divided by 255. That leads to really invalid colors, because sRGB The rule of thumb for color space handling is to keep the sRGB color storage format in an array of 4 bytes (values 0-255), and linear-space color storage format in an array of 4 floats (values 0.0-1.0). Any conversion between these formats must go through gamma correction (e.g. the As @matt77hias mentioned, color pickers usually operate in sRGB space, so it makes sense for it to output This is actually a very easy problem to solve, but it is a convoluted challenge and practically everybody gets it wrong. As long as you stick to the rule of thumb, though, it's hard to screw it up. |
I see two problems related to this approach:
|
The first point is a good one. But that's why SemVer exists. Breaking changes will have to be made at some point, and it can be done safely. For the second point, you're also not wrong! The storage format can technically be anything you want. But if you allow room for confusion, you will end up with users doing things they should not. The only practical choice is to make the concrete types incompatible without an explicit transformation.
This is the part I disagree with. That may be an implementation detail of the existing code, but it is just that; an implementation detail. Other implementation details are that the colors also eventually end up as four floats on their way to the vertex shader (which may be 16-bit, 24-bit, etc.), and then eventually end up as any number of formats as they are rasterized to a framebuffer (RGBA8, BGRA8, R5G6B5, etc.), and eventually they get emitted as linear voltages. These are implementation details of various storage formats along the fixed and programmable pipelines, but they say nothing about the color space of these storage formats. The only thing I am arguing here (and I hope I shouldn't have to argue it) is that the storage representation is only important at the API level for end users. If theoretically we had types called |
In many cases, you'll end up with a R8G8B8. But you're right that you can use less or more precision. If you know this in advance, however, you can dedicate less or more precision to the format used for representing these colors on the CPU to save memory. My color space spectra are implemented as classes inheriting my base "containers" (e.g. |
my 2ct: i'm rendering everything in linear rec2020 internally and want to display colour managed for my screen. i like how currently imgui is oblique to colour spaces, so everybody can do as they like. the only change i need is at the very end i need to transform the output like so: which i suppose could be a no-op (gamma=1, matrix = identity) for folks who'd rather composite in sRGB/display colour space. to get right ui colours, i un-gamma all theme colours (convert from sRGB to linear) when initing imgui. |
We developed a quick solution to this problem for DX9: // SRGB corrects according to https://github.com/ocornut/imgui/issues/578
D3DCOLOR D3DCOLOR_sRGB_CORRECTION(D3DCOLOR color)
{
#define GAMMAVAL 2.2
// unpack D3DCOLOR into something we can manipulate
D3DCOLORVALUE color2;
color2.a = ((color & 0xff000000) >> 24) / 255.0f;
color2.r = ((color & 0x00ff0000) >> 16) / 255.0f;
color2.g = ((color & 0x0000ff00) >> 8) / 255.0f;
color2.b = (color & 0x000000ff) / 255.0f;
// gamma correct colors...
color2.r = pow(color2.r, GAMMAVAL);
color2.g = pow(color2.g, GAMMAVAL);
color2.b = pow(color2.b, GAMMAVAL);
// repack the aforementioned data into the colorvalue for returning
return D3DCOLOR_COLORVALUE(color2.r, color2.g, color2.b, color2.a);
} A practical example can be found at https://gist.github.com/gdianaty/7b81dacbb28b38afc3ba7be09edc7904 |
- Add sRGB helper class to allow user to input colors in non-linear sRGB (which will be what most people are used to). Conversion to linear sRGB is done automatic - Shaders assume linear color inputs - Textures can be marked as either sRGB (for things that are colors) or not (for things that are "maps", like normal, bump, etc). Marking color textures as sRGB encoded means that samplers will decode "automatically" to linear for use in shaders - Output framebuffers (both the backbuffer and intermediate framebuffers) are marked as sRGB, so that writes from the shaders (linear) will automatcally sRGB encode the outputs (i.e. no need to manually gamma correct as the final step of the shader) - Add "012 - sRGB" demo to show gamma correct alpha blending - Move "ImGuiEx" into Pikzel "ImGui" library. Saves duplicating it in clients. - Add a method to initialise ImGui to a "default" Pikzel ImGui style Note: ImGui does not play nice with sRGB framebuffers. This is a known issue (eg. ocornut/imgui#578 and also issues 1724, and 2468).
Another problem that was not thoroughly discussed here is the implications of alpha-blending performed by the GPU. All ImGui examples perform blending in sRGB space, so a final color output by the display can be expressed as
If colors are manually converted to linear space in the shader, then blending will be equivalent to the following:
which clearly is quite a different equation. This manifests itself as color shift which is especially bad on color pickers. Compare what correct color picker looks like: with the one that is rendered with sRGB frame buffer and manual conversion of colors to linear space: If you want to get results that are visually close to ImGui examples when using sRGB framebuffer, the following changes are needed:
With these changes to the shader, the color picker (and other UI elements) will look properly when rendered to sRGB framebuffer: |
@TheMostDiligent Fwiiw a quick fix is adding something like // Convert from sRGB to linear color space before the RS.
output.spectrum = SRgbaToLinear(input.m_spectrum);
// Premultiply with alpha before the RS.
output.spectrum.rgb *= output.spectrum.a; in your vertex shader. |
I do remember. However the question is what you are trying to achieve and what problem to solve. Imgui does all math in sRGB space which is mathematically incorrect at each step (barycentric interpolation, texture filtering, alpha blending), but which is also a given now. |
I've implemented SRGB colors for ImGui dark, light and classic style. code I've added basically looks like this
then in imgui_draw.cpp using dark theme as an example
This works with the default shader provided by ImGui, the only thing that'd probably require investigation is if all the hardcoded colors in the demo window are still correct |
Could you open a new issue for this? Could be a PR even, but i am curious if you can make this mostly computed. Please note that I am likely to leave pr dangling for a while until i tackle those issues in batch, but giving it pr visibility and having a reference is always useful. |
A pull request has been made :) |
Colors in linear spaces currently aren't acting linearly. For example, if you create an
ImGui::ColorButton
with (0.5,0.5,0.5), you get a pixel with exactly (128,128,128) as the RGB components showing up on your screen. This value of (128,128,128) is actually darker than mid-grey because display devices have a non-linear response to color. (see: http://docs.cryengine.com/pages/viewpage.action?pageId=1605651)If you render to an sRGB framebuffer, then the ColorButton will appear correctly as mid-grey. However, the rest of ImGui is too bright. This problem appears many times in the screenshot thread (#123), where different aspects of ImGui are either too dark or too bright depending on whether the renderer is writing to an sRGB framebuffer or not.
Compare:
In both screenshots there's an example green-blue-ish "clear color" slider. Despite having values generally over 128, it appears darker than a proper sRGB mid-grey (see below). This is because the sliders represent an already encoded sRGB color, meaning that settings the sliders to (128,128,128) will produce a color much darker than mid-grey.
For example, this also affects the color palette the in custom rendering example in the README (below). The exact middle of the palette is (128,128,0), which means that the exact middle of the palette is not actually a yellow shade of mid-grey, instead it appears much darker. The line between the red and green half should actually be yellow, rather than brown.
When manipulating RGBA8 colors, it makes sense for ImGui to treat them as already encoded in sRGB space, since allocating more bits to darker colors means allocating more bits to differences in color more noticeable to humans. However, when manipulating RGBA32F colors, there's no reason to use this compression. Instead, RGBA32F colors should be in linear space, since this is required for blending to be correct. Another confusing aspect is that the font textures are linear RGBA8 (I think?), so the multiplication of non-linear RGB (from eg ColorButton) with the fonts' linear RGB in the pixel shader is just plain wrong.
There's many ways to fix these problems. First off, we should be certain about which color space should be used where in ImGui. Secondly, I suggest that ImGui outputs linear colors in the end, with an optional post-processing step to convert everything to sRGB for renderers that expect everything to arrive already encoded in sRGB. Naturally, we should try to handle this in a way that doesn't break existing ImGui applications.
Good background info for this topic: https://www.youtube.com/watch?v=LKnqECcg6Gw
The text was updated successfully, but these errors were encountered: