Additional color transfers #81

kodawah · 2017-09-18T22:17:23Z

Hi,
I've added a few color transfer characteristics used in some old files.
Please let me know if you would like me to change anything. In particular I'd like to know your opinion on how I implemented the log transfers.
Thanks

sekrit-twc

xvYCC requires more thought than merely adding some equations. The obvious issue is that it produces linear RGB values outside of [0.0,1.0], which is not handled correctly by the LUT implementations. There is also the semantic issue of what it means to convert (RGB, xvYCC, 709) to (RGB, 709, 709), given that xvYCC uses the same non-linearity as Rec.709.

In addition, there is the issue of scene vs. display-referred grading at play. The question being whether xvYCC should be handled by inverting a symmetrical BT.1886 (pure-power) function, or if the reflected Rec.709 OETF should be used in all cases.

In summary, I can not merge this commit without test data and industry best-practice guidelines. Since xvYCC is a defunct SONY initiative, such information may be difficult to obtain.

sekrit-twc · 2017-09-19T02:13:06Z

src/zimg/colorspace/gamma.cpp

+		x = SMPTE_240M_ALPHA * zimg_x_powf(x, 0.45f) - (SMPTE_240M_ALPHA - 1.0f);
+
+	return x;
+}


SMPTE 240M, like all other legacy gamma functions, is an OETF. The EOTF should be BT.1886.

oh good point, should I just reflect that in the function name or is there something else I am missing?

The function name should be updated, and in select_transfer_function, the OETF should only be returned if scene_referred is true. See the other gamma functions.

Attached.

s240m.pdf

sekrit-twc · 2017-09-19T02:15:35Z

I can not review the log or gamma 2.2/2.8 commits without real test data or best-practice guidelines.

sekrit-twc · 2017-09-19T02:22:12Z

src/zimg/colorspace/gamma.cpp

@@ -140,6 +140,26 @@ float smpte_240m_inverse_eotf(float x) noexcept
 	return x;
 }

+float log10_eotf(float x) noexcept
+{
+	if (x < 0.01f)


Why is the threshold 0.01f? Similarly, why does 0 map to 0.01f in the inverse? That causes non-monotonic behaviour at the next greatest value.

It is defined as such in the specifications. This is for log10 but log316 is similar

V=1.0+Log10(Lc)÷2 for 1 >= Lc >= 0.01 V=0.0 for 0.01>Lc >= 0

I mapped 0 -> 0.01 in the inverse because I thought it was the right thing to do, but I was unsure as well.

kodawah · 2017-09-19T10:04:21Z

thanks for the review, I should have some samples I can share for most of the transfers.
I wasn't aware of the added complexity of xvYCC, however I know it's unfortunately still used by some screen recording software (which record the screen transfer afaict).

I fixed the build while I prepare the samples in the meantime

sekrit-twc · 2017-09-20T03:26:58Z

Can you provide a reference for "screen recording software" that uses xvYCC? I know FRAPS and such record a full-range YUV signal, but that is not the same as xvYCC. xvYCC specifically refers to the SONY method of increasing the gamut by using the 0-15 and 236-255 bands to encode negative and super-positive colors.

sekrit-twc · 2017-09-29T16:25:53Z

@kodabb Were you able to find the test content?

kodawah · 2017-09-29T17:04:37Z

hi sorry, got busy while sorting them out, I'll be able to provide them within next week

kodawah · 2017-10-03T21:25:07Z

Hi, here are finally some samples, the filename is matrix-primaries-transfer order

all these vids are public domain

A few references

for log100 and log316 code Support log100 and log316 transfer functions mpv-player/mpv#4897
for xvycc I have some slides from sony https://hopa.memberclicks.net/assets/documents/2007_xvYCC_forHPA_FINAL.pdf
for bt470 m and bg https://www.itu.int/rec/R-REC-BT.1700-0-200502-I/en

kodawah · 2017-10-03T22:02:18Z

updated code to fix function naming and oetf selection

sekrit-twc · 2017-10-04T00:25:18Z

src/zimg/colorspace/gamma.cpp

@@ -250,6 +260,10 @@ TransferFunction select_transfer_function(TransferCharacteristics transfer, doub
 	func.to_gamma_scale = 1.0f;

 	switch (transfer) {
+	case TransferCharacteristics::LOG_100:
+		func.to_linear = scene_referred ? log100_inverse_oetf : rec_1886_eotf;


I don't think Rec.1886 is correct for LOG-encoded images.

hm you're right, the colours do not appear correctly
do you think the log transfers could be describing EOTFs instead?

Could be that there is no OETF/EOTF distinction for log.

sekrit-twc · 2017-10-04T00:30:07Z

src/zimg/colorspace/gamma.cpp

@@ -134,6 +137,26 @@ float srgb_inverse_eotf(float x) noexcept
 	return x;
 }

+float xvycc_eotf(float x) noexcept
+{


xvYCC should be implemented as a mirrored Rec.709 OETF. In other words, copysign(rec709_oetf(fabs(x)), x).

It is also still unclear if the standard gamut portion of xvYCC should be handled with Rec.1886, which is what would occur if the signal were to reach a non-xv TV.

I tried using rec 709 but the colors when not display correctly with quicktime. I had to use the pure power bt1886 instead.

Regarding the last point we could maybe introduce a new range type? This is what the specs say on that regard

Otherwise (TransferCharacteristics is equal to 11 (IEC 61966-2-4) or 12 (Rec. ITU-R BT.1361 extended colour gamut system)), E′R, E′G and E′B are analogue with a larger range not specified in this part of ISO/IEC 23001.

or maybe let the applications deal with it?

Can you elaborate on how you used QuickTime to test this? The problem with using BT.1886 instead of Rec.709 outside of the [0-1] range, i.e. the extended gamut, is that it affects the gamut width. For example, 1.2^2.2 (BT.709) is less saturated than 1.2^2.4 (BT.1886). The behavior needs to be compared to a reference xvYCC implementation.

I converted a video containing xvycc with zimg and made sure colors matches the ones of the original video, can post pics if needed.

sekrit-twc · 2017-10-04T01:24:55Z

src/zimg/colorspace/gamma.cpp

@@ -120,6 +120,16 @@ float rec_1886_inverse_eotf(float x) noexcept
 	return x < 0.0f ? 0.0f : zimg_x_powf(x, 1.0f / 2.4f);
 }

+float log100_oetf(float x) noexcept
+{
+	return x < 0.01f ? 0 : 1.0f + zimg_x_logf(x) / 2.0f;


Isn't logf the base-e logarithm? Based on the threshold of 0.01, it seems that the base-10 log10f needs to be used.

hmm logf should be natural log, you're right log10 should be used, thanks for spotting

kodawah · 2017-10-04T23:35:34Z

i'm getting the feeling that gamma22 and gamma28 are eotf only, i have wrong colors with the scene_referred thing

sekrit-twc · 2017-10-05T02:13:31Z

How do you determine if something has the right or wrong colors?

kodawah · 2017-10-05T21:11:51Z

How do you determine if something has the right or wrong colors?

@sekrit-twc I tried to replicate conversion with different tools, ie vf_colorspace in ffmpeg or mpv, and visually check that the colors were the same.

kodawah · 2017-10-05T21:19:40Z

btw any suggestions for this error?

src/zimg/colorspace/gamma.cpp: In function ‘float zimg::colorspace::log100_oetf(float)’:
src/zimg/colorspace/gamma.cpp:122:33: error: ‘log10f’ is not a member of ‘std’

it seems the solution is to include the <cmath> header, but it's already there...

sekrit-twc · 2017-10-06T09:05:00Z

But how do you know the conversion in ffmpeg and mpv are correct? Also, log10f is in the global namespace. In std:: there is only "log10", which is overloaded.

kodawah · 2017-10-08T05:41:21Z

wrt mpv I trust the guy who wrote the rendering engine, wrt ffmpeg after I added the conversion parameters I stopped getting complaints from users about incorrect color support

kodawah · 2017-10-09T16:42:59Z

build fixed

kodawah · 2017-10-16T18:36:00Z

@sekrit-twc hi, do you have any advice on how I can move this set of PRs forward? Thank you

sekrit-twc · 2017-10-16T19:41:24Z

@kodabb,

I need to find the time to review the transfer characteristics changes. I can merge the other PRs without further action.

PLEASE NOTE: You must agree to either assign copyright of code to the project owner or license contributions under WTFPL (see COPYING).

kodawah · 2017-10-19T16:15:14Z

I agree to license my contributions under the WTFPL terms.

thanks for all the help and time spent on these PRs

Kodabb additional color transfers Fix #81.

kodawah · 2017-10-21T00:40:24Z

🎉

See #81

See #81 (cherry picked from commit a1a4fb5)

sekrit-twc · 2021-08-07T14:38:34Z

The commit was reversed following a customer escalation. I have created #155 for further discussion regarding this topic.

See sekrit-twc#81

sekrit-twc reviewed Sep 19, 2017

View reviewed changes

kodawah force-pushed the additional_color_transfers branch from 94bf39f to e44523d Compare September 19, 2017 09:58

kodawah force-pushed the additional_color_transfers branch from e44523d to 782bfe5 Compare September 19, 2017 14:40

kodawah force-pushed the additional_color_transfers branch from 782bfe5 to 7cc84f2 Compare October 3, 2017 22:01

sekrit-twc reviewed Oct 4, 2017

View reviewed changes

kodawah added 2 commits October 4, 2017 15:19

colorspace: Add support for xvycc/iec61966-2-4 transfer characteristics

a3b4375

colorspace: Add support for smpte240m transfer characteristics

97917d8

kodawah force-pushed the additional_color_transfers branch from 7cc84f2 to ff8e35b Compare October 4, 2017 23:36

kodawah added 4 commits October 9, 2017 11:31

colorspace: Add support for log100 transfer characteristics

d0a79b4

colorspace: Add support for log316 transfer characteristics

4d4eb54

colorspace: Add support for bt470bg/gamma28 transfer characteristics

662c8d6

colorspace: Add support for bt470m/gamma22 transfer characteristics

bfad217

kodawah force-pushed the additional_color_transfers branch from ff8e35b to ef1f49f Compare October 9, 2017 15:32

kodawah force-pushed the additional_color_transfers branch from ef1f49f to bfad217 Compare October 16, 2017 18:35

sekrit-twc force-pushed the master branch from 61305e8 to ecceaa7 Compare October 19, 2017 16:22

sekrit-twc merged commit bfad217 into sekrit-twc:master Oct 20, 2017

sekrit-twc added a commit that referenced this pull request Oct 20, 2017

Merge pull request #85 from sekrit-twc/kodabb-additional_color_transfers

ae9a278

Kodabb additional color transfers Fix #81.

sekrit-twc pushed a commit that referenced this pull request May 9, 2021

colorspace: use correct EOTF for SMPTE-240M

a1a4fb5

See #81

sekrit-twc pushed a commit that referenced this pull request Jul 6, 2021

colorspace: use correct EOTF for SMPTE-240M

865175b

See #81 (cherry picked from commit a1a4fb5)

Ichunjo pushed a commit to Varde-s-Forks/zimg that referenced this pull request Jan 4, 2022

colorspace: use correct EOTF for SMPTE-240M

4469d35

See sekrit-twc#81

Additional color transfers #81

Additional color transfers #81

Conversation

kodawah commented Sep 18, 2017

sekrit-twc left a comment

Choose a reason for hiding this comment

sekrit-twc Sep 19, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sekrit-twc commented Sep 19, 2017

Choose a reason for hiding this comment

kodawah Sep 19, 2017 • edited Loading

Choose a reason for hiding this comment

kodawah commented Sep 19, 2017

sekrit-twc commented Sep 20, 2017

sekrit-twc commented Sep 29, 2017

kodawah commented Sep 29, 2017

kodawah commented Oct 3, 2017 • edited Loading

kodawah commented Oct 3, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sekrit-twc Oct 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sekrit-twc Oct 4, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kodawah commented Oct 4, 2017

sekrit-twc commented Oct 5, 2017

kodawah commented Oct 5, 2017

kodawah commented Oct 5, 2017 • edited Loading

sekrit-twc commented Oct 6, 2017

kodawah commented Oct 8, 2017

kodawah commented Oct 9, 2017

kodawah commented Oct 16, 2017

sekrit-twc commented Oct 16, 2017

kodawah commented Oct 19, 2017

kodawah commented Oct 21, 2017

sekrit-twc commented Aug 7, 2021

sekrit-twc Sep 19, 2017 •

edited

Loading

kodawah Sep 19, 2017 •

edited

Loading

kodawah commented Oct 3, 2017 •

edited

Loading

sekrit-twc Oct 5, 2017 •

edited

Loading

sekrit-twc Oct 4, 2017 •

edited

Loading

kodawah commented Oct 5, 2017 •

edited

Loading