cv2.normalize with NORM_MINMAX produces small values below set minimum on float32

[isenberg]

System Information

OpenCV python version: 4.10.0
Operating System / Platform: macOS 15.1.1 arm64 M2
Python version: 3.9.6 (/usr/bin/python3 provided by macOS)

Detailed description

The following code shows unexpected small negative numbers when trying to normalize an image between 0 and 1 while a comparison, which is mathematically not exactly the same, works fine. Note: The comparison img1 is keeping the offset from 0, I just used it for a quick check.

This appears to be the same as #6125 and #6170 which apparently has been fixed once in dev but then has been everted due to performance impacts.

This bug, if I don't have a misunderstanding of the OpenCV NORM_MINMAX definition, is serious as even small negative numbers can lead to critical follow up errors. For safety one can of course follow it with a img = np.clip(img, 0, 1) which is advised anyway for critical applications, but the result is mathematically slightly wrong.

If the performance impact can still be seen today with a fix, then I suggest to add at least internally a clip.

Datatype of img is np.float32. Changing to np.double like suggested in the older bug doesn't change anything.

print(cv2.__version__)
print(img.min(axis=(0, 1)), img.max(axis=(0, 1)))
img1 = cv2.normalize(img, None, 0, 1, norm_type=cv2.NORM_MINMAX)
img2 = img / max(img.max(axis=(0, 1)))
print(img1.min(axis=(0, 1)), img1.max(axis=(0, 1)))
print(img2.min(axis=(0, 1)), img2.max(axis=(0, 1)))

4.10.0
[0.02032561 0.04122998 0.04707103] [0.6832291  0.98649204 0.9428176 ]
[-1.0251444e-10  2.1636410e-02  2.7682003e-02] [0.68611723 0.99999994 0.95479614]
[0.02060392 0.04179454 0.04771557] [0.6925845 1.        0.9557276]

If needed I guess I would be able to attach a reproducer input image and code.

Steps to reproduce

On arm64 macOS:

#!/usr/bin/env python3
import cv2
import numpy as np
m = np.array([[ 1888, 1692, 369, 263, 199, 280, 326, 129, 143, 126, 233, 221, 130, 126, 150, 249, 575, 574, 63, 12]], dtype=np.float32)
print(cv2.__version__)
print(m.min(axis=(0, 1)), m.max(axis=(0, 1)))
mn = cv2.normalize(m, None, 0, 1, norm_type=cv2.NORM_MINMAX)
print(mn.min(axis=(0, 1)), mn.max(axis=(0, 1)))

Writes wrong result, small negative min:

% ~/python311/bin/python3 testnorm.py
4.10.0
12.0 1888.0
-2.3283064e-10 1.0

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues, forum.opencv.org, Stack Overflow, etc and have not found any solution
• I updated to the latest OpenCV version and the issue is still there
• There is reproducer code and related data files (videos, images, onnx, etc)

[AliHaroonT]

Any update regarding this? I'm also facing same issue.
Right now I'm manually normalizing the pixel values using the solution mentioned here (https://stackoverflow.com/questions/29100722/equivalent-im2double-function-in-opencv-python)

[denshade]

Not sure if useful, but this seems to work fine on Windows.
The above code yields the following on Windows:
4.10.0
[190 145 161] [249 244 228]
[0 0 0] [1 1 1]
[0.76305221 0.58232932 0.64658635] [1. 0.97991968 0.91566265]

[isenberg]

Still produces the wrong results on arm64 macOS when using float 32.

Reproducer:

#!/usr/bin/env python3
import cv2
import numpy as np
m = np.array([[ 1888, 1692, 369, 263, 199, 280, 326, 129, 143, 126, 233, 221, 130, 126, 150, 249, 575, 574, 63, 12]], dtype=np.float32)
print(cv2.__version__)
print(m.min(axis=(0, 1)), m.max(axis=(0, 1)))
mn = cv2.normalize(m, None, 0, 1, norm_type=cv2.NORM_MINMAX)
print(mn.min(axis=(0, 1)), mn.max(axis=(0, 1)))

Result is small negative which is wrong:

% ~/python311/bin/python3 testnorm.py
4.10.0
12.0 1888.0
-2.3283064e-10 1.0

[denshade]

Confirmed reproducer triggers a problem on Windows as well.

[denshade]

I was able to reproduce it in the C++ code

TEST(Normalize, regression_5877_inplace_change_type)
{
    std::vector<float> initial_values = { 1888, 1692, 369, 263, 280, 326, 129, 143, 126, 233, 221, 130, 126, 150, 249, 12 };
    Mat m(Size(16, 1), CV_32F, initial_values.data());
    normalize(m, m, 0, 1, NORM_MINMAX);
    float value = m.at<float>(0, 15);
    ASSERT_EQ(0, value);
}

test_arithm.cpp(2228): error: Expected equality of these values:
0
value
Which is: -2.32831e-10

[denshade]

I've traced the problem to the following line
opencv\modules\core\src\norm.cpp:1388. It's obviously a rounding error.

        if( rtype == CV_32F )
        {
            scale = (float)scale; //<---
            shift = (float)dmin - (float)(smin*scale);
        }

Changing the code to this:

        if( rtype == CV_32F )
        {
            scale = scale; //<--- Or you can just omit this line.
            shift = (float)dmin - (float)(smin*scale);
        }

Not rounding this yields
2.32831e-10 Which is > 0.

[isenberg]

The line scale = (float)scale indeed doesn't look like making sense as smin is a double and in the next line the result is rounded anyway.

[denshade]

Added a small PR for this:
#26658 Feedback welcome.

[denshade]

I've created some code to find the smallest fit within the boundaries.

It appears tests are failing because it expects:

    TEST_CYCLE() cv::normalize(src, dst, 20., 100., NORM_MINMAX);
    SANITY_CHECK(dst, 1e-6, ERROR_RELATIVE);

expect_max evaluates to 100,
actual_max evaluates to 99.997268676757812, and
eps evaluates to 9.9999999999999991e-05.

If I'm not mistaking this means it allows to be outside of the interval. I hope an active contributor with more context can pitch in and decide what is most important:
a. the normalized results must fall within the interval
b. the normalized results may fall outside of the interval, in such a way that it is minimized.

[isenberg]

That's too much change now and could cause the performance problems again the first try years ago caused which lead to being it reverted.

What about just clamping the result to the min value if it exceeds it? I.e. just adding something like this after shift = dmin - smin*scale;

if (shift < dmin)
shift = dmin

[isenberg]

Ah I see the problem, the actual calculation is done in convertTo and not in the function where the current changes apply. Then the performance change risk is low.

[kallaballa]

I'm just a contributor but given the central nature of the function and that the tests suggests this is know: Avoid breaking changes by introducing a new mode? e.g. NORM_MINMAX_CLAMP or such? It isn't strictly a beautiful solution but it avoids red lamps (that expect a certain behaviour) from lighting up over the planet :)

[isenberg]

I'm just a contributor but given the central nature of the function and that the tests suggests this is know:
Avoid breaking changes by introducing a new mode?

If you are a code contributor to opencv: Do you have a practical example where the current wrong (by the function documentation) behavior is expected or required? I couldn't think of any except for intentionally having those bugs to train people to be aware that third party code can have bugs which skipped years of usage.

[kallaballa]

Any code that is using it. As you can see that behavior seems to be known (see the tests) and changes to such a central function are delicate. This is probably not the answer you seek but easily thousands of projects depend on it and might break or at least change behavior unexpectedly.
I am not an official and I tried to introduce breaking changes as well, that's why I try to assist you. I think putting the code in a separate code path might have a chance, but I am not sure about that either.

Anyway it's a good catch and by creating this issue alone you are helping lots of people!

[isenberg]

I haven't checked the existing test code yet, but if the test code "knows" about the bug, that would be really interesting. I'll look into it later and if I find a hint let's add add it here.

[isenberg]

Btw, I found it as I process images in float32 and there the small negative number inverts the colorchannel which makes it immediately visible.

[kallaballa]

I've created some code to find the smallest fit within the boundaries.

It appears tests are failing because it expects:

    TEST_CYCLE() cv::normalize(src, dst, 20., 100., NORM_MINMAX);
    SANITY_CHECK(dst, 1e-6, ERROR_RELATIVE);

I ment this.

[isenberg]

I don't see a hint that this test code expected to see numbers outside of min-max range. I only see some fuzziness expected in the results for the many different norms tested. If someone uses the norm min-max in code they expect that no results appear outside of that range because that's one of the main purposes of using this norm.

Also: In the previous two bug reports about this I linked to in my initial posting, all discussions there appear to to accept that this bug needs to be fixed, just the solution back then was not optimal regarding performance.

[kallaballa]

ic :)