Integer-overflow in function opj_dwt_decode_partial_1_parallel()

[skorpion98]

Summary

We found an integer-overflow in function opj_dwt_decode_partial_1_parallel() from dwt.c after testing one of the harnesses provided on the OSS-Fuzz repository (opj_decompress_fuzzer_J2K).

openjpeg/src/lib/openjp2/dwt.c

Line 2404 in 606304d

OPJ_S_off(i, off) -= (OPJ_D__off(i - 1, off) + OPJ_D__off(i, off) + 2) >> 2;

Expected behavior and actual behavior.

The program performs a series of mathematical operations and bitwise shifts before storing the result in a variable of type “OPJ_INT32”, but there are no checks related to possible overflows/underflows of such operations.

It’s important to notice that the bug is not consistent, i.e. it is not always triggered given the same input, and we think that this may be due to offset approximations as the bug is triggered by the operation OPJ_D__off(i,off).
We cannot provide more info on which is the cause for this error, but we are confident that developers with knowledge on the codebase will be able to find the root of the issue with little effort and remain available to assist you in case of need.

Steps to reproduce the problem.

In the attached archive you will find:

• the executable on which we performed our tests
• the input file that caused the bug
• the outputs of UBSan confirming our finding

To reproduce the error, simply run the binary against the provided input, using a command as ./opj_decompress_fuzzer_J2K /path_to_testcases/input.

The program has been tested on the standard Docker image provided on OSS-Fuzz using Ubuntu 20.04, providing AFL++ as fuzzing engine and build flag --sanitizer=undefined.

Operating system

Ubuntu 20.04 (x86_64)

openjpeg version

Commit hash 606304d