Invalid memory address dereference in hf_assembly (in libfaad/sbr_hfadj.c:1536)

[fantasy7082]

Hi, i found a issue in Freeware Advanced Audio Decoder 2 (FAAD2) 2.8.8. It crashed in function hf_assembly .the details are below(ASAN):

./faad faad_res/009-invalid-def-sbr_hfadj_1536 -o out.wav
 *********** Ahead Software MPEG-4 AAC Decoder V2.8.8 ******************

 Build: Dec 13 2018
 Copyright 2002-2004: Ahead Software AG
 http://www.audiocoding.com
 bug tracking: https://sourceforge.net/p/faac/bugs/
 Floating point version

 This program is free software; you can redistribute it and/or modify
 it under the terms of the GNU General Public License.

 **************************************************************************

faad_res/009-invalid-def-sbr_hfadj_1536 file info:
ADTS, 0.299 sec, 42 kbps, 48000 Hz

  ---------------------
 | Config:  2 Ch       |
  ---------------------
 | Ch |    Position    |
  ---------------------
 | 00 | Left front     |
 | 01 | Right front    |
  ---------------------

ASAN:SIGSEGVfaad_res/009-invalid-def-sbr_hfadj_1536.
=================================================================
==7089==ERROR: AddressSanitizer: SEGV on unknown address 0x0000000030da (pc 0x7f2a44f66a79 bp 0x7ffc06965b70 sp 0x7ffc06965ab0 T0)
    #0 0x7f2a44f66a78 in hf_assembly /root/faad2_asan/libfaad/sbr_hfadj.c:1536
    #1 0x7f2a44f633b6 in hf_adjustment /root/faad2_asan/libfaad/sbr_hfadj.c:90
    #2 0x7f2a44f81725 in sbr_process_channel /root/faad2_asan/libfaad/sbr_dec.c:363
    #3 0x7f2a44f837fa in sbrDecodeSingleFramePS /root/faad2_asan/libfaad/sbr_dec.c:637
    #4 0x7f2a44f2bb54 in reconstruct_single_channel /root/faad2_asan/libfaad/specrec.c:1071
    #5 0x7f2a44f33e28 in single_lfe_channel_element /root/faad2_asan/libfaad/syntax.c:631
    #6 0x7f2a44f32354 in decode_sce_lfe /root/faad2_asan/libfaad/syntax.c:351
    #7 0x7f2a44f332da in raw_data_block /root/faad2_asan/libfaad/syntax.c:441
    #8 0x7f2a44eed9c3 in aac_frame_decode /root/faad2_asan/libfaad/decoder.c:990
    #9 0x7f2a44eed566 in NeAACDecDecode /root/faad2_asan/libfaad/decoder.c:821
    #10 0x40f8ae in decodeAACfile /root/faad2_asan/frontend/main.c:679
    #11 0x411dd4 in faad_main /root/faad2_asan/frontend/main.c:1323
    #12 0x411fe5 in main /root/faad2_asan/frontend/main.c:1366
    #13 0x7f2a44b2582f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #14 0x401aa8 in _start (/usr/local/faad-asan/bin/faad+0x401aa8)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /root/faad2_asan/libfaad/sbr_hfadj.c:1536 hf_assembly
==7089==ABORTING

POC FILE:https://github.com/fantasy7082/image_test/blob/master/009-invalid-def-sbr_hfadj_1536

[hlef]

Unreproducible on the current master.

Fixed by 6b4a7cd.

This can be confirmed by doing some quick debugging:

Before 6b4a7cd, on my system:

$ ASAN_OPTIONS=abort_on_error=1 libtool --mode=execute
 gdb --args ./frontend/faad ../009-invalid-def-sbr_hfadj_1536 -o /tmp/out.wav
GNU gdb (Debian 8.2.1-2) 8.2.1
[...]
Reading symbols from /home/hle/Development/C/faad2/faad2/frontend/.libs/faad...done.
(gdb) r
Starting program: /home/hle/Development/C/faad2/faad2/frontend/.libs/faad ../009-invalid-def-sbr_hfadj_1536
 -o /tmp/out.wav
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
 *********** Ahead Software MPEG-4 AAC Decoder V2.8.8 ******************

 Build: Aug 10 2019
[...]

 **************************************************************************

../009-invalid-def-sbr_hfadj_1536 file info:
ADTS, 0.299 sec, 42 kbps, 48000 Hz

  ---------------------
 | Config:  2 Ch       |
  ---------------------
 | Ch |    Position    |
  ---------------------
 | 00 | Left front     |
 | 01 | Right front    |
  ---------------------

=================================================================
==4990==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7ffffffeccb0 at pc 0x7ffff71973cd bp 0
x7ffffffec8c0 sp 0x7ffffffec8b8
WRITE of size 4 at 0x7ffffffeccb0 thread T0
    #0 0x7ffff71973cc in calculate_gain /home/hle/Development/C/faad2/faad2/libfaad/sbr_hfadj.c:1311
    #1 0x7ffff7194930 in hf_adjustment /home/hle/Development/C/faad2/faad2/libfaad/sbr_hfadj.c:83
    #2 0x7ffff71b3b28 in sbr_process_channel /home/hle/Development/C/faad2/faad2/libfaad/sbr_dec.c:363
    #3 0x7ffff71b5c60 in sbrDecodeSingleFramePS /home/hle/Development/C/faad2/faad2/libfaad/sbr_dec.c:637
    #4 0x7ffff715bb72 in reconstruct_single_channel /home/hle/Development/C/faad2/faad2/libfaad/specrec.c:1
071
    #5 0x7ffff716409e in single_lfe_channel_element /home/hle/Development/C/faad2/faad2/libfaad/syntax.c:63
1
    #6 0x7ffff71623f3 in decode_sce_lfe /home/hle/Development/C/faad2/faad2/libfaad/syntax.c:351
    #7 0x7ffff71633e2 in raw_data_block /home/hle/Development/C/faad2/faad2/libfaad/syntax.c:441
    #8 0x7ffff711c4f5 in aac_frame_decode /home/hle/Development/C/faad2/faad2/libfaad/decoder.c:990
    #9 0x7ffff711bfbb in NeAACDecDecode /home/hle/Development/C/faad2/faad2/libfaad/decoder.c:821
    #10 0x55555556e07f in decodeAACfile /home/hle/Development/C/faad2/faad2/frontend/main.c:679
    #11 0x55555557141a in faad_main /home/hle/Development/C/faad2/faad2/frontend/main.c:1323
    #12 0x555555571640 in main /home/hle/Development/C/faad2/faad2/frontend/main.c:1366
    #13 0x7ffff6f6009a in __libc_start_main ../csu/libc-start.c:308
    #14 0x55555555f409 in _start (/home/hle/Development/C/faad2/faad2/frontend/.libs/faad+0xb409)

Address 0x7ffffffeccb0 is located in stack of thread T0 at offset 0 in frame
    #0 0x7ffff71942e1 in hf_adjustment /home/hle/Development/C/faad2/faad2/libfaad/sbr_hfadj.c:60

  This frame has 1 object(s):
    [32, 2972) 'adj'
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-underflow /home/hle/Development/C/faad2/faad2/libfaad/sbr_hfadj.c:1
311 in calculate_gain
[...]
==4990==ABORTING

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff6f5e535 in __GI_abort () at abort.c:79
#2  0x00007ffff72eae6b in ?? () from /lib/x86_64-linux-gnu/libasan.so.5
#3  0x00007ffff72f2ed8 in ?? () from /lib/x86_64-linux-gnu/libasan.so.5
#4  0x00007ffff72d797d in ?? () from /lib/x86_64-linux-gnu/libasan.so.5
#5  0x00007ffff72d854b in __asan_report_store4 () from /lib/x86_64-linux-gnu/libasan.so.5
#6  0x00007ffff71973cd in calculate_gain (sbr=0x62f000000400, adj=0x7ffffffeccd0, ch=0 '\000')
    at sbr_hfadj.c:1311
#7  0x00007ffff7194931 in hf_adjustment (sbr=0x62f000000400, Xsbr=0x62f000003508, ch=0 '\000')
    at sbr_hfadj.c:83
#8  0x00007ffff71b3b29 in sbr_process_channel (sbr=0x62f000000400, channel_buf=0x625000000100,
    X=0x7ffffffed940, ch=0 '\000', dont_process=0 '\000', downSampledSBR=0 '\000') at sbr_dec.c:363
#9  0x00007ffff71b5c61 in sbrDecodeSingleFramePS (sbr=0x62f000000400, left_channel=0x625000000100,
    right_channel=0x625000002900, just_seeked=0 '\000', downSampledSBR=0 '\000') at sbr_dec.c:637
#10 0x00007ffff715bb73 in reconstruct_single_channel (hDecoder=0x620000000080, ics=0x7fffffff8ae6,
    sce=0x7fffffff8ae0, spec_data=0x7fffffff82c0) at specrec.c:1071
#11 0x00007ffff716409f in single_lfe_channel_element (hDecoder=0x620000000080, ld=0x7fffffffdeb0,
    channel=0 '\000', tag=0x7fffffffdd40 "") at syntax.c:631
#12 0x00007ffff71623f4 in decode_sce_lfe (hDecoder=0x620000000080, hInfo=0x7fffffffe1e0,
    ld=0x7fffffffdeb0, id_syn_ele=0 '\000') at syntax.c:351
#13 0x00007ffff71633e3 in raw_data_block (hDecoder=0x620000000080, hInfo=0x7fffffffe1e0,
    ld=0x7fffffffdeb0, pce=0x620000000bd1, drc=0x610000000040) at syntax.c:441
#14 0x00007ffff711c4f6 in aac_frame_decode (hDecoder=0x620000000080, hInfo=0x7fffffffe1e0,
    buffer=0x621000002900 "\377\361X@\033\337\374\001X4,\264u\034\tHȀ\260\220\302%N\001\342\241\023b\340D\2
41\002\201k\356\a", buffer_size=4608, sample_buffer2=0x0, sample_buffer_size=0) at decoder.c:990
#15 0x00007ffff711bfbc in NeAACDecDecode (hpDecoder=0x620000000080, hInfo=0x7fffffffe1e0,
    buffer=0x621000002900 "\377\361X@\033\337\374\001X4,\264u\034\tHȀ\260\220\302%N\001\342\241\023b\340D\2
41\002\201k\356\a", buffer_size=4608) at decoder.c:821
#16 0x000055555556e080 in decodeAACfile (aacfile=0x604000000010 "../009-invalid-def-sbr_hfadj_1536",
    sndfile=0x602000000010 "/tmp/out.wav", adts_fn=0x0, to_stdout=0, def_srate=0, object_type=2,
    outputFormat=1, fileType=1, downMatrix=0, infoOnly=0, adts_out=0, old_format=0,
    song_length=0x7fffffffe540) at main.c:679
#17 0x000055555557141b in faad_main (argc=4, argv=0x7fffffffe8b8) at main.c:1323
#18 0x0000555555571641 in main (argc=4, argv=0x7fffffffe8b8) at main.c:1366
(gdb) up 6
#6  0x00007ffff71973cd in calculate_gain (sbr=0x62f000000400, adj=0x7ffffffeccd0, ch=0 '\000')
    at sbr_hfadj.c:1311
1311                        Q_M_lim[m] = Q_M;
(gdb) l
1306
1307                    /* limit the additional noise energy level */
1308                    /* and apply the limiter */
1309                    if (G_max > G)
1310                    {
1311                        Q_M_lim[m] = Q_M;
1312                        G_lim[m] = G;
1313                    } else {
1314                        Q_M_lim[m] = Q_M * G_max / G;
1315                        G_lim[m] = G_max;
(gdb) p m
$1 = 216 '\330'

@fabiangreffrath it would be great if you could double check, but I think this can be safely closed.

[hlef]

FTR, this was assigned CVE-2018-20361.

[fabiangreffrath]

Closing this issue.