SIGABRT: double free or corruption (out) #562
Comments
|
I tested every single directory in |
|
Build with |
GNU gdb (GDB; SUSE Linux Enterprise 15) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.opensuse.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from rmlint...
Starting program: /usr/local/src/rmlint/rmlint -vvv /mnt/temp/jd2/download -g -T all,-badids --xattr-read --xattr-write --write-unfinished
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff10f6700 (LWP 23650)]
[New Thread 0x7ffff08f5700 (LWP 23651)]
▕░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░Thread 0x7ffff08f5700 (LWP 23651) exited] Traversing (13000 usable files / 0 + 0 ignored files / folders)
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Traversing (13030 usable files / 0 + 0 ignored files / folders)
▕▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒New Thread 0x7ffff08f5700 (LWP 24041)]
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Preprocessing (reduces files to 11145 / found 0 other lint)
▕▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░New Thread 0x7ffff00f4700 (LWP 24046)]
[New Thread 0x7fffef8f3700 (LWP 24051)]
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Matching (3076 dupes of 2515 originals; 0 B to scan in 0 files, ETA: ...)
▕▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓Thread 0x7ffff00f4700 (LWP 24046) exited]
[Thread 0x7ffff08f5700 (LWP 24041) exited]
[Thread 0x7ffff10f6700 (LWP 23650) exited]
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Merging files into directories (stand by...)
==> In total 13030 files, whereof 3076 are duplicates in 2515 groups.
==> This equals 853.19 GB of duplicates which could be removed.
==> Scanning took in total 8.099s.
Wrote a json file to: /usr/local/src/rmlint/rmlint.json
Wrote a sh file to: /usr/local/src/rmlint/rmlint.sh
=================================================================
==23589==ERROR: AddressSanitizer: attempting double-free on 0x60d00007c2d0 in thread T0:
#0 0x7ffff6efd1a8 in __interceptor_free (/usr/lib64/libasan.so.4+0xdc1a8)
#1 0x7ffff60af3b8 in g_free (/usr/lib64/libglib-2.0.so.0+0x583b8)
#2 0x45e115 in rm_file_destroy lib/file.c:141
#3 0x41b92c in rm_fmt_group_destroy lib/formats.c:52
#4 0x41cf04 in rm_fmt_close lib/formats.c:438
#5 0x42bbed in rm_session_clear lib/session.c:131
#6 0x408bd2 in main src/rmlint.c:146
#7 0x7ffff552e34c in __libc_start_main (/lib64/libc.so.6+0x2534c)
#8 0x4084c9 in _start (/usr/local/src/rmlint/rmlint+0x4084c9)
0x60d00007c2d0 is located 0 bytes inside of 129-byte region [0x60d00007c2d0,0x60d00007c351)
freed by thread T0 here:
#0 0x7ffff6efd1a8 in __interceptor_free (/usr/lib64/libasan.so.4+0xdc1a8)
#1 0x7ffff60af3b8 in g_free (/usr/lib64/libglib-2.0.so.0+0x583b8)
previously allocated by thread T2 (pool-rmlint) here:
#0 0x7ffff6efd500 in malloc (/usr/lib64/libasan.so.4+0xdc500)
#1 0x7ffff60af2b8 in g_malloc (/usr/lib64/libglib-2.0.so.0+0x582b8)
Thread T2 (pool-rmlint) created by T0 here:
#0 0x7ffff6e5ac80 in pthread_create (/usr/lib64/libasan.so.4+0x39c80)
#1 0x7ffff60f500b (/usr/lib64/libglib-2.0.so.0+0x9e00b)
SUMMARY: AddressSanitizer: double-free (/usr/lib64/libasan.so.4+0xdc1a8) in __interceptor_free
==23589==ABORTING
[Thread 0x7fffef8f3700 (LWP 24051) exited]
[Inferior 1 (process 23589) exited with code 01]
No stack.
Missing separate debuginfos, use: zypper install libasan4-debuginfo-7.5.0+r278197-4.25.1.x86_64 libgcc_s1-debuginfo-10.3.0+git1587-1.6.4.x86_64 libstdc++6-debuginfo-10.3.0+git1587-1.6.4.x86_64
(gdb) |
|
That's helpful, but the backtraces are incomplete. If you change the build command to |
|
It does change a bit. One entry has a bit more information. GNU gdb (GDB; SUSE Linux Enterprise 15) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-suse-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://bugs.opensuse.org/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from rmlint...
Starting program: /usr/local/src/rmlint/rmlint -vvv /mnt/temp/jd2/download -g -T all,-badids --xattr-read --xattr-write --write-unfinished
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff10f6700 (LWP 28886)]
[New Thread 0x7ffff08f5700 (LWP 28887)]
▕░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░Thread 0x7ffff10f6700 (LWP 28886) exited]░▒░░▒░░▏ Traversing (13003 usable files / 0 + 0 ignored files / folders)
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Traversing (13030 usable files / 0 + 0 ignored files / folders)
▕░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░▒░░New Thread 0x7ffff10f6700 (LWP 29636)]
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Preprocessing (reduces files to 11282 / found 0 other lint)
▕▒░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░New Thread 0x7ffff00f4700 (LWP 29637)]
[New Thread 0x7fffef8f3700 (LWP 29639)]
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Matching (3076 dupes of 2515 originals; 0 B to scan in 0 files, ETA: ...)
▕▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓Thread 0x7fffef8f3700 (LWP 29639) exited]
[Thread 0x7ffff10f6700 (LWP 29636) exited]
[Thread 0x7ffff08f5700 (LWP 28887) exited]
▕░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▏ Merging files into directories (stand by...)
==> In total 13030 files, whereof 3076 are duplicates in 2515 groups.
==> This equals 853.19 GB of duplicates which could be removed.
==> Scanning took in total 8.905s.
Wrote a json file to: /usr/local/src/rmlint/rmlint.json
Wrote a sh file to: /usr/local/src/rmlint/rmlint.sh
=================================================================
==28750==ERROR: AddressSanitizer: attempting double-free on 0x60d00007c2d0 in thread T0:
#0 0x7ffff6efd1a8 in __interceptor_free (/usr/lib64/libasan.so.4+0xdc1a8)
#1 0x7ffff60af3b8 in g_free (/usr/lib64/libglib-2.0.so.0+0x583b8)
#2 0x4607c2 in rm_file_destroy lib/file.c:141
#3 0x41bf06 in rm_fmt_group_destroy lib/formats.c:52
#4 0x41d55a in rm_fmt_close lib/formats.c:438
#5 0x42c865 in rm_session_clear lib/session.c:131
#6 0x408bd6 in main src/rmlint.c:146
#7 0x7ffff552e34c in __libc_start_main (/lib64/libc.so.6+0x2534c)
#8 0x4084c9 in _start (/usr/local/src/rmlint/rmlint+0x4084c9)
0x60d00007c2d0 is located 0 bytes inside of 129-byte region [0x60d00007c2d0,0x60d00007c351)
freed by thread T0 here:
#0 0x7ffff6efd1a8 in __interceptor_free (/usr/lib64/libasan.so.4+0xdc1a8)
#1 0x7ffff60af3b8 in g_free (/usr/lib64/libglib-2.0.so.0+0x583b8)
#2 0x41bf06 in rm_fmt_group_destroy lib/formats.c:52
#3 0x41d55a in rm_fmt_close lib/formats.c:438
#4 0x42c865 in rm_session_clear lib/session.c:131
#5 0x408bd6 in main src/rmlint.c:146
#6 0x7ffff552e34c in __libc_start_main (/lib64/libc.so.6+0x2534c)
previously allocated by thread T2 (pool-rmlint) here:
#0 0x7ffff6efd500 in malloc (/usr/lib64/libasan.so.4+0xdc500)
#1 0x7ffff60af2b8 in g_malloc (/usr/lib64/libglib-2.0.so.0+0x582b8)
Thread T2 (pool-rmlint) created by T0 here:
#0 0x7ffff6e5ac80 in pthread_create (/usr/lib64/libasan.so.4+0x39c80)
#1 0x7ffff60f500b (/usr/lib64/libglib-2.0.so.0+0x9e00b)
SUMMARY: AddressSanitizer: double-free (/usr/lib64/libasan.so.4+0xdc1a8) in __interceptor_free
==28750==ABORTING
[Thread 0x7ffff00f4700 (LWP 29637) exited]
[Inferior 1 (process 28750) exited with code 01]
No stack.
Missing separate debuginfos, use: zypper install libasan4-debuginfo-7.5.0+r278197-4.25.1.x86_64 libgcc_s1-debuginfo-10.3.0+git1587-1.6.4.x86_64 libstdc++6-debuginfo-10.3.0+git1587-1.6.4.x86_64
(gdb) |
|
The closest I have gotten to reproducing the issue is this: Which does not occur on the develop branch, so maybe your bug is already fixed? To debug further: If you download this patch, apply it with |
|
I've compiled the develop branch and it seems to work just fine. Guess this is only on master. |
|
If you can't reproduce on master anymore, then running develop with @SeeSpotRun It looks like we need a new release soon or a backport of whatever changes may have fixed this, including dcdab3b, which removes the implementation of --write-unfinished that uses rm_fmt_write and possibly could have caused an RmFile instance to be written to the formatter twice. |
|
Minimal reproducer:
It looks like the combination of |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Long story short, I was deduping two locations with
rmlint DUPES // ORIGINAL --keep-all-tagged --must-match-tagged -g -T 'all,-badids' --xattr-read --xattr-write --write-unfinishedThese locations are MergerFS FUSE mounts, but everything worked fine. MergerFS config is as follows
ergerfs -o defaults,allow_other,category.create=epmfs,minfreespace=50G,dropcacheonclose=true,moveonenospc=trueThe FS on disk to which MergerFS is pointing to, is BTRFS.
After I was done deduping the said locations, I wanted to dedupe the "DUPES" location, where all the duplicates were, so I run
rmlint DUPES -g -T all,-badids --xattr-read --xattr-write --write-unfinishedon it and the application stopped working with a "double free or corruption (out)" message, the json file is cutoff in the middle.To make sure that it is not MergerFS/FUSE fuckery, I run rmlint against the disk mount directly and got the same "double free or corruption (out)". Seems to be the same thing as in #447.
rmlint built with
scons DEBUG=1 VERBOSE=0 GDB=1Point to MergerFS:
Pointing to Mount directly:
The text was updated successfully, but these errors were encountered: