AddressSanitizer: heap-use-after-free

[rogerbinns]

With apsw-sqlite3mc:

import apsw


class memvfs(apsw.VFS):
    name = "mine"

    def __init__(self):
        super().__init__(self.name, "memdb")


xx = memvfs()

con = apsw.Connection("testdb", vfs="mine")

con.execute("create table x(y); insert into x values(randomblob(65536))")
con.pragma("hexrekey", "aabbccdd")
con.execute("insert into x select * from x")

Result:

=================================================================                                                                                                                                                                                                                                                           
==3238205==ERROR: AddressSanitizer: heap-use-after-free on address 0x53100007882c at pc 0x7a4c366dd5c0 bp 0x7ffd3e244c90 sp 0x7ffd3e244c80                                                                                                                                                                                  
READ of size 4 at 0x53100007882c thread T0                                                                                                                                                                                                                                                                                  
    #0 0x7a4c366dd5bf in sqlite3mcGetReservedWriteCipher /space/mc/sqlite3/sqlite3.c:282071                                                                                                                                                                                                                                 
    #1 0x7a4c36c3c2cd in sqlite3_rekey_v2 /space/mc/sqlite3/sqlite3.c:284256                                                                                                                                                                                                                                                
    #2 0x7a4c36b6ede3 in sqlite3mcFileControlPragma /space/mc/sqlite3/sqlite3.c:283280                                                                                                                                                                                                                                      
    #3 0x7a4c36b6ffd8 in sqlite3Pragma /space/mc/sqlite3/sqlite3.c:139364                                                                                                                                                                                                                                                   
    #4 0x7a4c36b980c6 in yy_reduce /space/mc/sqlite3/sqlite3.c:177397                                                                                                                                                                                                                                                       
    #5 0x7a4c36ba0140 in sqlite3Parser /space/mc/sqlite3/sqlite3.c:178028                                                                                                                                                                                                                                                   
    #6 0x7a4c36ba1dc7 in sqlite3RunParser /space/mc/sqlite3/sqlite3.c:179362                                                                                                                                                                                                                                                
    #7 0x7a4c36ba2e95 in sqlite3Prepare /space/mc/sqlite3/sqlite3.c:142756                                                                                                                                                                                                                                                  
    #8 0x7a4c36ba370f in sqlite3LockAndPrepare /space/mc/sqlite3/sqlite3.c:142831                                                                                                                                                                                                                                           
    #9 0x7a4c36ba45c2 in sqlite3_prepare_v3 /space/mc/sqlite3/sqlite3.c:142939                                                                                                                                                                                                                                              
    #10 0x7a4c36ba576f in statementcache_prepare_internal src/statementcache.c:243                                                                                                                                                                                                                                          
    #11 0x7a4c36ba7001 in statementcache_prepare src/statementcache.c:344                                                                                                                                                                                                                                                   
    #12 0x7a4c36c4c603 in APSWCursor_execute src/cursor.c:956                                                                                                                                                                                                                                                               
    #13 0x62bf7d6dc71e in cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:438                                                                                                                                                                                                                                 
    #14 0x62bf7d5bd43d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92                                                                                                                                                                                                                                      
    #15 0x62bf7d5bd586 in PyObject_Vectorcall Objects/call.c:325                                                                                                                                                                                                                                                            
    #16 0x7a4c36908eb6 in Connection_execute src/connection.c:4338                                                                                                                                                                                                                                                          
    #17 0x7a4c36909e6d in Connection_pragma src/connection.c:4435                                                                                                                                                                                                                                                           
    #18 0x62bf7d5f00f1 in method_vectorcall_FASTCALL_KEYWORDS Objects/descrobject.c:427                                                                                                                                                                                                                                     
    #19 0x62bf7d5bd43d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92                                                                                                                                                                                                                                      
    #20 0x62bf7d5bd586 in PyObject_Vectorcall Objects/call.c:325                                                                                                                                                                                                                                                            
    #21 0x62bf7d969a8c in _PyEval_EvalFrameDefault Python/bytecodes.c:2706                                                                                                                                                                                                                                                  
    #22 0x62bf7d980160 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:89                                                                                                                                                                                                                                              
    #23 0x62bf7d980412 in _PyEval_Vector Python/ceval.c:1683                                                                                                                                                                                                                                                                
    #24 0x62bf7d980676 in PyEval_EvalCode Python/ceval.c:578                                                                                                                                                                                                                                                                
    #25 0x62bf7dad9e85 in run_eval_code_obj Python/pythonrun.c:1722                                                                                                                                                                                                                                                         
    #26 0x62bf7dada09c in run_mod Python/pythonrun.c:1743                                                                                                                                                                                                                                                                   
    #27 0x62bf7dadd52d in pyrun_file Python/pythonrun.c:1643                                                                                                                                                                                                                                                                
    #28 0x62bf7dae1eae in _PyRun_SimpleFileObject Python/pythonrun.c:433                                                                                                                                                                                                                                                    
    #29 0x62bf7dae2193 in _PyRun_AnyFileObject Python/pythonrun.c:78                                                                                                                                                                                                                                                        
    #30 0x62bf7db64721 in pymain_run_file_obj Modules/main.c:360                                                                                                                                                                                                                                                            
    #31 0x62bf7db649fb in pymain_run_file Modules/main.c:379                                                                                                                                                                                                                                                                
    #32 0x62bf7db6759e in pymain_run_python Modules/main.c:629                                                                                                                                                                                                                                                              
    #33 0x62bf7db677a6 in Py_RunMain Modules/main.c:709                                                                                                                                                                                                                                                                     
    #34 0x62bf7db679bb in pymain_main Modules/main.c:739                                                                                                                                                                                                                                                                    
    #35 0x62bf7db67d40 in Py_BytesMain Modules/main.c:763                                                                                                                                                                                                                                                                   
    #36 0x62bf7d312aa5 in main Programs/python.c:15                                                                                                                                                                                                                                                                         
    #37 0x7a4c48c2a1c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58                                                                                                                                                                                                                                  
    #38 0x7a4c48c2a28a in __libc_start_main_impl ../csu/libc-start.c:360                                                                                                                                                                                                                                                    
    #39 0x62bf7d3129d4 in _start (/space/pydebug/bin/python3.12+0x10859d4) (BuildId: 2e0f261b4b9f3e971f6e374ca89e9e487c24f8ac)                                                                                                                                                                                              
                                                                                                                                                                                                                                                                                                                            
0x53100007882c is located 44 bytes inside of 65680-byte region [0x531000078800,0x531000088890)                                                                                                                                                                                                                              
freed by thread T0 here:                                                                                                                                                                                                                                                                                                    
    #0 0x7a4c498fa678 in free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52                                                                                                                                                                                                                                    
    #1 0x7a4c36741f17 in sqlite3MemFree /space/mc/sqlite3/sqlite3.c:26956                                                                                                                                                                                                                                                   
    #2 0x7a4c36809898 in sqlite3_free /space/mc/sqlite3/sqlite3.c:30789
    #3 0x7a4c3680e175 in sqlite3mcCodecFree /space/mc/sqlite3/sqlite3.c:283876
    #4 0x7a4c369da32a in sqlite3mcSetCodec /space/mc/sqlite3/sqlite3.c:307009
    #5 0x7a4c36c3c2c1 in sqlite3_rekey_v2 /space/mc/sqlite3/sqlite3.c:284255
    #6 0x7a4c36b6ede3 in sqlite3mcFileControlPragma /space/mc/sqlite3/sqlite3.c:283280
    #7 0x7a4c36b6ffd8 in sqlite3Pragma /space/mc/sqlite3/sqlite3.c:139364
    #8 0x7a4c36b980c6 in yy_reduce /space/mc/sqlite3/sqlite3.c:177397
    #9 0x7a4c36ba0140 in sqlite3Parser /space/mc/sqlite3/sqlite3.c:178028
    #10 0x7a4c36ba1dc7 in sqlite3RunParser /space/mc/sqlite3/sqlite3.c:179362
    #11 0x7a4c36ba2e95 in sqlite3Prepare /space/mc/sqlite3/sqlite3.c:142756
    #12 0x7a4c36ba370f in sqlite3LockAndPrepare /space/mc/sqlite3/sqlite3.c:142831
    #13 0x7a4c36ba45c2 in sqlite3_prepare_v3 /space/mc/sqlite3/sqlite3.c:142939 
    #14 0x7a4c36ba576f in statementcache_prepare_internal src/statementcache.c:243
    #15 0x7a4c36ba7001 in statementcache_prepare src/statementcache.c:344
    #16 0x7a4c36c4c603 in APSWCursor_execute src/cursor.c:956
    #17 0x62bf7d6dc71e in cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:438
    #18 0x62bf7d5bd43d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
    #19 0x62bf7d5bd586 in PyObject_Vectorcall Objects/call.c:325
    #20 0x7a4c36908eb6 in Connection_execute src/connection.c:4338
    #21 0x7a4c36909e6d in Connection_pragma src/connection.c:4435
    #22 0x62bf7d5f00f1 in method_vectorcall_FASTCALL_KEYWORDS Objects/descrobject.c:427
    #23 0x62bf7d5bd43d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
    #24 0x62bf7d5bd586 in PyObject_Vectorcall Objects/call.c:325
    #25 0x62bf7d969a8c in _PyEval_EvalFrameDefault Python/bytecodes.c:2706
    #26 0x62bf7d980160 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:89
    #27 0x62bf7d980412 in _PyEval_Vector Python/ceval.c:1683
    #28 0x62bf7d980676 in PyEval_EvalCode Python/ceval.c:578
    #29 0x62bf7dad9e85 in run_eval_code_obj Python/pythonrun.c:1722

previously allocated by thread T0 here:
    #0 0x7a4c498fbb37 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7a4c368ea34a in sqlite3MemMalloc /space/mc/sqlite3/sqlite3.c:26929
    #2 0x7a4c36806b7d in mallocWithAlarm /space/mc/sqlite3/sqlite3.c:30646
    #3 0x7a4c36806f1a in sqlite3Malloc /space/mc/sqlite3/sqlite3.c:30692
    #4 0x7a4c369ecd04 in sqlite3_malloc /space/mc/sqlite3/sqlite3.c:30710
    #5 0x7a4c36c3c0f6 in sqlite3_rekey_v2 /space/mc/sqlite3/sqlite3.c:284238
    #6 0x7a4c36b6ede3 in sqlite3mcFileControlPragma /space/mc/sqlite3/sqlite3.c:283280
    #7 0x7a4c36b6ffd8 in sqlite3Pragma /space/mc/sqlite3/sqlite3.c:139364
    #8 0x7a4c36b980c6 in yy_reduce /space/mc/sqlite3/sqlite3.c:177397
    #9 0x7a4c36ba0140 in sqlite3Parser /space/mc/sqlite3/sqlite3.c:178028
    #10 0x7a4c36ba1dc7 in sqlite3RunParser /space/mc/sqlite3/sqlite3.c:179362
    #11 0x7a4c36ba2e95 in sqlite3Prepare /space/mc/sqlite3/sqlite3.c:142756
    #12 0x7a4c36ba370f in sqlite3LockAndPrepare /space/mc/sqlite3/sqlite3.c:142831
    #13 0x7a4c36ba45c2 in sqlite3_prepare_v3 /space/mc/sqlite3/sqlite3.c:142939 
    #14 0x7a4c36ba576f in statementcache_prepare_internal src/statementcache.c:243
    #15 0x7a4c36ba7001 in statementcache_prepare src/statementcache.c:344
    #16 0x7a4c36c4c603 in APSWCursor_execute src/cursor.c:956
    #17 0x62bf7d6dc71e in cfunction_vectorcall_FASTCALL_KEYWORDS Objects/methodobject.c:438
    #18 0x62bf7d5bd43d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
    #19 0x62bf7d5bd586 in PyObject_Vectorcall Objects/call.c:325
    #20 0x7a4c36908eb6 in Connection_execute src/connection.c:4338
    #21 0x7a4c36909e6d in Connection_pragma src/connection.c:4435
    #22 0x62bf7d5f00f1 in method_vectorcall_FASTCALL_KEYWORDS Objects/descrobject.c:427
    #23 0x62bf7d5bd43d in _PyObject_VectorcallTstate Include/internal/pycore_call.h:92
    #24 0x62bf7d5bd586 in PyObject_Vectorcall Objects/call.c:325
    #25 0x62bf7d969a8c in _PyEval_EvalFrameDefault Python/bytecodes.c:2706
    #26 0x62bf7d980160 in _PyEval_EvalFrame Include/internal/pycore_ceval.h:89
    #27 0x62bf7d980412 in _PyEval_Vector Python/ceval.c:1683
    #28 0x62bf7d980676 in PyEval_EvalCode Python/ceval.c:578
    #29 0x62bf7dad9e85 in run_eval_code_obj Python/pythonrun.c:1722

SUMMARY: AddressSanitizer: heap-use-after-free /space/mc/sqlite3/sqlite3.c:282071 in sqlite3mcGetReservedWriteCipher

[rogerbinns]

It doesn't matter what parent VFS is used - eg unix gives the same error. And leaving out the rekey has everything fine too.

[rogerbinns]

If "memdb" in the code is replaced with "" which means use the default vfs, then this issue does not happen. Using "multipleciphers-unix" has the same effect as it is the default VFS.

[utelle]

The problem here is that the implementation of PRAGMA rekey should check first whether the VFS used for the current database supports encryption (that is, includes the multiple ciphers VFS shim in the VFS stack).

The current implementation of PRAGMA rekey does not perform this check. The reason can be found in the history: before the SQLITE_HAS_CODEC feature was removed in early 2020 the implementation of the encryption extension implicitly worked for all VFSes automatically, because the encryption/decryption took place, before data were written/read to/from disk.

Now, the user has to select a VFS which supports encryption through the multiple ciphers VFS shim.

The fix will be to add checks for encryption support to the implementation of sqlite3_key and sqlite3_rekey.

[utelle]

Commit 5208c16 should fix the issue.