MDEV-4682 - Include statistic fields to query cache and qc_info #6

rspadim · 2014-09-20T02:55:19Z

Todo:

variable to use/not use statistics (maybe just a compile time option, maybe not, must check what is easier to implement)

rpbouman · 2014-09-27T01:20:35Z

Hi Roberto! You send me a message to ask if I could check something for you. Could you please email me at roland.bouman@gmail.com so we can continue the conversation? Thanks in advance, and kind regards,

Roland.

rspadim · 2014-09-27T01:30:26Z

ops this should not be here, just at mcshadow :)

rspadim · 2014-09-30T19:41:03Z

config.h.cmake

+  Query cache stats, should be used with qc_info plugin (MDEV-4682)
+  Should be turned off by default?
+*/
+#define HAVE_QUERY_CACHE_STATS 1


uncomment to use statistics

…d files

svoj · 2016-02-18T05:56:44Z

Closing since corresponding JIRA task is closed and this seem to be addressed by PR#67. Feel free to reopen if I'm wrong.

Without sorting the test produces indeterministic result

This only merges MDEV-12253, adapting it to MDEV-12602 which is already present in 10.2 but not yet in the 10.1 revision that is being merged. TODO: Error handling in crash recovery needs to be improved. If a page cannot be decrypted (or read), we should cleanly abort the startup. If innodb_force_recovery is specified, we should ignore the problematic page and apply redo log to other pages. Currently, the test encryption.innodb-redo-badkey randomly fails like this (the last messages are from cmake -DWITH_ASAN): 2017-05-05 10:19:40 140037071685504 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1635994 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE before MLOG_CHECKPOINT for tablespace 1 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Data structure corruption 2017-05-05 10:19:41 140037071685504 [Note] InnoDB: Starting shutdown... i================================================================= ==5226==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x612000018588 in thread T0 #0 0x736750 in operator delete(void*) (/mariadb/server/build/sql/mysqld+0x736750) #1 0x1e4833f in LatchCounter::~LatchCounter() /mariadb/server/storage/innobase/include/sync0types.h:599:4 #2 0x1e480b8 in LatchMeta<LatchCounter>::~LatchMeta() /mariadb/server/storage/innobase/include/sync0types.h:786:17 #3 0x1e35509 in sync_latch_meta_destroy() /mariadb/server/storage/innobase/sync/sync0debug.cc:1622:3 #4 0x1e35314 in sync_check_close() /mariadb/server/storage/innobase/sync/sync0debug.cc:1839:2 #5 0x1dfdc18 in innodb_shutdown() /mariadb/server/storage/innobase/srv/srv0start.cc:2888:2 #6 0x197e5e6 in innobase_init(void*) /mariadb/server/storage/innobase/handler/ha_innodb.cc:4475:3

…-12916 This is a joint patch fixing the following problems: MDEV-12875 Wrong VIEW column data type for COALESCE(int_column) MDEV-12886 Different default for INT and BIGINT column in a VIEW for a SELECT with ROLLUP MDEV-12916 Wrong column data type for an INT field of a cursor-anchored ROW variable All above problem happened because the global function ::create_tmp_field() called the top-level Item::create_tmp_field(), which made some tranformation for INT-result data types. For example, INT(11) became BIGINT(11), because 11 is a corner case and it's not known if it fits or does not fit into INT range, so Item::create_tmp_field() converted it to BIGINT(11) for safety. The main idea of this patch is to avoid such tranformations. 1. Fixing Item::create_tmp_field() not to have a special case for INT_RESULT. Item::create_tmp_field() is changed not to have a special case for INT_RESULT (which earlier made a decision based on Item's max_length). It now calls tmp_table_field_from_field_type() for INT_RESULT, therefore preserves the original data type (e.g. INT, YEAR) without conversion to BIGINT. This change is valid, because a number of recent fixes (e.g. in Item_func_int, Item_hybrid_func, Item_int, Item_splocal) guarantee that item->type_handler() now properly returns type_handler_long vs type_handler_longlong. So no adjustment by length is needed any more for Items returning INT_RESULT. After this change, Item::create_tmp_field() calls tmp_table_field_from_field_type() for all XXX_RESULT, except REAL_RESULT. 2. Fixing Item::create_tmp_field() not to have a special case for REAL_RESULT. Note, the reason for a special case for REAL_RESULT is to have a special constructor for Field_double(), forcing Field_real::not_fixed to be set to true. Taking into account that only Item_sum descendants actually need a special constructor call Field_double(not_fixed=true), not too loose precision when mixing individual rows to the aggregate result: - renaming Item::create_tmp_field() to Item_sum::create_tmp_field(). - changing Item::create_tmp_field() just to call tmp_table_field_from_field_type() for all XXX_RESULT types. A special case for REAL_RESULT in Item::create_tmp_field() is now gone. Item::create_tmp_field() is now symmetric for all XXX_RESULT types, and now just calls tmp_table_field_from_field_type(). 3. Fixing Item_func::create_field_for_create_select() not to have a special case for STRING_RESULT. After changes #1 and #2, the code in Item_func::create_field_for_create_select(), testing result_type(), becomes useless, because: now Item::create_tmp_field() and tmp_table_field_from_field_type() do exactly the same thing for all XXX_RESULT types for Item_func descendants: a. It calls tmp_table_field_from_field_type for STRING_RESULT directly. b. For other XXX_RESULT, it goes through Item::create_tmp_field(), which calls the global function ::create_tmp_field(), which calls item->create_tmp_field() for FUNC_ITEM, which calls tmp_table_field_from_field_type() again. So removing the virtual implementation of Item_func::create_field_for_create_select(). The inherited Item::create_field_for_create_select() now perfectly does the job, as it also calls tmp_table_field_from_field_type() for FUNC_ITEM, independently from XXX_RESULT type. 4. Taking into account #1 and #2, as well as some recent changes, removing virtual implementations: - Item_hybrid_func::create_tmp_field() - Item_hybrid_func::create_field_for_create_select() - Item_int_func::create_tmp_field() - Item_int_func::create_field_for_create_select() - Item_temporal_func::create_field_for_create_select() The derived versions from Item now perfectly work. 5. Moving a piece of code from create_tmp_field_from_item() to a new function create_tmp_field_from_item_finalize(), to reuse it in two places (see #6). 6. Changing the code responsible for BIT->INT/BIGIN tranformation (which is called for the cases when the created table, e.g. HEAP, does not fully support BIT) not to call create_tmp_field_from_item(), because the latter now calls tmp_table_field_from_field_type() instead of create_tmp_field() and thefore cannot do BIT transformation. So rewriting this code using a sequence of these calls: - item->type_handler_long_or_longlong() - handler->make_and_init_table_field() - create_tmp_field_from_item_finalize() 7. Miscelaneous changes: - Moving type_handler_long_or_longlong() from "protected" to "public", as it's now needed in the global function create_tmp_field(). 8. The above changes fixed MDEV-12875, MDEV-12886, MDEV-12916. So adding tests for these bugs.

THREE BYTES ON X86 Analysis: ========= The macro uint3korr reads 4 bytes of data instead of 3 on on x86 machines. Multiple definitions were created for this macro for optimization in WIN32. The idea was to optimize reading of 3 byte ints by reading an ordinary int and masking away the unused byte. However this is an undefined behavior. It will be an issue unless users are aware of allocating an extra byte for using this macro. Fix: ==== Removing the definition which reads 4 bytes of data. The only definition of this macro would now read just 3 bytes of data thus prohibiting the usage of an extra byte. Note: ===== This is a backport of Patches #5 and #6 for Bug#17922198.

Fixes this report: ==3165==ERROR: AddressSanitizer: use-after-poison on address 0x61e0000270a0 at pc 0x00000114b78c bp 0x7f15d65fe120 sp 0x7f15d65fd8d0 WRITE of size 1366 at 0x61e0000270a0 thread T28 #0 0x114b78b in __asan_memcpy fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cc:23 #1 0x208208d in TABLE::init(THD*, TABLE_LIST*) work/mariadb/sql/table.cc:4662:3 #2 0x19df85b in open_table(THD*, TABLE_LIST*, Open_table_context*) work/mariadb/sql/sql_base.cc:1993:10 #3 0x19eb968 in open_and_process_table(THD*, LEX*, TABLE_LIST*, unsigned int*, unsigned int, Prelocking_strategy*, bool, Open_table_context*) work/mariadb/sql/sql_base.cc:3483:14 #4 0x19e7c05 in open_tables(THD*, DDL_options_st const&, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) work/mariadb/sql/sql_base.cc:4001:14 #5 0x19f4dac in open_and_lock_tables(THD*, DDL_options_st const&, TABLE_LIST*, bool, unsigned int, Prelocking_strategy*) work/mariadb/sql/sql_base.cc:4879:7 #6 0x1627263 in open_and_lock_tables(THD*, TABLE_LIST*, bool, unsigned int) work/mariadb/sql/sql_base.h:487:10 #7 0x1c3839c in mysql_execute_command(THD*) work/mariadb/sql/sql_parse.cc:5113:13 #8 0x1c1b72c in mysql_parse(THD*, char*, unsigned int, Parser_state*, bool, bool) work/mariadb/sql/sql_parse.cc:7980:18 #9 0x1c13464 in handle_bootstrap_impl(THD*) work/mariadb/sql/sql_parse.cc:1044:5 #10 0x1c11ff7 in do_handle_bootstrap(THD*) work/mariadb/sql/sql_parse.cc:1096:3 #11 0x1c11d14 in handle_bootstrap work/mariadb/sql/sql_parse.cc:1079:3 #12 0x115a6ae in __asan::AsanThread::ThreadStart(unsigned long, __sanitizer::atomic_uintptr_t*) fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_thread.cc:259 #13 0x7f15fe1407fb in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x77fb) #14 0x7f15fbb64b5e in clone /build/glibc-itYbWN/glibc-2.26/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

srv_last_monitor_time: make all accesses relaxed atomical WARNING: ThreadSanitizer: data race (pid=12041) Write of size 8 at 0x000003949278 by thread T26 (mutexes: write M226445748578513120): #0 thd_destructor_proxy storage/innobase/handler/ha_innodb.cc:314:14 (mysqld+0x19b5505) Previous read of size 8 at 0x000003949278 by main thread: #0 innobase_init(void*) storage/innobase/handler/ha_innodb.cc:4180:11 (mysqld+0x1a03404) #1 ha_initialize_handlerton(st_plugin_int*) sql/handler.cc:522:31 (mysqld+0xc5ec73) #2 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) sql/sql_plugin.cc:1447:9 (mysqld+0x134908d) #3 plugin_init(int*, char**, int) sql/sql_plugin.cc:1729:15 (mysqld+0x13484f0) #4 init_server_components() sql/mysqld.cc:5345:7 (mysqld+0xbf720f) #5 mysqld_main(int, char**) sql/mysqld.cc:5940:7 (mysqld+0xbf107d) #6 main sql/main.cc:25:10 (mysqld+0xbe971b) Location is global 'srv_running' of size 8 at 0x000003949278 (mysqld+0x000003949278) WARNING: ThreadSanitizer: data race (pid=27869) Atomic write of size 4 at 0x7b4800000c00 by thread T8: #0 __tsan_atomic32_exchange llvm/projects/compiler-rt/lib/tsan/rtl/tsan_interface_atomic.cc:589 (mysqld+0xbd4eac) #1 TTASEventMutex<GenericPolicy>::exit() storage/innobase/include/ib0mutex.h:467:7 (mysqld+0x1a8d4cb) #2 PolicyMutex<TTASEventMutex<GenericPolicy> >::exit() storage/innobase/include/ib0mutex.h:609:10 (mysqld+0x1a7839e) #3 fil_validate() storage/innobase/fil/fil0fil.cc:5535:2 (mysqld+0x1abd913) #4 fil_validate_skip() storage/innobase/fil/fil0fil.cc:204:9 (mysqld+0x1aba601) #5 fil_aio_wait(unsigned long) storage/innobase/fil/fil0fil.cc:5296:2 (mysqld+0x1abbae6) #6 io_handler_thread storage/innobase/srv/srv0start.cc:340:3 (mysqld+0x21abe1e) Previous read of size 4 at 0x7b4800000c00 by main thread (mutexes: write M1273, write M1271): #0 TTASEventMutex<GenericPolicy>::state() const storage/innobase/include/ib0mutex.h:530:10 (mysqld+0x21c66e2) #1 sync_array_detect_deadlock(sync_array_t*, sync_cell_t*, sync_cell_t*, unsigned long) storage/innobase/sync/sync0arr.cc:746:14 (mysqld+0x21c1c7a) #2 sync_array_wait_event(sync_array_t*, sync_cell_t*&) storage/innobase/sync/sync0arr.cc:465:6 (mysqld+0x21c1708) #3 TTASEventMutex<GenericPolicy>::enter(unsigned int, unsigned int, char const*, unsigned int) storage/innobase/include/ib0mutex.h:516:6 (mysqld+0x1a8c206) #4 PolicyMutex<TTASEventMutex<GenericPolicy> >::enter(unsigned int, unsigned int, char const*, unsigned int) storage/innobase/include/ib0mutex.h:635:10 (mysqld+0x1a782c3) #5 fil_mutex_enter_and_prepare_for_io(unsigned long) storage/innobase/fil/fil0fil.cc:1131:3 (mysqld+0x1a9a92e) #6 fil_io(IORequest const&, bool, page_id_t const&, page_size_t const&, unsigned long, unsigned long, void*, void*, bool) storage/innobase/fil/fil0fil.cc:5082:2 (mysqld+0x1ab8de2) #7 buf_flush_write_block_low(buf_page_t*, buf_flush_t, bool) storage/innobase/buf/buf0flu.cc:1112:3 (mysqld+0x1cb970a) #8 buf_flush_page(buf_pool_t*, buf_page_t*, buf_flush_t, bool) storage/innobase/buf/buf0flu.cc:1270:3 (mysqld+0x1cb7d70) #9 buf_flush_try_neighbors(page_id_t const&, buf_flush_t, unsigned long, unsigned long) storage/innobase/buf/buf0flu.cc:1493:9 (mysqld+0x1cc9674) #10 buf_flush_page_and_try_neighbors(buf_page_t*, buf_flush_t, unsigned long, unsigned long*) storage/innobase/buf/buf0flu.cc:1565:13 (mysqld+0x1cbadf3) #11 buf_do_flush_list_batch(buf_pool_t*, unsigned long, unsigned long) storage/innobase/buf/buf0flu.cc:1825:3 (mysqld+0x1cbbcb8) #12 buf_flush_batch(buf_pool_t*, buf_flush_t, unsigned long, unsigned long, flush_counters_t*) storage/innobase/buf/buf0flu.cc:1895:16 (mysqld+0x1cbb459) #13 buf_flush_do_batch(buf_pool_t*, buf_flush_t, unsigned long, unsigned long, flush_counters_t*) storage/innobase/buf/buf0flu.cc:2065:2 (mysqld+0x1cbcfe1) #14 buf_flush_lists(unsigned long, unsigned long, unsigned long*) storage/innobase/buf/buf0flu.cc:2167:8 (mysqld+0x1cbd5a3) #15 log_preflush_pool_modified_pages(unsigned long) storage/innobase/log/log0log.cc:1400:13 (mysqld+0x1eefc3b) #16 log_make_checkpoint_at(unsigned long, bool) storage/innobase/log/log0log.cc:1751:10 (mysqld+0x1eefb16) #17 buf_dblwr_create() storage/innobase/buf/buf0dblwr.cc:335:2 (mysqld+0x1cd2141) #18 innobase_start_or_create_for_mysql() storage/innobase/srv/srv0start.cc:2539:10 (mysqld+0x21b4d8e) #19 innobase_init(void*) storage/innobase/handler/ha_innodb.cc:4193:8 (mysqld+0x1a5e3d7) #20 ha_initialize_handlerton(st_plugin_int*) sql/handler.cc:522:31 (mysqld+0xc74d33) #21 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) sql/sql_plugin.cc:1447:9 (mysqld+0x1376d5d) #22 plugin_init(int*, char**, int) sql/sql_plugin.cc:1729:15 (mysqld+0x13761c0) #23 init_server_components() sql/mysqld.cc:5348:7 (mysqld+0xc0d0ff) #24 mysqld_main(int, char**) sql/mysqld.cc:5943:7 (mysqld+0xc06f9d) #25 main sql/main.cc:25:10 (mysqld+0xbff71b) WARNING: ThreadSanitizer: data race (pid=29031) Write of size 8 at 0x0000039e48e0 by thread T15: #0 srv_monitor_thread storage/innobase/srv/srv0srv.cc:1699:24 (mysqld+0x21a254e) Previous write of size 8 at 0x0000039e48e0 by thread T14: #0 srv_refresh_innodb_monitor_stats() storage/innobase/srv/srv0srv.cc:1165:24 (mysqld+0x21a3124) #1 srv_error_monitor_thread storage/innobase/srv/srv0srv.cc:1836:3 (mysqld+0x21a2d40) Location is global 'srv_last_monitor_time' of size 8 at 0x0000039e48e0 (mysqld+0x0000039e48e0)

fix compiler cast warnings in unittests

…larations 1. Adding LEX::make_item_sysvar() and reusing it in sql_yacc.yy and sql_yacc_ora.yy. Removing the "opt_component" rule. 2. Renaming rules to better reflect their purpose: - keyword to keyword_ident - keyword_sp to keyword_label - keyword_sp_not_data_type to keyword_sp_var_and_label Also renaming: - sp_decl_ident_keyword to keyword_sp_decl for naming consistency - keyword_alias to keyword_table_alias, for consistency with ident_table_alias - keyword_sp_data_type to keyword_data_type, as it has nothing SP-specific. 3. Moving GLOBAL_SYM, LOCAL_SYM, SESSION_SYM from keyword_sp_var_and_label to a separate rule keyword_sysvar_type. We don't have system variables with these names anyway. Adding ident_sysvar_name and using it in the grammar that needs a system variable name instead of ident_or_text. This removed a number of shift/reduce conflicts between GLOBAL_SYM/LOCAL_SYM/SESSION_SYM as a variable scope and as a variable name. 4. Moving keywords BEGIN_SYM, END (in both *.yy fiels) and EXCEPTION_SYM (in sql_yacc_ora.yy) into a separate rule keyword_sp_block_section, because in Oracle verb keywords (COMMIT, DO, HANDLER, OPEN, REPAIR, ROLLBACK, SAVEPOINT, SHUTDOWN, TRUNCATE) are good variables names and can appear in e.g. DECLARE, while block keywords (BEGIN, END, EXCEPTION) are not good variable names and cannot appear in DECLARE. 5. Further splitting keyword_directly_not_assignable in sql_yacc_ora.yy: moving keyword_sp_verb_clause out. Renaming the rest of keyword_directly_not_assignable to keyword_sp_head, which represents keywords that can appear in optional clauses in CREATE PROCEDURE/FUNCTION/TRIGGER. 6. Renaming keyword_sp_verb_clause to keyword_verb_clause, as now it does not contains anything SP-specific. As a result or #4,#5,#6, the rule keyword_directly_not_assignable was replaced to three separate rules: - keyword_sp_block - keyword_sp_head - keyword_verb_clause Adding the same rules in sql_yacc.yy, for unification. 6. Adding keyword_sp_head and keyword_verb_clause into keyword_sp_decl. This fixes MDEV-16244. 7. Reorganizing the rest of keyword related rules into two groups: a. Rules defining a list of keywords and consisting of only terminal symbols: - keyword_sp_var_not_label - keyword_sp_head - keyword_sp_verb_clause - keyword_sp_block_section - keyword_sysvar_type b. Rules that combine the above lists into keyword places: - keyword_table_alias - keyword_ident - keyword_label - keyword_sysvar_name - keyword_sp_decl Rules from the group "b" use on the right side only rules from the group "a" (with optional terminal symbols added). Rules from the group "b" DO NOT mutually use each other any more. This makes them easier to read (and see the difference between them). Sorting the right sides of the group "b" keyword rules alphabetically, for yet better readability.

specific temporary errors The optimistic parallel slave's worker thread could face a run-time error due to the algorithm's specifics which allows for conflicts like the reported "Can't find record in 'table'". A typical stack is like {noformat} #0 handler::print_error (this=0x61c00008f8a0, error=149, errflag=0) at handler.cc:3650 #1 0x0000555555e95361 in write_record (thd=thd@entry=0x62a0000a2208, table=table@entry=0x61f00008ce88, info=info@entry=0x7fffdee356d0) at sql_insert.cc:1944 #2 0x0000555555ea7767 in mysql_insert (thd=thd@entry=0x62a0000a2208, table_list=0x61b00012ada0, fields=..., values_list=..., update_fields=..., update_values=..., duplic=<optimized out>, ignore=<optimized out>) at sql_insert.cc:1039 #3 0x0000555555efda90 in mysql_execute_command (thd=thd@entry=0x62a0000a2208) at sql_parse.cc:3927 #4 0x0000555555f0cc50 in mysql_parse (thd=0x62a0000a2208, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at sql_parse.cc:7449 #5 0x00005555566d4444 in Query_log_event::do_apply_event (this=0x61200005b9c8, rgi=<optimized out>, query_arg=<optimized out>, q_len_arg=<optimized out>) at log_event.cc:4508 #6 0x00005555566d639e in Query_log_event::do_apply_event (this=<optimized out>, rgi=<optimized out>) at log_event.cc:4185 #7 0x0000555555d738cf in Log_event::apply_event (rgi=0x61d0001ea080, this=0x61200005b9c8) at log_event.h:1343 #8 apply_event_and_update_pos_apply (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080, reason=<optimized out>) at slave.cc:3479 #9 0x0000555555d8596b in apply_event_and_update_pos_for_parallel (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080) at slave.cc:3623 #10 0x00005555562aca83 in rpt_handle_event (qev=qev@entry=0x6190000fa088, rpt=rpt@entry=0x62200002bd68) at rpl_parallel.cc:50 #11 0x00005555562bd04e in handle_rpl_parallel_thread (arg=arg@entry=0x62200002bd68) at rpl_parallel.cc:1258 {noformat} Here {{handler::print_error}} computes whether to error log the current error when --log-warnings > 1. The decision flag is consulted bu {{my_message_sql()}} which can be eventually called. In the bug case the decision is to log. However in the optimistic mode slave applier case any conflict is attempted to resolve with rollback and retry to success. Hence the logging is at least extraneous. The case is fixed with refining the flags computation for my_message_sql() to downgrade it to the warning level when the error comes from the *optimistically* {{rpl_group_info::SPECULATE_OPTIMISTIC}} running parallel slave thread. As this change has a side effect to stop poluting the slave worker's THD::main_da the slave warning reporting is slightly refined so that {{convert_handler_error()}} does not log any more warning message when there's one already. Secondly, post temporary error {{convert_kill_to_deadlock_error()}} is also refined to accept a "manual" (not being in THD da) error code. This change is necessary to force a pseudo-deadlock error reporting and consequent retry.

specific temporary errors The optimistic parallel slave's worker thread could face a run-time error due to the algorithm's specifics which allows for conflicts like the reported "Can't find record in 'table'". A typical stack is like {noformat} #0 handler::print_error (this=0x61c00008f8a0, error=149, errflag=0) at handler.cc:3650 #1 0x0000555555e95361 in write_record (thd=thd@entry=0x62a0000a2208, table=table@entry=0x61f00008ce88, info=info@entry=0x7fffdee356d0) at sql_insert.cc:1944 #2 0x0000555555ea7767 in mysql_insert (thd=thd@entry=0x62a0000a2208, table_list=0x61b00012ada0, fields=..., values_list=..., update_fields=..., update_values=..., duplic=<optimized out>, ignore=<optimized out>) at sql_insert.cc:1039 #3 0x0000555555efda90 in mysql_execute_command (thd=thd@entry=0x62a0000a2208) at sql_parse.cc:3927 #4 0x0000555555f0cc50 in mysql_parse (thd=0x62a0000a2208, rawbuf=<optimized out>, length=<optimized out>, parser_state=<optimized out>) at sql_parse.cc:7449 #5 0x00005555566d4444 in Query_log_event::do_apply_event (this=0x61200005b9c8, rgi=<optimized out>, query_arg=<optimized out>, q_len_arg=<optimized out>) at log_event.cc:4508 #6 0x00005555566d639e in Query_log_event::do_apply_event (this=<optimized out>, rgi=<optimized out>) at log_event.cc:4185 #7 0x0000555555d738cf in Log_event::apply_event (rgi=0x61d0001ea080, this=0x61200005b9c8) at log_event.h:1343 #8 apply_event_and_update_pos_apply (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080, reason=<optimized out>) at slave.cc:3479 #9 0x0000555555d8596b in apply_event_and_update_pos_for_parallel (ev=ev@entry=0x61200005b9c8, thd=thd@entry=0x62a0000a2208, rgi=rgi@entry=0x61d0001ea080) at slave.cc:3623 #10 0x00005555562aca83 in rpt_handle_event (qev=qev@entry=0x6190000fa088, rpt=rpt@entry=0x62200002bd68) at rpl_parallel.cc:50 #11 0x00005555562bd04e in handle_rpl_parallel_thread (arg=arg@entry=0x62200002bd68) at rpl_parallel.cc:1258 {noformat} Here {{handler::print_error}} computes whether to error log the current error when --log-warnings > 1. The decision flag is consulted bu {{my_message_sql()}} which can be eventually called. In the bug case the decision is to log. However in the optimistic mode slave applier case any conflict is attempted to resolve with rollback and retry to success. Hence the logging is at least extraneous. The case is fixed with adding a new flag {{ME_LOG_AS_WARN}} which {{handler::print_error}} may propagate further on through {{my_error}} when the error comes from an optimistically running slave worker thread. The new flag effectively requests the warning level for the errlog record, while the thread's DA records the actual error (which is regarded as temporary one by the parallel slave error handler).

To reproduce this bug just following below steps, client 1: USE test; CREATE TABLE t1 (i INT) ENGINE=MyISAM; HANDLER t1 OPEN h; CREATE TABLE t2 (i INT) ENGINE=RocksDB; LOCK TABLES t2 WRITE; client 2: FLUSH TABLES WITH READ LOCK; client 1: INSERT INTO t2 VALUES (1); So client 1 acquired the lock and set m_lock_rows = RDB_LOCK_WRITE. Then client 2 calls store_lock(TL_IGNORE) and m_lock_rows was wrongly set to RDB_LOCK_NONE, as below ``` #0 myrocks::ha_rocksdb::store_lock (this=0x7fffbc03c7c8, thd=0x7fffc0000ba0, to=0x7fffc0011220, lock_type=TL_IGNORE) MariaDB#1 get_lock_data (thd=0x7fffc0000ba0, table_ptr=0x7fffe84b7d20, count=1, flags=2) MariaDB#2 mysql_lock_abort_for_thread (thd=0x7fffc0000ba0, table=0x7fffbc03bbc0) MariaDB#3 THD::notify_shared_lock (this=0x7fffc0000ba0, ctx_in_use=0x7fffbc000bd8, needs_thr_lock_abort=true) MariaDB#4 MDL_lock::notify_conflicting_locks (this=0x555557a82380, ctx=0x7fffc0000cc8) MariaDB#5 MDL_context::acquire_lock (this=0x7fffc0000cc8, mdl_request=0x7fffe84b8350, lock_wait_timeout=2) MariaDB#6 Global_read_lock::lock_global_read_lock (this=0x7fffc0003fe0, thd=0x7fffc0000ba0) ``` Finally, client 1 "INSERT INTO..." hits the Assertion 'm_lock_rows == RDB_LOCK_WRITE' failed in myrocks::ha_rocksdb::write_row() Fix this bug by not setting m_locks_rows if lock_type == TL_IGNORE.

…test pass

Close connection handler on connection failure. This fixes 14 failing tests in main suite under clang+ASAN build. ASAN report for main.connect looks like this: ================================================================= ==25495==ERROR: LeakSanitizer: detected memory leaks Direct leak of 146280 byte(s) in 115 object(s) allocated from: #0 0x4fba47 in calloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:138 #1 0x5a7a02 in mysql_init /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:977:26 #2 0x570a7a in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6096:26 #3 0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9 #4 0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310 Indirect leak of 7065600 byte(s) in 115 object(s) allocated from: #0 0x4fb80f in __interceptor_malloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:129 #1 0x637a83 in my_context_init /work/mariadb/libmariadb/libmariadb/ma_context.c:367:23 #2 0x59fd16 in mysql_optionsv /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:2738:9 #3 0x5bc1d4 in mysql_options /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:3242:10 #4 0x570b94 in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6103:7 #5 0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9 #6 0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310 Indirect leak of 940240 byte(s) in 115 object(s) allocated from: #0 0x4fb80f in __interceptor_malloc /fun/cpp_projects/llvm_toolchain/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:129 #1 0x64386e in ma_init_dynamic_array /work/mariadb/libmariadb/libmariadb/ma_array.c:49:31 #2 0x649ead in _hash_init /work/mariadb/libmariadb/libmariadb/ma_hash.c:52:7 #3 0x5a3080 in mysql_optionsv /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:2938:13 #4 0x5bc20c in mysql_options4 /work/mariadb/libmariadb/libmariadb/mariadb_lib.c:3248:10 #5 0x56f63b in connect_n_handle_errors(st_command*, st_mysql*, char const*, char const*, char const*, char const*, int, char const*) /work/mariadb/client/mysqltest.cc:5874:3 #6 0x57146b in do_connect(st_command*) /work/mariadb/client/mysqltest.cc:6193:7 #7 0x584c39 in main /work/mariadb/client/mysqltest.cc:9321:9 #8 0x7fd15514db96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310 ... Closes #809

Wrong value F_WRLCK for thr_lock_type.

=============================================================================== I. If some record is deleted by rollback, it's lock is inherited as gap lock to the next record. And if the next record is then purged while the lock is still held, the debug check will fail. The scenario is the following: 1) Some thread executes "INSERT" and checks clustered index for duplicates, it sets shared lock for checked record(let's call it record A) converting implicit lock to explicit one. Note that the record's transaction id is the same as the current transaction id: ------------------- 0x000055f1e65c6bd6 in lock_rec_create_low (c_lock=0x0, thr=0x0, type_mode=1059, space=11, page_no=3, page=0x2e167079c000 "l\206", <incomplete sequence \372\221>, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, holds_trx_mutex=true) at ./storage/innobase/lock/lock0lock.cc:1466 1466 lock->type_mode = (type_mode & ~LOCK_TYPE_MASK) | LOCK_REC; (rr) bt \#0 0x000055f1e65c6bd6 in lock_rec_create_low (c_lock=0x0, thr=0x0, type_mode=1059, space=11, page_no=3, page=0x2e167079c000 "l\206", <incomplete sequence \372\221>, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, holds_trx_mutex=true) at ./storage/innobase/lock/lock0lock.cc:1466 \#1 0x000055f1e65c2b29 in lock_rec_create (c_lock=0x0, thr=0x0, type_mode=1059, block=0x2e1670080560, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=true) at ./storage/innobase/include/lock0lock.ic:133 \#2 0x000055f1e65c83fe in lock_rec_add_to_queue (type_mode=1059, block=0x2e1670080560, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=true) at ./storage/innobase/lock/lock0lock.cc:1941 \#3 0x000055f1e65d228f in lock_rec_convert_impl_to_expl_for_trx (block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, trx=0x55522d18d188, heap_no=9) at ./storage/innobase/lock/lock0lock.cc:5832 \#4 0x000055f1e65d2537 in lock_rec_convert_impl_to_expl (block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, offsets=0x663f4cace6d0) at ./storage/innobase/lock/lock0lock.cc:5886 \#5 0x000055f1e65d32d9 in lock_clust_rec_read_check_and_lock (flags=0, block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, offsets=0x663f4cace6d0, mode=LOCK_S, gap_mode=1024, thr=0x611370025b88) at ./storage/innobase/lock/lock0lock.cc:6194 \#6 0x000055f1e666b03c in row_ins_set_shared_rec_lock (type=1024, block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, offsets=0x663f4cace6d0, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:1427 \#7 0x000055f1e666cf4f in row_ins_duplicate_error_in_clust (flags=0, cursor=0x663f4cace9e0, entry=0x61137c044900, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:2360 \#8 0x000055f1e666db48 in row_ins_clust_index_entry_low (flags=0, mode=2, index=0x149444393cd0, n_uniq=1, entry=0x61137c044900, n_ext=0, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:2658 \#9 0x000055f1e666f0cb in row_ins_clust_index_entry (index=0x149444393cd0, entry=0x61137c044900, thr=0x611370025b88, n_ext=0) at ./storage/innobase/row/row0ins.cc:3146 \#10 0x000055f1e666f4a8 in row_ins_index_entry (index=0x149444393cd0, entry=0x61137c044900, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3265 \#11 0x000055f1e666f9a4 in row_ins_index_entry_step (node=0x611370025658, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3416 \#12 0x000055f1e666fd2a in row_ins (node=0x611370025658, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3553 \#13 0x000055f1e66700c6 in row_ins_step (thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3677 \#14 0x000055f1e668c7e0 in row_insert_for_mysql (mysql_rec=0x61137005c460 "\377", prebuilt=0x611370024d50) at ./storage/innobase/row/row0mysql.cc:1408 \#15 0x000055f1e6556c1f in ha_innobase::write_row (this=0x611370033a00, record=0x61137005c460 "\377") at ./storage/innobase/handler/ha_innodb.cc:8284 \#16 0x000055f1e6378051 in handler::ha_write_row (this=0x611370033a00, buf=0x61137005c460 "\377") at ./sql/handler.cc:6118 \#17 0x000055f1e60f21c4 in write_record (thd=0xa48640010a8, table=0x61137005b8c8, info=0x663f4cacfa00) at ./sql/sql_insert.cc:1939 \#18 0x000055f1e60f00f4 in mysql_insert (thd=0xa48640010a8, table_list=0xa4864010d40, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at ./sql/sql_insert.cc:1066 \#19 0x000055f1e61149d7 in mysql_execute_command (thd=0xa48640010a8) at ./sql/sql_parse.cc:4220 \#20 0x000055f1e611faf6 in mysql_parse (thd=0xa48640010a8, rawbuf=0xa48640109e0 "INSERT INTO t6 (col1,col2, col_int, col_string, col_text) VALUES /* NULL */ (NULL,NULL,NULL,REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), 10),REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), @fill_amount) ), (NULL,N"..., length=347, parser_state=0x663f4cad0670, is_com_multi=false, is_next_command=false) at ./sql/sql_parse.cc:7796 ------------------- 2) Then duplicate key is found in row_ins_duplicate_error_in_clust(), and the transaction is rolled back. When it's rolled back, the lock is inherited to the next record(let's call it record B) as a gap lock: ------------------- \#0 lock_rec_create_low (c_lock=0x0, thr=0x0, type_mode=547, space=11, page_no=3, page=0x2e167079c000 "l\206", <incomplete sequence \372\221>, heap_no=33, index=0x149444393cd0, trx=0x55522d18d188, holds_trx_mutex=false) at ./storage/innobase/lock/lock0lock.cc:1467 \#1 0x000055f1e65c2b29 in lock_rec_create (c_lock=0x0, thr=0x0, type_mode=547, block=0x2e1670080560, heap_no=33, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=false) at ./storage/innobase/include/lock0lock.ic:133 \#2 0x000055f1e65c83fe in lock_rec_add_to_queue (type_mode=547, block=0x2e1670080560, heap_no=33, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=false) at ./storage/innobase/lock/lock0lock.cc:1941 \#3 0x000055f1e65c9fee in lock_rec_inherit_to_gap (heir_block=0x2e1670080560, block=0x2e1670080560, heir_heap_no=33, heap_no=9) at ./storage/innobase/lock/lock0lock.cc:2580 \#4 0x000055f1e65cc057 in lock_update_delete (block=0x2e1670080560, rec=0x2e167079c299 "\200", from_purge=false) at ./storage/innobase/lock/lock0lock.cc:3559 \#5 0x000055f1e6788e57 in btr_cur_optimistic_delete (cursor=0xa486405aff0, flags=0, mtr=0x663f4cacf250, from_purge=false) at ./storage/innobase/btr/btr0cur.cc:5252 \#6 0x000055f1e68af6aa in row_undo_ins_remove_clust_rec (node=0xa486405af80) at ./storage/innobase/row/row0uins.cc:141 \#7 0x000055f1e68b05b5 in row_undo_ins (node=0xa486405af80, thr=0xa4864042d78) at ./storage/innobase/row/row0uins.cc:518 \#8 0x000055f1e66d7d80 in row_undo (node=0xa486405af80, thr=0xa4864042d78) at ./storage/innobase/row/row0undo.cc:298 \#9 0x000055f1e66d7f2d in row_undo_step (thr=0xa4864042d78) at ./storage/innobase/row/row0undo.cc:351 \#10 0x000055f1e663e6a0 in que_thr_step (thr=0xa4864042d78) at ./storage/innobase/que/que0que.cc:1039 \#11 0x000055f1e663e8c1 in que_run_threads_low (thr=0xa4864042d78) at ./storage/innobase/que/que0que.cc:1103 \#12 0x000055f1e663ea73 in que_run_threads (thr=0xa4864042d78) at ./storage/innobase/que/que0que.cc:1143 \#13 0x000055f1e6733cb9 in trx_rollback_to_savepoint_low (trx=0x55522d18d188, savept=0x55522d18e198) at ./storage/innobase/trx/trx0roll.cc:107 \#14 0x000055f1e6733f5f in trx_rollback_to_savepoint (trx=0x55522d18d188, savept=0x55522d18e198) at ./storage/innobase/trx/trx0roll.cc:148 \#15 0x000055f1e6734756 in trx_rollback_last_sql_stat_for_mysql (trx=0x55522d18d188) at ./storage/innobase/trx/trx0roll.cc:281 \#16 0x000055f1e654fb65 in innobase_rollback (hton=0x55f1e8c17968, thd=0xa48640010a8, rollback_trx=false) at ./storage/innobase/handler/ha_innodb.cc:4875 \#17 0x000055f1e636dfb4 in ha_rollback_trans (thd=0xa48640010a8, all=false) at ./sql/handler.cc:1708 \#18 0x000055f1e6262a1b in trans_rollback_stmt (thd=0xa48640010a8) at ./sql/transaction.cc:565 \#19 0x000055f1e611b5e4 in mysql_execute_command (thd=0xa48640010a8) at ./sql/sql_parse.cc:6067 \#20 0x000055f1e611faf6 in mysql_parse (thd=0xa48640010a8, rawbuf=0xa48640109e0 "INSERT INTO t6 (col1,col2, col_int, col_string, col_text) VALUES /* NULL */ (NULL,NULL,NULL,REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), 10),REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), @fill_amount) ), (NULL,N"..., length=347, parser_state=0x663f4cad0670, is_com_multi=false, is_next_command=false) at ./sql/sql_parse.cc:7796 ------------------- 3) purge is invoked, it tries to purge record B, record B has gap lock, but the record next to the record B does not have gap lock, the debug check is failed. But initially on step 1 the acquired lock is not gap lock: ----------------- dberr_t row_ins_duplicate_error_in_clust(...) { ... if (cursor->low_match >= n_unique) { ... if (flags & BTR_NO_LOCKING_FLAG) { /* Do nothing if no-locking is set */ err = DB_SUCCESS; } else if (trx->duplicates) { /* If the SQL-query will update or replace duplicate key we will take X-lock for duplicates ( REPLACE, LOAD DATAFILE REPLACE, INSERT ON DUPLICATE KEY UPDATE). */ err = row_ins_set_exclusive_rec_lock( LOCK_REC_NOT_GAP, btr_cur_get_block(cursor), rec, cursor->index, offsets, thr); } else { err = row_ins_set_shared_rec_lock( LOCK_REC_NOT_GAP, btr_cur_get_block(cursor), rec, cursor->index, offsets, thr); } } ... } ----------------- Then lock_rec_inherit_to_gap() copies this non-gap lock to gap lock to the next record when transaction is rolled back and the record is being deleted with btr_cur_optimistic_delete(). So, rollback converted that into a gap lock, what is wrong, the lock should simply be deleted. For this purpose convert_lock_to_gap flag is added to lock_rec_inherit_to_gap() function arguments. When this flag is not set, lock_rec_inherit_to_gap() ignores non-gap locks, and this flag is set when lock_rec_inherit_to_gap() is invoked from rollback. =============================================================================== II. When locking read is in progress, and requested ordinary-lock can not be granted for delete-marked record due to conflicting lock, mtr is committed, page latch is released, and purge thread can try to purge the record. The debug check will fail as the record next to delete-marked ordinary- locked one is not ordinary-locked. To solve this issue hash-table of scanned record ids(page_id, heap_no pairs) is stored in trx_t. After locking read is finished at the end of row_search_mvcc() and rol_sel(), the hash-table is cleaned-up. When permanent cursor is restored after the lock is granted and the transaction thread is woken up, and the record stored in the cursor is purged, then the position will be set to the previous or next record dependin on the direction of scanning. =============================================================================== Warning: the current implementation for row_sel() is wrong, because the behaviour when there is conflicting lock is not the same as in row_search_mvcc(), i.e. when transaction is suspended/woken up, the execution does not leave row_search_mvcc(), while for row_sel() all necessary steps to suspend/wake-up the thread are executed outside on row_sel(), at the higher layer.

…========== This is the initial patch to show how gap lock is inherited during purge. The tests are for debugging. The code can be used to remove gap lock inherit code from purge process. =============================================================================== Extend gap locks in row_search_mvcc(). This is preliminary code without a good testing. The general logic is the following: 1) Use two directions to extend gap locks - FORWARD and BACKWARD only if "direction" argument of row_search_mvcc() is 0, otherwise use only FORWARD. 2) FORWARD and BACKWARD does not really mean forward or backward iteration through B-tree leafs, FORWARD corresponds to the same direction which was choosen in row_search_mvcc() when "moves_up" variables is set, while BACKWARD means opposite direction. 3) If "direction" argument of row_search_mvcc() is 0 then cursor position is stored in local cursor object before going to the next record in BACKWARD direction. 4) When the first non-delete-marked record is reached in BACKWARD scan, mini-transaction is committed, the cursor position is restored from local cursor object and scan direction is changed. Currently all innodb tests are passed. Things to do: 1) Copy changes in row_sel(), 2) Remove gap lock inheritance from purge process, 3) Add more cases in mtr test: a) spatial indexes b) tests for row_sel() c) ... d) PROFIT!!! Notes for code reviewer: I count on the preliminary review to be sure I am moving in the correct direction and did not make some obvoius errors, so please please don't pay attention on code format and non-full testing. =============================================================================== row_sel() changes =============================================================================== do not inherit gap locks on purge =============================================================================== Test for row_search_mvcc =============================================================================== Foreign keys constraints check fix. The problem of the current fix is that it's complexity is n*m. Because there will be one pass of parent gap for each record in childs gap. Duplicates check is not implemented. =============================================================================== This is a try to implement the way of row_sel() testing. The idea is to have special debug variable innobase_debug_que_eval_sql, when this variable is set, the inernal innodb query parser is invoked, and the result is sent to user. =============================================================================== Add forward scan and insert intention locking for insert operation. Removed backward scan from foreign key contraints check. Removed backward scan from row_search_mvcc(), leave it only for the case of ROW_SEL_EXACT_PREFIX(i.e. for ORDER BY ... DESC). Removed backward scan from rol_sel() (except ORDER BY ... DESC). Added forward scan for secondary indexes duplicates check. =============================================================================== Add debug check. If purgeable record has gap, the next record must has gap too. =============================================================================== Some debug tests. Can be useful for research. =============================================================================== Code cleanup. =============================================================================== Revert "This is a try to implement the way of row_sel() testing." This reverts commit dc33c72c3ba69989e11f377aa902ed9b32f8854a. =============================================================================== Do not set LOCK_REC_NOT_GAP for delete-marked records in row_search_mvcc() =============================================================================== Check if lock_update_delete() is invoked from purge process, the deleted record must be delete-marked. =============================================================================== Do not take into account insert intention locks on debug check. =============================================================================== The current commit solves the following issues: ------------------------------------------------------------------------------- I. If some record is deleted by rollback, it's lock is inherited as gap lock to the next record. And if the next record is then purged while the lock is still held, the debug check will fail. The scenario is the following: 1) Some thread executes "INSERT" and checks clustered index for duplicates, it sets shared lock for checked record(let's call it record A) converting implicit lock to explicit one. Note that the record's transaction id is the same as the current transaction id: ------------------- 0x000055f1e65c6bd6 in lock_rec_create_low (c_lock=0x0, thr=0x0, type_mode=1059, space=11, page_no=3, page=0x2e167079c000 "l\206", <incomplete sequence \372\221>, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, holds_trx_mutex=true) at ./storage/innobase/lock/lock0lock.cc:1466 1466 lock->type_mode = (type_mode & ~LOCK_TYPE_MASK) | LOCK_REC; (rr) bt \#0 0x000055f1e65c6bd6 in lock_rec_create_low (c_lock=0x0, thr=0x0, type_mode=1059, space=11, page_no=3, page=0x2e167079c000 "l\206", <incomplete sequence \372\221>, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, holds_trx_mutex=true) at ./storage/innobase/lock/lock0lock.cc:1466 \#1 0x000055f1e65c2b29 in lock_rec_create (c_lock=0x0, thr=0x0, type_mode=1059, block=0x2e1670080560, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=true) at ./storage/innobase/include/lock0lock.ic:133 \#2 0x000055f1e65c83fe in lock_rec_add_to_queue (type_mode=1059, block=0x2e1670080560, heap_no=9, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=true) at ./storage/innobase/lock/lock0lock.cc:1941 \#3 0x000055f1e65d228f in lock_rec_convert_impl_to_expl_for_trx (block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, trx=0x55522d18d188, heap_no=9) at ./storage/innobase/lock/lock0lock.cc:5832 \#4 0x000055f1e65d2537 in lock_rec_convert_impl_to_expl (block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, offsets=0x663f4cace6d0) at ./storage/innobase/lock/lock0lock.cc:5886 \#5 0x000055f1e65d32d9 in lock_clust_rec_read_check_and_lock (flags=0, block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, offsets=0x663f4cace6d0, mode=LOCK_S, gap_mode=1024, thr=0x611370025b88) at ./storage/innobase/lock/lock0lock.cc:6194 \#6 0x000055f1e666b03c in row_ins_set_shared_rec_lock (type=1024, block=0x2e1670080560, rec=0x2e167079c299 "\200", index=0x149444393cd0, offsets=0x663f4cace6d0, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:1427 \#7 0x000055f1e666cf4f in row_ins_duplicate_error_in_clust (flags=0, cursor=0x663f4cace9e0, entry=0x61137c044900, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:2360 \#8 0x000055f1e666db48 in row_ins_clust_index_entry_low (flags=0, mode=2, index=0x149444393cd0, n_uniq=1, entry=0x61137c044900, n_ext=0, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:2658 \#9 0x000055f1e666f0cb in row_ins_clust_index_entry (index=0x149444393cd0, entry=0x61137c044900, thr=0x611370025b88, n_ext=0) at ./storage/innobase/row/row0ins.cc:3146 \#10 0x000055f1e666f4a8 in row_ins_index_entry (index=0x149444393cd0, entry=0x61137c044900, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3265 \#11 0x000055f1e666f9a4 in row_ins_index_entry_step (node=0x611370025658, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3416 \#12 0x000055f1e666fd2a in row_ins (node=0x611370025658, thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3553 \#13 0x000055f1e66700c6 in row_ins_step (thr=0x611370025b88) at ./storage/innobase/row/row0ins.cc:3677 \#14 0x000055f1e668c7e0 in row_insert_for_mysql (mysql_rec=0x61137005c460 "\377", prebuilt=0x611370024d50) at ./storage/innobase/row/row0mysql.cc:1408 \#15 0x000055f1e6556c1f in ha_innobase::write_row (this=0x611370033a00, record=0x61137005c460 "\377") at ./storage/innobase/handler/ha_innodb.cc:8284 \#16 0x000055f1e6378051 in handler::ha_write_row (this=0x611370033a00, buf=0x61137005c460 "\377") at ./sql/handler.cc:6118 \#17 0x000055f1e60f21c4 in write_record (thd=0xa48640010a8, table=0x61137005b8c8, info=0x663f4cacfa00) at ./sql/sql_insert.cc:1939 \#18 0x000055f1e60f00f4 in mysql_insert (thd=0xa48640010a8, table_list=0xa4864010d40, fields=..., values_list=..., update_fields=..., update_values=..., duplic=DUP_ERROR, ignore=false) at ./sql/sql_insert.cc:1066 \#19 0x000055f1e61149d7 in mysql_execute_command (thd=0xa48640010a8) at ./sql/sql_parse.cc:4220 \#20 0x000055f1e611faf6 in mysql_parse (thd=0xa48640010a8, rawbuf=0xa48640109e0 "INSERT INTO t6 (col1,col2, col_int, col_string, col_text) VALUES /* NULL */ (NULL,NULL,NULL,REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), 10),REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), @fill_amount) ), (NULL,N"..., length=347, parser_state=0x663f4cad0670, is_com_multi=false, is_next_command=false) at ./sql/sql_parse.cc:7796 ------------------- 2) Then duplicate key is found in row_ins_duplicate_error_in_clust(), and the transaction is rolled back. When it's rolled back, the lock is inherited to the next record(let's call it record B) as a gap lock: ------------------- \#0 lock_rec_create_low (c_lock=0x0, thr=0x0, type_mode=547, space=11, page_no=3, page=0x2e167079c000 "l\206", <incomplete sequence \372\221>, heap_no=33, index=0x149444393cd0, trx=0x55522d18d188, holds_trx_mutex=false) at ./storage/innobase/lock/lock0lock.cc:1467 \#1 0x000055f1e65c2b29 in lock_rec_create (c_lock=0x0, thr=0x0, type_mode=547, block=0x2e1670080560, heap_no=33, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=false) at ./storage/innobase/include/lock0lock.ic:133 \#2 0x000055f1e65c83fe in lock_rec_add_to_queue (type_mode=547, block=0x2e1670080560, heap_no=33, index=0x149444393cd0, trx=0x55522d18d188, caller_owns_trx_mutex=false) at ./storage/innobase/lock/lock0lock.cc:1941 \#3 0x000055f1e65c9fee in lock_rec_inherit_to_gap (heir_block=0x2e1670080560, block=0x2e1670080560, heir_heap_no=33, heap_no=9) at ./storage/innobase/lock/lock0lock.cc:2580 \#4 0x000055f1e65cc057 in lock_update_delete (block=0x2e1670080560, rec=0x2e167079c299 "\200", from_purge=false) at ./storage/innobase/lock/lock0lock.cc:3559 \#5 0x000055f1e6788e57 in btr_cur_optimistic_delete (cursor=0xa486405aff0, flags=0, mtr=0x663f4cacf250, from_purge=false) at ./storage/innobase/btr/btr0cur.cc:5252 \#6 0x000055f1e68af6aa in row_undo_ins_remove_clust_rec (node=0xa486405af80) at ./storage/innobase/row/row0uins.cc:141 \#7 0x000055f1e68b05b5 in row_undo_ins (node=0xa486405af80, thr=0xa4864042d78) at ./storage/innobase/row/row0uins.cc:518 \#8 0x000055f1e66d7d80 in row_undo (node=0xa486405af80, thr=0xa4864042d78) at ./storage/innobase/row/row0undo.cc:298 \#9 0x000055f1e66d7f2d in row_undo_step (thr=0xa4864042d78) at ./storage/innobase/row/row0undo.cc:351 \#10 0x000055f1e663e6a0 in que_thr_step (thr=0xa4864042d78) at ./storage/innobase/que/que0que.cc:1039 \#11 0x000055f1e663e8c1 in que_run_threads_low (thr=0xa4864042d78) at ./storage/innobase/que/que0que.cc:1103 \#12 0x000055f1e663ea73 in que_run_threads (thr=0xa4864042d78) at ./storage/innobase/que/que0que.cc:1143 \#13 0x000055f1e6733cb9 in trx_rollback_to_savepoint_low (trx=0x55522d18d188, savept=0x55522d18e198) at ./storage/innobase/trx/trx0roll.cc:107 \#14 0x000055f1e6733f5f in trx_rollback_to_savepoint (trx=0x55522d18d188, savept=0x55522d18e198) at ./storage/innobase/trx/trx0roll.cc:148 \#15 0x000055f1e6734756 in trx_rollback_last_sql_stat_for_mysql (trx=0x55522d18d188) at ./storage/innobase/trx/trx0roll.cc:281 \#16 0x000055f1e654fb65 in innobase_rollback (hton=0x55f1e8c17968, thd=0xa48640010a8, rollback_trx=false) at ./storage/innobase/handler/ha_innodb.cc:4875 \#17 0x000055f1e636dfb4 in ha_rollback_trans (thd=0xa48640010a8, all=false) at ./sql/handler.cc:1708 \#18 0x000055f1e6262a1b in trans_rollback_stmt (thd=0xa48640010a8) at ./sql/transaction.cc:565 \#19 0x000055f1e611b5e4 in mysql_execute_command (thd=0xa48640010a8) at ./sql/sql_parse.cc:6067 \#20 0x000055f1e611faf6 in mysql_parse (thd=0xa48640010a8, rawbuf=0xa48640109e0 "INSERT INTO t6 (col1,col2, col_int, col_string, col_text) VALUES /* NULL */ (NULL,NULL,NULL,REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), 10),REPEAT(SUBSTR(CAST( NULL AS CHAR),1,1), @fill_amount) ), (NULL,N"..., length=347, parser_state=0x663f4cad0670, is_com_multi=false, is_next_command=false) at ./sql/sql_parse.cc:7796 ------------------- 3) purge is invoked, it tries to purge record B, record B has gap lock, but the record next to the record B does not have gap lock, the debug check is failed. But initially on step 1 the acquired lock is not gap lock: ----------------- dberr_t row_ins_duplicate_error_in_clust(...) { ... if (cursor->low_match >= n_unique) { ... if (flags & BTR_NO_LOCKING_FLAG) { /* Do nothing if no-locking is set */ err = DB_SUCCESS; } else if (trx->duplicates) { /* If the SQL-query will update or replace duplicate key we will take X-lock for duplicates ( REPLACE, LOAD DATAFILE REPLACE, INSERT ON DUPLICATE KEY UPDATE). */ err = row_ins_set_exclusive_rec_lock( LOCK_REC_NOT_GAP, btr_cur_get_block(cursor), rec, cursor->index, offsets, thr); } else { err = row_ins_set_shared_rec_lock( LOCK_REC_NOT_GAP, btr_cur_get_block(cursor), rec, cursor->index, offsets, thr); } } ... } ----------------- Then lock_rec_inherit_to_gap() copies this non-gap lock to gap lock to the next record when transaction is rolled back and the record is being deleted with btr_cur_optimistic_delete(). So, rollback converted that into a gap lock, what is wrong, the lock should simply be deleted. For this purpose convert_lock_to_gap flag is added to lock_rec_inherit_to_gap() function arguments. When this flag is not set, lock_rec_inherit_to_gap() ignores non-gap locks, and this flag is set when lock_rec_inherit_to_gap() is invoked from rollback. ------------------------------------------------------------------------------- II. When locking read is in progress, and requested ordinary-lock can not be granted for delete-marked record due to conflicting lock, mtr is committed, page latch is released, and purge thread can try to purge the record. The debug check will fail as the record next to delete-marked ordinary- locked one is not ordinary-locked. To solve this issue hash-table of scanned record ids(page_id, heap_no pairs) is stored in trx_t. After locking read is finished at the end of row_search_mvcc() and rol_sel(), the hash-table is cleaned-up. When permanent cursor is restored after the lock is granted and the transaction thread is woken up, and the record stored in the cursor is purged, then the position will be set to the previous or next record dependin on the direction of scanning. ------------------------------------------------------------------------------- Warning: the current implementation for row_sel() is wrong, because the behaviour when there is conflicting lock is not the same as in row_search_mvcc(), i.e. when transaction is suspended/woken up, the execution does not leave row_search_mvcc(), while for row_sel() all necessary steps to suspend/wake-up the thread are executed outside on row_sel(), at the higher layer. =============================================================================== The new system variable is added to test row_sel(). Some initial test is also added.

Read of size 8 at 0x7fecf2e75fc8 by thread T2 (mutexes: write M1318): #0 tpool::thread_pool_generic::submit_task(tpool::task*) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../tpool/tpool_generic.cc:823:9 (mariadbd+0x25fd2d2) #1 (anonymous namespace)::aio_uring::thread_routine((anonymous namespace)::aio_uring*) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../tpool/aio_liburing.cc:173:20 (mariadbd+0x260b21b) #2 void std::__invoke_impl<void, void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>(std::__invoke_other, void (*&&)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x260c62a) #3 std::__invoke_result<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>::type std::__invoke<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>(void (*&&)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x260c4ba) #4 void std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x260c442) #5 std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x260c3c5) #6 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x260c189) #7 <null> <null> (libstdc++.so.6+0xd230f) Previous write of size 8 at 0x7fecf2e75fc8 by main thread: #0 tpool::task::task(void (*)(void*), void*, tpool::task_group*) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../tpool/task.cc:40:46 (mariadbd+0x260a138) #1 tpool::aiocb::aiocb() /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../tpool/tpool.h:147:13 (mariadbd+0x2355943) #2 void std::_Construct<tpool::aiocb>(tpool::aiocb*) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:109:38 (mariadbd+0x2355845) #3 tpool::aiocb* std::__uninitialized_default_n_1<false>::__uninit_default_n<tpool::aiocb*, unsigned long>(tpool::aiocb*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:579:3 (mariadbd+0x235576c) #4 tpool::aiocb* std::__uninitialized_default_n<tpool::aiocb*, unsigned long>(tpool::aiocb*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:638:14 (mariadbd+0x23556e9) #5 tpool::aiocb* std::__uninitialized_default_n_a<tpool::aiocb*, unsigned long, tpool::aiocb>(tpool::aiocb*, unsigned long, std::allocator<tpool::aiocb>&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:704:14 (mariadbd+0x2355641) #6 std::vector<tpool::aiocb, std::allocator<tpool::aiocb> >::_M_default_initialize(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1606:4 (mariadbd+0x2354f3d) #7 std::vector<tpool::aiocb, std::allocator<tpool::aiocb> >::vector(unsigned long, std::allocator<tpool::aiocb> const&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:512:9 (mariadbd+0x2354a19) #8 tpool::cache<tpool::aiocb>::cache(unsigned long, tpool::cache_notification_mode) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../tpool/tpool_structs.h:73:20 (mariadbd+0x2354784) #9 io_slots::io_slots(int, int) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../storage/innobase/os/os0file.cc:93:3 (mariadbd+0x235343b) #10 os_aio_init() /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../storage/innobase/os/os0file.cc:3780:22 (mariadbd+0x234ebce) #11 srv_start(bool) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../storage/innobase/srv/srv0start.cc:1190:6 (mariadbd+0x256720c) #12 innodb_init(void*) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../storage/innobase/handler/ha_innodb.cc:4188:8 (mariadbd+0x1ed3bda) #13 ha_initialize_handlerton(st_plugin_int*) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../sql/handler.cc:659:31 (mariadbd+0xf7be06) #14 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../sql/sql_plugin.cc:1463:9 (mariadbd+0x160fa1b) #15 plugin_init(int*, char**, int) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../sql/sql_plugin.cc:1756:15 (mariadbd+0x160f07f) #16 init_server_components() /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../sql/mysqld.cc:5043:7 (mariadbd+0xd70fb2) #17 mysqld_main(int, char**) /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../sql/mysqld.cc:5655:7 (mariadbd+0xd6a9d7) #18 main /home/kevgs/work/m/bb-10.6-kevgs/build_tsan/../sql/main.cc:34:10 (mariadbd+0xd65d18) Here T2 accesses tpool::task while the main thread still initializes it! aio_uring accesses io_slots and thus io_slots should be initialized before it. So, fixing by changing the order or initialization.

Read of size 8 at 0x7fecf2e75fc8 by thread T2 (mutexes: write M1318): #0 tpool::thread_pool_generic::submit_task(tpool::task*) /tpool/tpool_generic.cc:823:9 (mariadbd+0x25fd2d2) #1 (anonymous namespace)::aio_uring::thread_routine((anonymous namespace)::aio_uring*) /tpool/aio_liburing.cc:173:20 (mariadbd+0x260b21b) #2 void std::__invoke_impl<void, void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>(std::__invoke_other, void (*&&)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x260c62a) #3 std::__invoke_result<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>::type std::__invoke<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*>(void (*&&)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x260c4ba) #4 void std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x260c442) #5 std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x260c3c5) #6 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)((anonymous namespace)::aio_uring*), (anonymous namespace)::aio_uring*> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x260c189) #7 <null> <null> (libstdc++.so.6+0xd230f) Previous write of size 8 at 0x7fecf2e75fc8 by main thread: #0 tpool::task::task(void (*)(void*), void*, tpool::task_group*) /tpool/task.cc:40:46 (mariadbd+0x260a138) #1 tpool::aiocb::aiocb() /tpool/tpool.h:147:13 (mariadbd+0x2355943) #2 void std::_Construct<tpool::aiocb>(tpool::aiocb*) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:109:38 (mariadbd+0x2355845) #3 tpool::aiocb* std::__uninitialized_default_n_1<false>::__uninit_default_n<tpool::aiocb*, unsigned long>(tpool::aiocb*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:579:3 (mariadbd+0x235576c) #4 tpool::aiocb* std::__uninitialized_default_n<tpool::aiocb*, unsigned long>(tpool::aiocb*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:638:14 (mariadbd+0x23556e9) #5 tpool::aiocb* std::__uninitialized_default_n_a<tpool::aiocb*, unsigned long, tpool::aiocb>(tpool::aiocb*, unsigned long, std::allocator<tpool::aiocb>&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_uninitialized.h:704:14 (mariadbd+0x2355641) #6 std::vector<tpool::aiocb, std::allocator<tpool::aiocb> >::_M_default_initialize(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:1606:4 (mariadbd+0x2354f3d) #7 std::vector<tpool::aiocb, std::allocator<tpool::aiocb> >::vector(unsigned long, std::allocator<tpool::aiocb> const&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_vector.h:512:9 (mariadbd+0x2354a19) #8 tpool::cache<tpool::aiocb>::cache(unsigned long, tpool::cache_notification_mode) /tpool/tpool_structs.h:73:20 (mariadbd+0x2354784) #9 io_slots::io_slots(int, int) /storage/innobase/os/os0file.cc:93:3 (mariadbd+0x235343b) #10 os_aio_init() /storage/innobase/os/os0file.cc:3780:22 (mariadbd+0x234ebce) #11 srv_start(bool) /storage/innobase/srv/srv0start.cc:1190:6 (mariadbd+0x256720c) #12 innodb_init(void*) /storage/innobase/handler/ha_innodb.cc:4188:8 (mariadbd+0x1ed3bda) #13 ha_initialize_handlerton(st_plugin_int*) /sql/handler.cc:659:31 (mariadbd+0xf7be06) #14 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) /sql/sql_plugin.cc:1463:9 (mariadbd+0x160fa1b) #15 plugin_init(int*, char**, int) /sql/sql_plugin.cc:1756:15 (mariadbd+0x160f07f) #16 init_server_components() /sql/mysqld.cc:5043:7 (mariadbd+0xd70fb2) #17 mysqld_main(int, char**) /sql/mysqld.cc:5655:7 (mariadbd+0xd6a9d7) #18 main /sql/main.cc:34:10 (mariadbd+0xd65d18) I think the report is incorrect: it's not possible to have such a race condition. I've checked it by reading the code and putting assertions. Namely, no aio I/O is possible before the end of os_aio_init(). Most probably it's some bug in TSAN. But the patch fixes around 5 related reports and this is a step toward TSAN usefullness. Currently it reports too much noise. std::unique_ptr is a step toward https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#r11-avoid-calling-new-and-delete-explicitly There is no std::make_unique() in C++11, however.

WARNING: ThreadSanitizer: data race (pid=1503350) Write of size 8 at 0x0000067b1f20 by thread T3: #0 os_file_sync_posix(int) /storage/innobase/os/os0file.cc:895:5 (mariadbd+0x23493f6) #1 os_file_flush_func(int) /storage/innobase/os/os0file.cc:983:8 (mariadbd+0x2349204) #2 file_os_io::flush() /storage/innobase/log/log0log.cc:326:10 (mariadbd+0x22eaaa9) #3 log_file_t::flush() /storage/innobase/log/log0log.cc:440:18 (mariadbd+0x22eb2d0) #4 log_t::file::flush() /storage/innobase/log/log0log.cc:507:29 (mariadbd+0x22ebe69) #5 log_write_flush_to_disk_low(unsigned long) /storage/innobase/log/log0log.cc:629:17 (mariadbd+0x22ed3f3) #6 log_write_up_to(unsigned long, bool, bool, completion_callback const*) /storage/innobase/log/log0log.cc:829:3 (mariadbd+0x22ecb04) #7 log_checkpoint_low(unsigned long, unsigned long) /storage/innobase/buf/buf0flu.cc:1734:5 (mariadbd+0x20d37f1) #8 buf_flush_sync_for_checkpoint(unsigned long) /storage/innobase/buf/buf0flu.cc:1947:7 (mariadbd+0x20d4193) #9 buf_flush_page_cleaner() /storage/innobase/buf/buf0flu.cc:2186:9 (mariadbd+0x20cdad7) #10 void std::__invoke_impl<void, void (*)()>(std::__invoke_other, void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x20c3aaa) #11 std::__invoke_result<void (*)()>::type std::__invoke<void (*)()>(void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x20c39bd) #12 void std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x20c3965) #13 std::thread::_Invoker<std::tuple<void (*)()> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x20c3905) #14 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x20c37f9) #15 <null> <null> (libstdc++.so.6+0xd230f) Previous write of size 8 at 0x0000067b1f20 by main thread: #0 os_file_sync_posix(int) /storage/innobase/os/os0file.cc:895:5 (mariadbd+0x23493f6) #1 os_file_flush_func(int) /storage/innobase/os/os0file.cc:983:8 (mariadbd+0x2349204) #2 fil_space_t::flush_low() /storage/innobase/fil/fil0fil.cc:504:5 (mariadbd+0x205cad5) #3 fil_flush_file_spaces() /storage/innobase/fil/fil0fil.cc:2947:13 (mariadbd+0x206523f) #4 log_checkpoint() /storage/innobase/buf/buf0flu.cc:1777:5 (mariadbd+0x20cd069) #5 buf_flush_wait_flushed(unsigned long) /storage/innobase/buf/buf0flu.cc:1867:5 (mariadbd+0x20ccf95) #6 log_make_checkpoint() /storage/innobase/buf/buf0flu.cc:1793:3 (mariadbd+0x20cc4c9) #7 buf_dblwr_t::create() /storage/innobase/buf/buf0dblwr.cc:216:3 (mariadbd+0x209076a) #8 srv_start(bool) /storage/innobase/srv/srv0start.cc:1685:20 (mariadbd+0x256b514) #9 innodb_init(void*) /storage/innobase/handler/ha_innodb.cc:4188:8 (mariadbd+0x1ed406a) #10 ha_initialize_handlerton(st_plugin_int*) /sql/handler.cc:659:31 (mariadbd+0xf7c246) #11 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) /sql/sql_plugin.cc:1463:9 (mariadbd+0x160fe6b) #12 plugin_init(int*, char**, int) /sql/sql_plugin.cc:1756:15 (mariadbd+0x160f4cf) #13 init_server_components() /sql/mysqld.cc:5043:7 (mariadbd+0xd713f2) #14 mysqld_main(int, char**) /sql/mysqld.cc:5655:7 (mariadbd+0xd6ae17) #15 main /sql/main.cc:34:10 (mariadbd+0xd66158) This is a correct report by TSAN for an obvious case: unprotected global counter. Fix it by making counter std::atomic.

WARNING: ThreadSanitizer: data race (pid=1506937) Write of size 8 at 0x0000067ab740 by thread T6: #0 buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:2946:8 (mariadbd+0x2014c7f) #1 buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:3047:10 (mariadbd+0x2016216) #2 btr_cur_search_to_nth_level_func(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, ssux_lock_impl<true>*, mtr_t*, unsigned long) /storage/innobase/btr/btr0cur.cc:1613:10 (mariadbd+0x1fb5bff) #3 btr_pcur_open_low(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_pcur_t*, unsigned long, mtr_t*) /storage/innobase/include/btr0pcur.ic:439:8 (mariadbd+0x24ddead) #4 row_search_on_row_ref(btr_pcur_t*, unsigned long, dict_table_t const*, dtuple_t const*, mtr_t*) /storage/innobase/row/row0row.cc:1215:7 (mariadbd+0x24dd537) #5 row_purge_reposition_pcur(unsigned long, purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:81:23 (mariadbd+0x24c5369) #6 row_purge_reset_trx_id(purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:748:6 (mariadbd+0x24c90c7) #7 row_purge_record_func(purge_node_t*, unsigned char*, que_thr_t const*, bool) /storage/innobase/row/row0purge.cc:1174:4 (mariadbd+0x24c8262) #8 row_purge(purge_node_t*, unsigned char*, que_thr_t*) /storage/innobase/row/row0purge.cc:1218:18 (mariadbd+0x24c5af3) #9 row_purge_step(que_thr_t*) /storage/innobase/row/row0purge.cc:1267:3 (mariadbd+0x24c5996) #10 que_thr_step(que_thr_t*) /storage/innobase/que/que0que.cc:653:9 (mariadbd+0x23d5298) #11 que_run_threads_low(que_thr_t*) /storage/innobase/que/que0que.cc:709:25 (mariadbd+0x23d3f29) #12 que_run_threads(que_thr_t*) /storage/innobase/que/que0que.cc:729:2 (mariadbd+0x23d3bdf) #13 srv_task_execute() /storage/innobase/srv/srv0srv.cc:1692:3 (mariadbd+0x2562841) #14 purge_worker_callback(void*) /storage/innobase/srv/srv0srv.cc:1864:10 (mariadbd+0x255f361) #15 tpool::task_group::execute(tpool::task*) /tpool/task_group.cc:55:9 (mariadbd+0x260a5ca) #16 tpool::task::execute() /tpool/task.cc:47:16 (mariadbd+0x260adf6) #17 tpool::thread_pool_generic::worker_main(tpool::worker_data*) /tpool/tpool_generic.cc:550:11 (mariadbd+0x25fc590) #18 void std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(std::__invoke_memfun_deref, void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74:14 (mariadbd+0x26061b5) #19 std::__invoke_result<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>::type std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x2605f57) #20 void std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x2605ecb) #21 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x2605e35) #22 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x2605ac9) #23 <null> <null> (libstdc++.so.6+0xd230f) Previous write of size 8 at 0x0000067ab740 by thread T8: #0 buf_page_get_low(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:2946:8 (mariadbd+0x2014c7f) #1 buf_page_get_gen(page_id_t, unsigned long, unsigned long, buf_block_t*, unsigned long, mtr_t*, dberr_t*, bool) /storage/innobase/buf/buf0buf.cc:3047:10 (mariadbd+0x2016216) #2 btr_cur_search_to_nth_level_func(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_cur_t*, ssux_lock_impl<true>*, mtr_t*, unsigned long) /storage/innobase/btr/btr0cur.cc:1613:10 (mariadbd+0x1fb5bff) #3 btr_pcur_open_low(dict_index_t*, unsigned long, dtuple_t const*, page_cur_mode_t, unsigned long, btr_pcur_t*, unsigned long, mtr_t*) /storage/innobase/include/btr0pcur.ic:439:8 (mariadbd+0x24ddead) #4 row_search_on_row_ref(btr_pcur_t*, unsigned long, dict_table_t const*, dtuple_t const*, mtr_t*) /storage/innobase/row/row0row.cc:1215:7 (mariadbd+0x24dd537) #5 row_purge_reposition_pcur(unsigned long, purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:81:23 (mariadbd+0x24c5369) #6 row_purge_reset_trx_id(purge_node_t*, mtr_t*) /storage/innobase/row/row0purge.cc:748:6 (mariadbd+0x24c90c7) #7 row_purge_record_func(purge_node_t*, unsigned char*, que_thr_t const*, bool) /storage/innobase/row/row0purge.cc:1174:4 (mariadbd+0x24c8262) #8 row_purge(purge_node_t*, unsigned char*, que_thr_t*) /storage/innobase/row/row0purge.cc:1218:18 (mariadbd+0x24c5af3) #9 row_purge_step(que_thr_t*) /storage/innobase/row/row0purge.cc:1267:3 (mariadbd+0x24c5996) #10 que_thr_step(que_thr_t*) /storage/innobase/que/que0que.cc:653:9 (mariadbd+0x23d5298) #11 que_run_threads_low(que_thr_t*) /storage/innobase/que/que0que.cc:709:25 (mariadbd+0x23d3f29) #12 que_run_threads(que_thr_t*) /storage/innobase/que/que0que.cc:729:2 (mariadbd+0x23d3bdf) #13 trx_purge(unsigned long, bool) /storage/innobase/trx/trx0purge.cc:1271:2 (mariadbd+0x25841b4) #14 srv_do_purge(unsigned long*) /storage/innobase/srv/srv0srv.cc:1784:20 (mariadbd+0x2563224) #15 purge_coordinator_callback_low() /storage/innobase/srv/srv0srv.cc:1881:35 (mariadbd+0x2562b3b) #16 purge_coordinator_callback(void*) /storage/innobase/srv/srv0srv.cc:1910:3 (mariadbd+0x255f4ab) #17 tpool::task_group::execute(tpool::task*) /tpool/task_group.cc:55:9 (mariadbd+0x260a5ca) #18 tpool::task::execute() /tpool/task.cc:47:16 (mariadbd+0x260adf6) #19 tpool::thread_pool_generic::worker_main(tpool::worker_data*) /tpool/tpool_generic.cc:550:11 (mariadbd+0x25fc590) #20 void std::__invoke_impl<void, void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(std::__invoke_memfun_deref, void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:74:14 (mariadbd+0x26061b5) #21 std::__invoke_result<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>::type std::__invoke<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*>(void (tpool::thread_pool_generic::*&&)(tpool::worker_data*), tpool::thread_pool_generic*&&, tpool::worker_data*&&) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x2605f57) #22 void std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::_M_invoke<0ul, 1ul, 2ul>(std::_Index_tuple<0ul, 1ul, 2ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x2605ecb) #23 std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x2605e35) #24 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (tpool::thread_pool_generic::*)(tpool::worker_data*), tpool::thread_pool_generic*, tpool::worker_data*> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x2605ac9) #25 <null> <null> (libstdc++.so.6+0xd230f) Location is global 'buf_dbg_counter' of size 8 at 0x0000067ab740 (mariadbd+0x67ab740) The obvious fix is to make counter atomic.

Write of size 1 at 0x0000067abe08 by thread T3 (mutexes: write M1372): #0 buf_flush_page_cleaner() /storage/innobase/buf/buf0flu.cc:2366:29 (mariadbd+0x20cea7c) #1 void std::__invoke_impl<void, void (*)()>(std::__invoke_other, void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x20c3a8a) #2 std::__invoke_result<void (*)()>::type std::__invoke<void (*)()>(void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x20c399d) #3 void std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x20c3945) #4 std::thread::_Invoker<std::tuple<void (*)()> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x20c38e5) #5 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x20c37d9) #6 <null> <null> (libstdc++.so.6+0xd230f) Previous read of size 1 at 0x0000067abe08 by main thread: #0 logs_empty_and_mark_files_at_shutdown() /storage/innobase/log/log0log.cc:1094:6 (mariadbd+0x22eeff3) #1 innodb_shutdown() /storage/innobase/srv/srv0start.cc:1970:3 (mariadbd+0x256ffd6) #2 innobase_end(handlerton*, ha_panic_function) /storage/innobase/handler/ha_innodb.cc:4265:3 (mariadbd+0x1ee3fc4) #3 ha_finalize_handlerton(st_plugin_int*) /sql/handler.cc:595:5 (mariadbd+0xf7bac9) #4 plugin_deinitialize(st_plugin_int*, bool) /sql/sql_plugin.cc:1266:9 (mariadbd+0x1611789) #5 reap_plugins() /sql/sql_plugin.cc:1342:7 (mariadbd+0x160e17d) #6 plugin_shutdown() /sql/sql_plugin.cc:2050:7 (mariadbd+0x1611f42) #7 clean_up(bool) /sql/mysqld.cc:1923:3 (mariadbd+0xd67a4c) #8 unireg_abort /sql/mysqld.cc:1835:3 (mariadbd+0xd67605) #9 mysqld_main(int, char**) /sql/mysqld.cc:5741:7 (mariadbd+0xd6b36a) #10 main /sql/main.cc:34:10 (mariadbd+0xd661a8) Location is global 'buf_page_cleaner_is_active' of size 1 at 0x0000067abe08 (mariadbd+0x67abe08)

WARNING: ThreadSanitizer: data race (pid=1510842) Write of size 8 at 0x0000067b1e98 by main thread: #0 os_file_pwrite(IORequest const&, int, unsigned char const*, unsigned long, unsigned long, dberr_t*) /storage/innobase/os/os0file.cc:2928:2 (mariadbd+0x234c5ac) #1 os_file_write_func(IORequest const&, char const*, int, void const*, unsigned long, unsigned long) /storage/innobase/os/os0file.cc:2963:20 (mariadbd+0x234c019) #2 file_os_io::write(char const*, unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:320:10 (mariadbd+0x22eaa50) #3 log_file_t::write(unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:434:18 (mariadbd+0x22eb1d8) #4 log_t::file::write(unsigned long, st_::span<unsigned char>) /storage/innobase/log/log0log.cc:496:29 (mariadbd+0x22ebb55) #5 log_write_buf(unsigned char*, unsigned long, unsigned long, unsigned long, unsigned long) /storage/innobase/log/log0log.cc:614:14 (mariadbd+0x22f1b51) #6 log_write(bool) /storage/innobase/log/log0log.cc:755:2 (mariadbd+0x22ed2ec) #7 log_write_up_to(unsigned long, bool, bool, completion_callback const*) /storage/innobase/log/log0log.cc:817:5 (mariadbd+0x22eca44) #8 log_checkpoint_low(unsigned long, unsigned long) /storage/innobase/buf/buf0flu.cc:1734:5 (mariadbd+0x20d37c1) #9 log_checkpoint() /storage/innobase/buf/buf0flu.cc:1787:10 (mariadbd+0x20cd155) #10 buf_flush_wait_flushed(unsigned long) /storage/innobase/buf/buf0flu.cc:1867:5 (mariadbd+0x20ccf8f) #11 log_make_checkpoint() /storage/innobase/buf/buf0flu.cc:1793:3 (mariadbd+0x20cc4c9) #12 buf_dblwr_t::create() /storage/innobase/buf/buf0dblwr.cc:216:3 (mariadbd+0x209076a) #13 srv_start(bool) /storage/innobase/srv/srv0start.cc:1685:20 (mariadbd+0x256b4aa) #14 innodb_init(void*) /storage/innobase/handler/ha_innodb.cc:4188:8 (mariadbd+0x1ed40da) #15 ha_initialize_handlerton(st_plugin_int*) /sql/handler.cc:659:31 (mariadbd+0xf7c2b6) #16 plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool) /sql/sql_plugin.cc:1463:9 (mariadbd+0x160fedb) #17 plugin_init(int*, char**, int) /sql/sql_plugin.cc:1756:15 (mariadbd+0x160f53f) #18 init_server_components() /sql/mysqld.cc:5043:7 (mariadbd+0xd71462) #19 mysqld_main(int, char**) /sql/mysqld.cc:5655:7 (mariadbd+0xd6ae87) #20 main /sql/main.cc:34:10 (mariadbd+0xd661c8) Previous write of size 8 at 0x0000067b1e98 by thread T3: #0 os_file_pwrite(IORequest const&, int, unsigned char const*, unsigned long, unsigned long, dberr_t*) /storage/innobase/os/os0file.cc:2928:2 (mariadbd+0x234c5ac) #1 os_file_write_func(IORequest const&, char const*, int, void const*, unsigned long, unsigned long) /storage/innobase/os/os0file.cc:2963:20 (mariadbd+0x234c019) #2 file_os_io::write(char const*, unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:320:10 (mariadbd+0x22eaa50) #3 log_file_t::write(unsigned long, st_::span<unsigned char const>) /storage/innobase/log/log0log.cc:434:18 (mariadbd+0x22eb1d8) #4 log_t::file::write(unsigned long, st_::span<unsigned char>) /storage/innobase/log/log0log.cc:496:29 (mariadbd+0x22ebb55) #5 log_write_checkpoint_info(unsigned long) /storage/innobase/log/log0log.cc:911:14 (mariadbd+0x22edd4e) #6 log_checkpoint_low(unsigned long, unsigned long) /storage/innobase/buf/buf0flu.cc:1755:3 (mariadbd+0x20d3a3d) #7 buf_flush_sync_for_checkpoint(unsigned long) /storage/innobase/buf/buf0flu.cc:1947:7 (mariadbd+0x20d4163) #8 buf_flush_page_cleaner() /storage/innobase/buf/buf0flu.cc:2186:9 (mariadbd+0x20cdab1) #9 void std::__invoke_impl<void, void (*)()>(std::__invoke_other, void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 (mariadbd+0x20c3aaa) #10 std::__invoke_result<void (*)()>::type std::__invoke<void (*)()>(void (*&&)()) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:96:14 (mariadbd+0x20c39bd) #11 void std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:253:13 (mariadbd+0x20c3965) #12 std::thread::_Invoker<std::tuple<void (*)()> >::operator()() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:260:11 (mariadbd+0x20c3905) #13 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() /usr/lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_thread.h:211:13 (mariadbd+0x20c37f9) #14 <null> <null> (libstdc++.so.6+0xd230f) Location is global 'os_n_file_writes' of size 8 at 0x0000067b1e98 (mariadbd+0x67b1e98) Make variable atomic.

Adding mysqld_multi_confd.patch, makes mysqld_multi reading conf.d

There seem to be 2 ASAN issues using mysqltest.cc (at least using test binlog.binlog_autocommit_off_no_hang): 1. (Fixed by this test) cur_con is not NULLed when freeing connections. At backtrace time, it can be read (though the backtrace is likely caused by point 2). 2. (Still to be fixed) There is a leak in mariadb_lib.c line 3863: ``` OPT_SET_EXTENDED_VALUE(&mysql->options, tls_verification_callback, arg1); ``` with stack ================================================================= ==288928==ERROR: LeakSanitizer: detected memory leaks Direct leak of 1568 byte(s) in 4 object(s) allocated from: #0 0x5ceaa4ebad7d in calloc (./server/build116_asan/client/mariadb-test+0x1dcd7d) (BuildId: 21b3097a37beada873a1eaa15c1ea1f16d3ad6d7) #1 0x5ceaa4f6457f in mysql_optionsv ./server/libmariadb/libmariadb/mariadb_lib.c:3863:7 #2 0x5ceaa4f698fd in mysql_init ./server/libmariadb/libmariadb/mariadb_lib.c:1320:3 #3 0x5ceaa4f5a06f in mariadb_reconnect ./server/libmariadb/libmariadb/mariadb_lib.c:2104:3 #4 0x5ceaa4f58c8b in mthd_my_send_cmd ./server/libmariadb/libmariadb/mariadb_lib.c:394:9 #5 0x5ceaa4f5add5 in ma_simple_command ./server/libmariadb/libmariadb/mariadb_lib.c:472:10 #6 0x5ceaa4f75044 in mysql_send_query ./server/libmariadb/libmariadb/mariadb_lib.c:2524:10 #7 0x5ceaa4efc226 in wrap_mysql_send_query(st_mysql*, char const*, unsigned long) ./server/client/../tests/nonblock-wrappers.h:211:1 #8 0x5ceaa4f2f8ce in run_query_normal(st_connection*, st_command*, int, char const*, unsigned long, st_dynamic_string*, st_dynamic_string*) ./server/client/mysqltest.cc:8230:9 #9 0x5ceaa4f36676 in run_query(st_connection*, st_command*, int) ./server/client/mysqltest.cc:9652:5 #10 0x5ceaa4f3a0ea in main ./server/client/mysqltest.cc:10484:2 #11 0x7efaf322a1c9 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 #12 0x7efaf322a28a in __libc_start_main csu/../csu/libc-start.c:360:3 #13 0x5ceaa4e1fd44 in _start (./server/build116_asan/client/mariadb-test+0x141d44) (BuildId: 21b3097a37beada873a1eaa15c1ea1f16d3ad6d7)

rspadim added 7 commits September 19, 2014 23:35

Include statistic fields to query cache and qc_info

79102cd

Use MY_MIN instead of min, downgrade from gamma to alpha

26001e1

forgot to define qc_info_now variable at query_cache::end_of_result

af2a39c

rewrote test files

9b5cfce

change #define #ifdef macros to always include stats columns

abf6f24

Include compile time macro HAVE_QUERY_CACHE_STATS

a57e57a

Include a new column to query_cache_queries_table with table suffix

5fe6231

rspadim mentioned this pull request Sep 26, 2014

FINALLY FOUND YOU!!! mcshadow/mysql_query_cache#1

Closed

include is_disabled() before try_lock

7bca392

rspadim reviewed Sep 30, 2014
View reviewed changes

When HAVE_QUERY_CACHE_STATS isn't defined, set all unused fields as null

0dfd1e0

janlindstrom force-pushed the 10.1 branch from f0a545c to 58888e2 Compare October 29, 2014 09:10

nirbhayc pushed a commit that referenced this pull request Apr 1, 2015

refs #6 fixed compilation errors with -DWITH_DEBUG=1, removed unwante…

6a03d7c

…d files

nirbhayc pushed a commit that referenced this pull request May 8, 2015

refs #6 fixed compilation errors with -DWITH_DEBUG=1, removed unwante…

0f405f2

…d files

grooverdan mentioned this pull request May 17, 2015

MDEV-4581 / MDEV-4682 plugin - qc_info - add Query Cache flags #67

Closed

svoj closed this Feb 18, 2016

elenst added a commit that referenced this pull request Feb 6, 2017

Additional test fixes for MDEV-7635 #6

93f1b22

Without sorting the test produces indeterministic result

nirbhayc pushed a commit that referenced this pull request Feb 8, 2017

Additional test fixes for MDEV-7635 #6

f675772

Without sorting the test produces indeterministic result

Peter-Sh mentioned this pull request Aug 25, 2017

Hits statistics for queries in query cache (for query cache information plugin) #441

Merged

kevgs mentioned this pull request Jan 11, 2018

MDEV-15111 - fix simple InnoDB data race #542

Closed

kevgs mentioned this pull request Jan 21, 2018

simplify and make counter atomic #567

Closed

kevgs mentioned this pull request Feb 20, 2018

ASAN unpoison system versioning related buffer #625

Merged

grooverdan referenced this pull request in grooverdan/mariadb-server Mar 25, 2018

Merge pull request #6 from ericherman/master

8daf65a

fix compiler cast warnings in unittests

janlindstrom mentioned this pull request Apr 5, 2018

MDEV-13549 Fix and re-enable MTR test galera.MW-284 #675

Merged

kevgs mentioned this pull request Jul 2, 2018

MDEV-16658 Memory leak in mysqltest on connect failure #809

Closed

spetrunia added a commit that referenced this pull request Aug 30, 2018

MyRocks: post-merge fixes part #6: make rocksdb.allow_no_primary_key …

290368e

…test pass

FooBarrior referenced this pull request Aug 9, 2019

MDEV-20068 History partition rotation is not done under LOCK TABLES

e647c32

Wrong value F_WRLCK for thr_lock_type.

varunraiko pushed a commit that referenced this pull request Mar 4, 2020

Fix #6 Performance regression

e7ed25d

spetrunia added a commit that referenced this pull request Mar 18, 2022

Test updates #6

27e74f5

janlindstrom pushed a commit that referenced this pull request Jun 14, 2022

Fix #6

973ace9

illuusio pushed a commit to illuusio/server that referenced this pull request Aug 14, 2023

Merge pull request MariaDB#6 from waja/mysqld_multi

cef4b6d

Adding mysqld_multi_confd.patch, makes mysqld_multi reading conf.d

mleich1 mentioned this pull request Nov 8, 2023

MDEV-31953 madvise(..., MADV_FREE) is causing a performance regression / MDEV-24670 avoid OOM by linux kernel co-operative memory management #2805

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MDEV-4682 - Include statistic fields to query cache and qc_info #6

MDEV-4682 - Include statistic fields to query cache and qc_info #6

rspadim commented Sep 20, 2014

rpbouman commented Sep 27, 2014

rspadim commented Sep 27, 2014

rspadim Sep 30, 2014

svoj commented Feb 18, 2016

MDEV-4682 - Include statistic fields to query cache and qc_info #6

MDEV-4682 - Include statistic fields to query cache and qc_info #6

Conversation

rspadim commented Sep 20, 2014

rpbouman commented Sep 27, 2014

rspadim commented Sep 27, 2014

rspadim Sep 30, 2014

Choose a reason for hiding this comment

svoj commented Feb 18, 2016