[mysql-5.6][PR] FB8-48: Option to run triggers on slave for row-based…

… events

Summary:
JIRA: https://jira.percona.com/browse/FB8-48

Reference Patch: 8225c64
Reference Patch: f5466d6
Reference Patch: 87e3650
Port the slave_run_triggers_for_rbr feature from mariadb 10.1.1.

When using statement based replication slave executes the sql statments
which runs the slave side triggers. Since in row based replication, slave
applies the row events directly to the storage engine, triggers on the slave
table are not executed. Add functionality to run triggers on slave side
when executing row based events.

The following triggers are invoked:

* Update_row_event runs an UPDATE trigger
* Delete_row_event runs a DELETE trigger
* Write_row_event action depends on whether the operation will require foreign key checks:
  1) when FK checks are not necessary, the operation will invoke a DELETE trigger if the record
  to be modified existed in the table. After that, an INSERT trigger will be invoked.
  2) when FK checks are necessary, either an UPDATE or or a combination of
     DELETE and INSERT triggers will be invoked.

slave_run_triggers_for_rbr option controls the feature.
Default value is NO which don't invoke trigger for row-based events; Setting the option
to YES will cause the SQL slave thread to invoke triggers for row based events; setting it to
LOGGING will also cause the changes made by the triggers to be written into the binary log.

There is a basic protection against triggers being invoked both on the master and slave.
If the master modifies a table that has triggers, it will produce row-based binlog events
with the "triggers were invoked for this event" flag. The slave will not invoke any
triggers for flagged events.

optionally disable binlogging while executing triggers

Online schema change (OSC) creates triggers only on master side. Since RBR logs row changes made by triggers, sql_thread will hit errors due to missing tables on slaves. Add a session variable SQL_LOG_BIN_TRIGGERS to optionally disable binlogging for trigger statements so that RBR and OSC are fine with each other.

disable_sql_log_bin_triggers flag is added in the TABLE_LIST struct to track tables that are opened during trigger execution. This flag is used to skip writing Table_map_log_events for such tables.

With sql_log_bin_triggers enabled, the trigger changes which may be necessary for slave are not propagated. To avoid this, slave_run_triggers_for_rbr option must be enabled on slave.
Pull Request resolved: #946
GitHub Author: Santosh Praneeth Banda <santoshb@fb.com>

Test Plan: mtr

Reviewers: abhinavsharma, mung

Reviewed By: mung

Subscribers: butterflybot, vinaybhat, webscalesql-eng@fb.com

Differential Revision: https://phabricator.intern.facebook.com/D13972562

Signature: 13972562:1552064733:0ad082766e8c3504694ed8e474fb454f1b891272

mysql-test/include/have_rbr_triggers.inc

@@ -0,0 +1,4 @@
+if (`select count(*) = 0 from performance_schema.session_variables where variable_name = 'slave_run_triggers_for_rbr'`)
+{
+  skip RBR triggers are not available;
+}

mysql-test/r/all_persisted_variables.result

@@ -44,17 +44,17 @@ include/assert.inc ['Expect 500+ variables in the table. Due to open Bugs, we ar
 
 # Test SET PERSIST
 
-include/assert.inc ['Expect 386 persisted variables in the table. Due to open Bugs, we are checking for 380']
+include/assert.inc ['Expect 388 persisted variables in the table. Due to open Bugs, we are checking for 382']
 
 ************************************************************
 * 3. Restart server, it must preserve the persisted variable
 *    settings. Verify persisted configuration.
 ************************************************************
 # restart
 
-include/assert.inc ['Expect 380 persisted variables in persisted_variables table.']
-include/assert.inc ['Expect 380 persisted variables shown as PERSISTED in variables_info table.']
-include/assert.inc ['Expect 380 persisted variables with matching peristed and global values.']
+include/assert.inc ['Expect 382 persisted variables in persisted_variables table.']
+include/assert.inc ['Expect 382 persisted variables shown as PERSISTED in variables_info table.']
+include/assert.inc ['Expect 382 persisted variables with matching peristed and global values.']
 
 ************************************************************
 * 4. Test RESET PERSIST IF EXISTS. Verify persisted variable

mysql-test/r/mysqld--help-notwin.result

@@ -1238,6 +1238,15 @@ The following options may be given as the first argument:
  TABLE_SCAN and HASH_SCAN. Any combination is allowed, and
  the slave will always pick the most suitable algorithm
  for any given scenario. (Default: INDEX_SCAN, HASH_SCAN).
+ --slave-run-triggers-for-rbr=name 
+ Modes for how triggers in row-base replication on slave
+ side will be executed. Legal values are NO (default), YES
+ and LOGGING. NO means that trigger for RBR will not be
+ running on slave. YES and LOGGING means that triggers
+ will be running on slave, if there was not triggers
+ running on the master for the statement. LOGGING also
+ means results of that the executed triggers work will be
+ written to the binlog.
  --slave-skip-errors=name 
  Tells the slave thread to continue replication when a
  query event returns an error from the provided list
@@ -1283,6 +1292,10 @@ The following options may be given as the first argument:
  --sporadic-binlog-dump-fail 
  Option used by mysql-test for debugging and testing of
  replication.
+ --sql-log-bin-triggers 
+ The row changes generated by execution of triggers are
+ not logged inbinlog if this option is false.
+ (Defaults to on; use --skip-sql-log-bin-triggers to disable.)
  --sql-mode=name     Syntax: sql-mode=mode[,mode[,mode...]]. See the manual
  for the complete list of valid sql modes
  --sql-require-primary-key 
@@ -1718,6 +1731,7 @@ slave-parallel-workers 0
 slave-pending-jobs-size-max 134217728
 slave-preserve-commit-order FALSE
 slave-rows-search-algorithms INDEX_SCAN,HASH_SCAN
+slave-run-triggers-for-rbr NO
 slave-skip-errors (No default value)
 slave-sql-verify-checksum TRUE
 slave-transaction-retries 10
@@ -1726,6 +1740,7 @@ slow-launch-time 2
 slow-query-log FALSE
 sort-buffer-size 262144
 sporadic-binlog-dump-fail FALSE
+sql-log-bin-triggers TRUE
 sql-mode ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
 sql-require-primary-key FALSE
 stored-program-cache 256

mysql-test/suite/binlog_nogtid/r/binlog_persist_only_variables.result

@@ -25,7 +25,7 @@ VARIABLE_NAME LIKE '%slave%') AND
 'reset_seconds_behind_master', 'skip_flush_master_info'))
 ORDER BY VARIABLE_NAME;
 
-include/assert.inc ['Expect 83 variables in the table.']
+include/assert.inc ['Expect 85 variables in the table.']
 
 # Test SET PERSIST_ONLY
 SET PERSIST_ONLY binlog_cache_size = @@GLOBAL.binlog_cache_size;
@@ -115,26 +115,28 @@ SET PERSIST_ONLY slave_parallel_workers = @@GLOBAL.slave_parallel_workers;
 SET PERSIST_ONLY slave_pending_jobs_size_max = @@GLOBAL.slave_pending_jobs_size_max;
 SET PERSIST_ONLY slave_preserve_commit_order = @@GLOBAL.slave_preserve_commit_order;
 SET PERSIST_ONLY slave_rows_search_algorithms = @@GLOBAL.slave_rows_search_algorithms;
+SET PERSIST_ONLY slave_run_triggers_for_rbr = @@GLOBAL.slave_run_triggers_for_rbr;
 SET PERSIST_ONLY slave_skip_errors = @@GLOBAL.slave_skip_errors;
 SET PERSIST_ONLY slave_sql_verify_checksum = @@GLOBAL.slave_sql_verify_checksum;
 SET PERSIST_ONLY slave_transaction_retries = @@GLOBAL.slave_transaction_retries;
 SET PERSIST_ONLY slave_type_conversions = @@GLOBAL.slave_type_conversions;
+SET PERSIST_ONLY sql_log_bin_triggers = @@GLOBAL.sql_log_bin_triggers;
 SET PERSIST_ONLY sql_slave_skip_counter = @@GLOBAL.sql_slave_skip_counter;
 SET PERSIST_ONLY sync_binlog = @@GLOBAL.sync_binlog;
 SET PERSIST_ONLY sync_master_info = @@GLOBAL.sync_master_info;
 SET PERSIST_ONLY sync_relay_log = @@GLOBAL.sync_relay_log;
 SET PERSIST_ONLY sync_relay_log_info = @@GLOBAL.sync_relay_log_info;
 
-include/assert.inc ['Expect 73 persisted variables in persisted_variables table.']
+include/assert.inc ['Expect 75 persisted variables in persisted_variables table.']
 
 ############################################################
 # 2. Restart server, it must preserve the persisted variable
 #    settings. Verify persisted configuration.
 # restart
 
-include/assert.inc ['Expect 73 persisted variables in persisted_variables table.']
-include/assert.inc ['Expect 73 persisted variables shown as PERSISTED in variables_info table.']
-include/assert.inc ['Expect 73 persisted variables with matching peristed and global values.']
+include/assert.inc ['Expect 75 persisted variables in persisted_variables table.']
+include/assert.inc ['Expect 75 persisted variables shown as PERSISTED in variables_info table.']
+include/assert.inc ['Expect 75 persisted variables with matching peristed and global values.']
 
 ############################################################
 # 3. Test RESET PERSIST. Verify persisted variable settings
@@ -223,10 +225,12 @@ RESET PERSIST slave_parallel_workers;
 RESET PERSIST slave_pending_jobs_size_max;
 RESET PERSIST slave_preserve_commit_order;
 RESET PERSIST slave_rows_search_algorithms;
+RESET PERSIST slave_run_triggers_for_rbr;
 RESET PERSIST slave_skip_errors;
 RESET PERSIST slave_sql_verify_checksum;
 RESET PERSIST slave_transaction_retries;
 RESET PERSIST slave_type_conversions;
+RESET PERSIST sql_log_bin_triggers;
 RESET PERSIST sql_slave_skip_counter;
 RESET PERSIST sync_binlog;
 RESET PERSIST sync_master_info;

mysql-test/suite/binlog_nogtid/r/binlog_persist_variables.result

@@ -25,7 +25,7 @@ VARIABLE_NAME LIKE '%slave%') AND
 'reset_seconds_behind_master', 'skip_flush_master_info'))
 ORDER BY VARIABLE_NAME;
 
-include/assert.inc ['Expect 83 variables in the table.']
+include/assert.inc ['Expect 85 variables in the table.']
 
 # Test SET PERSIST
 SET PERSIST binlog_cache_size = @@GLOBAL.binlog_cache_size;
@@ -119,27 +119,29 @@ SET PERSIST slave_parallel_workers = @@GLOBAL.slave_parallel_workers;
 SET PERSIST slave_pending_jobs_size_max = @@GLOBAL.slave_pending_jobs_size_max;
 SET PERSIST slave_preserve_commit_order = @@GLOBAL.slave_preserve_commit_order;
 SET PERSIST slave_rows_search_algorithms = @@GLOBAL.slave_rows_search_algorithms;
+SET PERSIST slave_run_triggers_for_rbr = @@GLOBAL.slave_run_triggers_for_rbr;
 SET PERSIST slave_skip_errors = @@GLOBAL.slave_skip_errors;
 ERROR HY000: Variable 'slave_skip_errors' is a read only variable
 SET PERSIST slave_sql_verify_checksum = @@GLOBAL.slave_sql_verify_checksum;
 SET PERSIST slave_transaction_retries = @@GLOBAL.slave_transaction_retries;
 SET PERSIST slave_type_conversions = @@GLOBAL.slave_type_conversions;
+SET PERSIST sql_log_bin_triggers = @@GLOBAL.sql_log_bin_triggers;
 SET PERSIST sql_slave_skip_counter = @@GLOBAL.sql_slave_skip_counter;
 SET PERSIST sync_binlog = @@GLOBAL.sync_binlog;
 SET PERSIST sync_master_info = @@GLOBAL.sync_master_info;
 SET PERSIST sync_relay_log = @@GLOBAL.sync_relay_log;
 SET PERSIST sync_relay_log_info = @@GLOBAL.sync_relay_log_info;
 
-include/assert.inc ['Expect 68 persisted variables in persisted_variables table.']
+include/assert.inc ['Expect 70 persisted variables in persisted_variables table.']
 
 ############################################################
 # 2. Restart server, it must preserve the persisted variable
 #    settings. Verify persisted configuration.
 # restart
 
-include/assert.inc ['Expect 68 persisted variables in persisted_variables table.']
-include/assert.inc ['Expect 68 persisted variables shown as PERSISTED in variables_info table.']
-include/assert.inc ['Expect 68 persisted variables with matching peristed and global values.']
+include/assert.inc ['Expect 70 persisted variables in persisted_variables table.']
+include/assert.inc ['Expect 70 persisted variables shown as PERSISTED in variables_info table.']
+include/assert.inc ['Expect 70 persisted variables with matching peristed and global values.']
 
 ############################################################
 # 3. Test RESET PERSIST IF EXISTS. Verify persisted variable
@@ -246,12 +248,14 @@ RESET PERSIST IF EXISTS slave_parallel_workers;
 RESET PERSIST IF EXISTS slave_pending_jobs_size_max;
 RESET PERSIST IF EXISTS slave_preserve_commit_order;
 RESET PERSIST IF EXISTS slave_rows_search_algorithms;
+RESET PERSIST IF EXISTS slave_run_triggers_for_rbr;
 RESET PERSIST IF EXISTS slave_skip_errors;
 Warnings:
 Warning	3615	Variable slave_skip_errors does not exist in persisted config file
 RESET PERSIST IF EXISTS slave_sql_verify_checksum;
 RESET PERSIST IF EXISTS slave_transaction_retries;
 RESET PERSIST IF EXISTS slave_type_conversions;
+RESET PERSIST IF EXISTS sql_log_bin_triggers;
 RESET PERSIST IF EXISTS sql_slave_skip_counter;
 RESET PERSIST IF EXISTS sync_binlog;
 RESET PERSIST IF EXISTS sync_master_info;

mysql-test/suite/binlog_nogtid/t/binlog_persist_only_variables.test

@@ -59,7 +59,7 @@ INSERT INTO rplvars (varname, varvalue)
 
 # If this count differs, it means a variable has been added or removed.
 # In that case, this testcase needs to be updated accordingly.
---let $var_count=83
+--let $var_count=85
 --echo
 --let $assert_text= 'Expect $var_count variables in the table.'
 --let $assert_cond= [SELECT COUNT(*) as count FROM rplvars, count, 1] = $var_count
@@ -80,7 +80,7 @@ while ( $varid <= $countvars )
   --eval SET PERSIST_ONLY $varnames = @@GLOBAL.$varnames
 
   #  TODO: Remove/update this once Bug#27322592 is FIXED.
-  if (`SELECT '$varnames' IN ('binlog_direct_non_transactional_updates', 'binlog_order_commits', 'binlog_rows_query_log_events', 'log_bin_trust_function_creators', 'log_bin_use_v1_row_events', 'log_slow_slave_statements', 'log_statements_unsafe_for_binlog', 'master_verify_checksum', 'slave_allow_batching', 'slave_compressed_protocol', 'slave_preserve_commit_order', 'slave_sql_verify_checksum', 'relay_log_purge', 'rpl_semi_sync_master_enabled', 'rpl_semi_sync_master_wait_no_slave', 'rpl_semi_sync_slave_enabled', 'binlog_gtid_simple_recovery', 'log_slave_updates', 'relay_log_recovery')`)
+  if (`SELECT '$varnames' IN ('binlog_direct_non_transactional_updates', 'binlog_order_commits', 'binlog_rows_query_log_events', 'log_bin_trust_function_creators', 'log_bin_use_v1_row_events', 'log_slow_slave_statements', 'log_statements_unsafe_for_binlog', 'master_verify_checksum', 'slave_allow_batching', 'slave_compressed_protocol', 'slave_preserve_commit_order', 'slave_sql_verify_checksum', 'relay_log_purge', 'rpl_semi_sync_master_enabled', 'rpl_semi_sync_master_wait_no_slave', 'rpl_semi_sync_slave_enabled', 'binlog_gtid_simple_recovery', 'log_slave_updates', 'relay_log_recovery', 'sql_log_bin_triggers')`)
   {
     --disable_query_log
       --eval SELECT varvalue INTO @varvalues FROM rplvars WHERE id=$varid
@@ -91,7 +91,7 @@ while ( $varid <= $countvars )
   --inc $varid
 }
 
---let $var_count=73
+--let $var_count=75
 --echo
 --let $assert_text= 'Expect $var_count persisted variables in persisted_variables table.'
 --let $assert_cond= [SELECT COUNT(*) as count FROM performance_schema.persisted_variables, count, 1] = $var_count

mysql-test/suite/binlog_nogtid/t/binlog_persist_variables.test

@@ -60,7 +60,7 @@ INSERT INTO rplvars (varname, varvalue)
 
 # If this count differs, it means a variable has been added or removed.
 # In that case, this testcase needs to be updated accordingly.
---let $var_count=83
+--let $var_count=85
 --echo
 --let $assert_text= 'Expect $var_count variables in the table.'
 --let $assert_cond= [SELECT COUNT(*) as count FROM rplvars, count, 1] = $var_count
@@ -83,7 +83,7 @@ while ( $varid <= $countvars )
   --inc $varid
 }
 
---let $var_count=68
+--let $var_count=70
 --echo
 --let $assert_text= 'Expect $var_count persisted variables in persisted_variables table.'
 --let $assert_cond= [SELECT COUNT(*) as count FROM performance_schema.persisted_variables, count, 1] = $var_count