FB8-48: Option to run triggers on slave for row-based events (facebook#946) (facebook#946)

Summary:
JIRA: https://jira.percona.com/browse/FB8-48

Reference Patch: facebook@8225c64
Reference Patch: facebook@f5466d6
Reference Patch: facebook@87e3650
Port the slave_run_triggers_for_rbr feature from mariadb 10.1.1.

When using statement based replication slave executes the sql statments
which runs the slave side triggers. Since in row based replication, slave
applies the row events directly to the storage engine, triggers on the slave
table are not executed. Add functionality to run triggers on slave side
when executing row based events.

The following triggers are invoked:

* Update_row_event runs an UPDATE trigger
* Delete_row_event runs a DELETE trigger
* Write_row_event action depends on whether the operation will require foreign key checks:
  1) when FK checks are not necessary, the operation will invoke a DELETE trigger if the record
  to be modified existed in the table. After that, an INSERT trigger will be invoked.
  2) when FK checks are necessary, either an UPDATE or or a combination of
     DELETE and INSERT triggers will be invoked.

slave_run_triggers_for_rbr option controls the feature.
Default value is NO which don't invoke trigger for row-based events; Setting the option
to YES will cause the SQL slave thread to invoke triggers for row based events; setting it to
LOGGING will also cause the changes made by the triggers to be written into the binary log.

There is a basic protection against triggers being invoked both on the master and slave.
If the master modifies a table that has triggers, it will produce row-based binlog events
with the "triggers were invoked for this event" flag. The slave will not invoke any
triggers for flagged events.

optionally disable binlogging while executing triggers

Online schema change (OSC) creates triggers only on master side. Since RBR logs row changes made by triggers, sql_thread will hit errors due to missing tables on slaves. Add a session variable SQL_LOG_BIN_TRIGGERS to optionally disable binlogging for trigger statements so that RBR and OSC are fine with each other.

disable_sql_log_bin_triggers flag is added in the TABLE_LIST struct to track tables that are opened during trigger execution. This flag is used to skip writing Table_map_log_events for such tables.

With sql_log_bin_triggers enabled, the trigger changes which may be necessary for slave are not propagated. To avoid this, slave_run_triggers_for_rbr option must be enabled on slave.
Pull Request resolved: facebook#946

Reviewed By: lloyd

Differential Revision: D13972562

Pulled By: lth

Author: santoshbanda

Diff for: mysql-test/include/have_rbr_triggers.inc

@@ -0,0 +1,4 @@
+if (`select count(*) = 0 from performance_schema.session_variables where variable_name = 'slave_run_triggers_for_rbr'`)
+{
+  skip RBR triggers are not available;
+}

Diff for: mysql-test/r/mysqld--help-notwin.result

@@ -2093,6 +2093,15 @@ The following options may be given as the first argument:
  HASH_SCAN. Any combination is allowed, and the applier
  picks the most efficient among them for any given
  scenario. (Default: INDEX_SCAN, HASH_SCAN).
+ --slave-run-triggers-for-rbr=name 
+ Modes for how triggers in row-base replication on slave
+ side will be executed. Legal values are NO (default), YES
+ and LOGGING. NO means that trigger for RBR will not be
+ running on slave. YES and LOGGING means that triggers
+ will be running on slave, if there was not triggers
+ running on the master for the statement. LOGGING also
+ means results of that the executed triggers work will be
+ written to the binlog.
  --slave-skip-errors=name 
  This option is deprecated. Use replica_skip_errors
  instead.
@@ -2133,6 +2142,10 @@ The following options may be given as the first argument:
  When set, if a table is created without a primary key
  then server generates invisible auto-increment column as
  a primary key for the table.
+ --sql-log-bin-triggers 
+ The row changes generated by execution of triggers are
+ not logged inbinlog if this option is false.
+ (Defaults to on; use --skip-sql-log-bin-triggers to disable.)
  --sql-mode=name     Syntax: sql-mode=mode[,mode[,mode...]]. See the manual
  for the complete list of valid sql modes
  --sql-require-primary-key 
@@ -2873,6 +2886,7 @@ slave-parallel-workers 4
 slave-pending-jobs-size-max 134217728
 slave-preserve-commit-order TRUE
 slave-rows-search-algorithms INDEX_SCAN,HASH_SCAN
+slave-run-triggers-for-rbr NO
 slave-skip-errors (No default value)
 slave-sql-verify-checksum TRUE
 slave-transaction-retries 10
@@ -2883,6 +2897,7 @@ sort-buffer-size 262144
 source-verify-checksum FALSE
 sporadic-binlog-dump-fail FALSE
 sql-generate-invisible-primary-key FALSE
+sql-log-bin-triggers TRUE
 sql-mode ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
 sql-require-primary-key FALSE
 stored-program-cache 256

Diff for: mysql-test/suite/binlog_nogtid/r/binlog_persist_only_variables.result

@@ -255,6 +255,7 @@ SET PERSIST_ONLY slave_rows_search_algorithms = @@GLOBAL.slave_rows_search_algor
 Warnings:
 Warning	1287	'@@slave_rows_search_algorithms' is deprecated and will be removed in a future release.
 Warning	1287	'@@slave_rows_search_algorithms' is deprecated and will be removed in a future release.
+SET PERSIST_ONLY slave_run_triggers_for_rbr = @@GLOBAL.slave_run_triggers_for_rbr;
 SET PERSIST_ONLY slave_skip_errors = @@GLOBAL.slave_skip_errors;
 Warnings:
 Warning	1287	'@@slave_skip_errors' is deprecated and will be removed in a future release. Please use replica_skip_errors instead.
@@ -272,6 +273,7 @@ Warnings:
 Warning	1287	'@@slave_type_conversions' is deprecated and will be removed in a future release. Please use replica_type_conversions instead.
 Warning	1287	'@@slave_type_conversions' is deprecated and will be removed in a future release. Please use replica_type_conversions instead.
 SET PERSIST_ONLY source_verify_checksum = @@GLOBAL.source_verify_checksum;
+SET PERSIST_ONLY sql_log_bin_triggers = @@GLOBAL.sql_log_bin_triggers;
 SET PERSIST_ONLY sql_replica_skip_counter = @@GLOBAL.sql_replica_skip_counter;
 SET PERSIST_ONLY sql_slave_skip_counter = @@GLOBAL.sql_slave_skip_counter;
 Warnings:
@@ -403,7 +405,9 @@ RESET PERSIST session_track_gtids;
 RESET PERSIST skip_replica_start;
 RESET PERSIST slave_compression_lib;
 RESET PERSIST slave_rows_search_algorithms;
+RESET PERSIST slave_run_triggers_for_rbr;
 RESET PERSIST source_verify_checksum;
+RESET PERSIST sql_log_bin_triggers;
 RESET PERSIST sql_replica_skip_counter;
 RESET PERSIST sync_binlog;
 RESET PERSIST sync_relay_log;

Diff for: mysql-test/suite/binlog_nogtid/r/binlog_persist_variables.result

@@ -234,6 +234,7 @@ SET PERSIST slave_rows_search_algorithms = @@GLOBAL.slave_rows_search_algorithms
 Warnings:
 Warning	1287	'@@slave_rows_search_algorithms' is deprecated and will be removed in a future release.
 Warning	1287	'@@slave_rows_search_algorithms' is deprecated and will be removed in a future release.
+SET PERSIST slave_run_triggers_for_rbr = @@GLOBAL.slave_run_triggers_for_rbr;
 SET PERSIST slave_skip_errors = @@GLOBAL.slave_skip_errors;
 ERROR HY000: Variable 'slave_skip_errors' is a read only variable
 SET PERSIST slave_sql_verify_checksum = @@GLOBAL.slave_sql_verify_checksum;
@@ -249,6 +250,7 @@ Warnings:
 Warning	1287	'@@slave_type_conversions' is deprecated and will be removed in a future release. Please use replica_type_conversions instead.
 Warning	1287	'@@slave_type_conversions' is deprecated and will be removed in a future release. Please use replica_type_conversions instead.
 SET PERSIST source_verify_checksum = @@GLOBAL.source_verify_checksum;
+SET PERSIST sql_log_bin_triggers = @@GLOBAL.sql_log_bin_triggers;
 SET PERSIST sql_replica_skip_counter = @@GLOBAL.sql_replica_skip_counter;
 SET PERSIST sql_slave_skip_counter = @@GLOBAL.sql_slave_skip_counter;
 Warnings:
@@ -460,6 +462,7 @@ RESET PERSIST IF EXISTS slave_preserve_commit_order;
 Warnings:
 Warning	3615	Variable slave_preserve_commit_order does not exist in persisted config file
 RESET PERSIST IF EXISTS slave_rows_search_algorithms;
+RESET PERSIST IF EXISTS slave_run_triggers_for_rbr;
 RESET PERSIST IF EXISTS slave_skip_errors;
 Warnings:
 Warning	3615	Variable slave_skip_errors does not exist in persisted config file
@@ -475,6 +478,7 @@ Warning	3615	Variable slave_type_conversions does not exist in persisted config
 RESET PERSIST IF EXISTS source_verify_checksum;
 Warnings:
 Warning	3615	Variable source_verify_checksum does not exist in persisted config file
+RESET PERSIST IF EXISTS sql_log_bin_triggers;
 RESET PERSIST IF EXISTS sql_replica_skip_counter;
 RESET PERSIST IF EXISTS sql_slave_skip_counter;
 Warnings:

Diff for: mysql-test/suite/binlog_nogtid/t/binlog_persist_only_variables.test

@@ -108,7 +108,7 @@ while ( $varid <= $countvars )
   --eval SET PERSIST_ONLY $varname = @@GLOBAL.$varname
 
   #  TODO: Remove/update this once Bug#27322592 is FIXED.
-  if (`SELECT '$varname' IN ('binlog_trx_meta_data', 'binlog_direct_non_transactional_updates', 'binlog_order_commits', 'binlog_rows_query_log_events', 'log_bin_trust_function_creators', 'log_slow_replica_statements', 'log_statements_unsafe_for_binlog', 'source_verify_checksum', 'replica_allow_batching', 'replica_compressed_protocol', 'replica_preserve_commit_order', 'replica_sql_verify_checksum', 'relay_log_purge', 'rpl_semi_sync_source_enabled', 'rpl_semi_sync_source_wait_no_replica', 'rpl_semi_sync_replica_enabled', 'binlog_gtid_simple_recovery', 'log_replica_updates', 'relay_log_recovery', 'binlog_rotate_encryption_master_key_at_startup', 'binlog_transaction_compression')`)
+  if (`SELECT '$varname' IN ('sql_log_bin_triggers', 'binlog_trx_meta_data', 'binlog_direct_non_transactional_updates', 'binlog_order_commits', 'binlog_rows_query_log_events', 'log_bin_trust_function_creators', 'log_slow_replica_statements', 'log_statements_unsafe_for_binlog', 'source_verify_checksum', 'replica_allow_batching', 'replica_compressed_protocol', 'replica_preserve_commit_order', 'replica_sql_verify_checksum', 'relay_log_purge', 'rpl_semi_sync_source_enabled', 'rpl_semi_sync_source_wait_no_replica', 'rpl_semi_sync_replica_enabled', 'binlog_gtid_simple_recovery', 'log_replica_updates', 'relay_log_recovery', 'binlog_rotate_encryption_master_key_at_startup', 'binlog_transaction_compression')`)
   {
     --disable_query_log
       --eval SELECT varvalue INTO @varvalue FROM rplvars WHERE id=$varid

Diff for: mysql-test/suite/rocksdb/r/rpl_row_triggers.result

@@ -1,7 +1,7 @@
 include/master-slave.inc
 Warnings:
 Note	####	Sending passwords in plain text without SSL/TLS is extremely insecure.
-Note	####	Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
+Note	####	Storing MySQL user name or password information in the connection metadata repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START REPLICA; see the 'START REPLICA Syntax' in the MySQL Manual for more information.
 [connection master]
 # Test of row replication with triggers on the slave side
 CREATE TABLE t1 (C1 CHAR(1) primary key, C2 CHAR(1));