Implemented PS-5937 (Merge MySQL 5.7.29) part 2 #3611
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Approved-by: Bjorn Munch <bjorn.munch@oracle.com>
Approved-by: Bjorn Munch <bjorn.munch@oracle.com>
Approved-by: Bjorn Munch <bjorn.munch@oracle.com>
Problem ======= Audit API generates MYSQL_AUDIT_TABLE_ACCESS_READ for SQL commands that performs table read operation. This was not done for ANALYZE TABLE command. Analysis ======== Audit event generation explicitly specifies the set of sql commands, which generate MYSQL_AUDIT_TABLE_ACCESS_READ event. The set must be extended for ANALYZE TABLE too, since it performs table read operation. Fix === Generate MYSQL_AUDIT_TABLE_ACCESS_READ event for ANALYZE TABLE syntax. This allows for better audit logging as well for audit rule filtering including aborting on command. RB: 22879 Approved by: Georgi 'Joro' Kodinov <georgi.kodinov@oracle.com> Approved by: Harin Vadodaria <harin.vadodaria@oracle.com>
Reviewed-by: Terje Rosten <terje.rosten@oracle.com>
- Update the server compat version Reviewed-by: Terje Rosten <terje.rosten@oracle.com>
WITHOUT START SLAVE Problem ------- 1. Execute CHANGE MASTER without MASTER_LOG_FILE and MASTER_LOG_POS options on slave server. 2. Restart slave with `--relay-log-recovery`. 3. Executing START SLAVE will report the following error. mysql> START SLAVE; ERROR 1872 (HY000): Slave failed to initialize relay log info structure from the repository Analysis -------- During relay log recovery, if `Relay_Master_Log_File` is not set, - The existing relay logs are parsed to find the first Rotate event from the master. - The `master_log_file` and `master_log_pos` are extracted from master's rotate event and are set to applier thread's `Relay_Master_Log_File` and `Exec_Master_Log_Pos` respectively. - Receiver thread starts replicating from the new position. The above logic was introduced by Bug#19021091 and it works only when receiver thread has started and has fetched events from master, and errors out when receiver thread has not started before recovery as it fails to find the real rotate events in the relay log. Fix --- The fix for the above problem is to skip the relay log recovery by making the function `find_first_relay_log_with_rotate_from_master()` return - success: when a ROTATE event from master is found; - error: some events are found replicated but no ROTATE from master; - not found: found no events replicated from any master on relay logs; and when server was not able to find any events replicated from master, it shall log a warning that "Server was not able to find a rotate event from a master server to initialize relay log recovery for channel '<CHANNEL_NAME>'. Skipping relay log recovery for the channel." into error log instead of treating this as an error. Also, during recovery, slave server used to ignore the Format description event, Previous_gtid log event and ignorable events within the relay log as they are generated by slave server. With this fix STOP_EVENT (generated by server during shut down) is also added to the above list of ignored events and the server shall continue to read the next available relay log to find the rotate from master. Reviewed by: Pedro Figueiredo <pedro.figueiredo@oracle.com> Reviewed by: Abhinav Agarwal <abhinav.ab.agarwal@oracle.com> RB: 22761, 22891
Additional patch clarifying what "system" SSL means on Windows. Change-Id: I809847f1097d98062bfcc695b18c957f1eab899f
… 5.6 & 5.7
Patch for 5.6
Workaround for compiler bug
ld.so.1: mysqld: fatal: relocation error: file sql/mysqld:
symbol OPENSSL_sk_new_null: referenced symbol not found
openssl/safestack.h has lots of pragma weak <function>.
All are tagged as WEAK in generated binaries.
Taking the address of OPENSSL_sk_new_null solves the problem.
We do it in sql/net_serv.cc because this file is used by both
client and server and "embedded" versions of client and server.
Change-Id: I49b27dade048d8e27715ecfe3d6bd6c076cb7b55
…ESTART' Memory allocated via get_name->token_copy is not freed in function invalidate_servers. Similar call in function update_servers does free the memory. Resolution: Free the memory before returning. ReviewBoard: 22880
PROBLEM
-------
The root cause of the problem was that delete marked rows can acquire a
external read lock at the stage where partial rollback is not complete
In partial rollback when we try to convert implicit lock to explicit
we get a assert saying that already it is locked by a external read lock.
1. For Secondary Index: During rollback, we can remove delete marked key (that is ok
to purge) even if the transaction hasn't modified it. In such case it is not
right to convert to explicit lock since the transaction is not holding
implicit lock on the key.
2. For Cluster Index: If INSERT has modified an existing delete marked key, then
during rollback there are 2 steps.
A. Rollback the update replacing with previous key (delete mark)
B. Remove the delete mark on the row if it is ok to purged
The implicit lock is released at step A. Currently we were creating the
explicit lock in step-B which keeps a window when the key is not locked
allowing other transaction to lock the row. We must convert implicit to
explicit lock before step-A.
FIX
---
We are fixing the problem with these three steps
1) When reverting back the change done in cluster record we were
leaving the protection of the implicit lock when we did
a commit in the function row_undo_mod_clust() which enabled
other connection to acquire a lock on the row. So we try to
do a implicit to explicit conversion before the commit.
2) For secondary index records we don't allow the implicit to explicit conversion
if the record is delete marked.
3) Regression caused by IODKU patch (# Bug #29718243 MYQL SERVER CRASHING)
is fixed by not allowing temporary tables to do implicit to explicit conversions
since temporary tables are per session.
#rb 22659 22909
Reviewed by : Debarun Banerjee <debarun.banerjee@oracle.com>
Change-Id: I6654b89af3b3a7e3d8add3cc47fce769280f1d5b
… 5.6 & 5.7
Patch for 5.7
Workaround for compiler bug
ld.so.1: mysqld: fatal: relocation error: file sql/mysqld:
symbol OPENSSL_sk_new_null: referenced symbol not found
openssl/safestack.h has lots of pragma weak <function>.
All are tagged as WEAK in generated binaries.
Taking the address of OPENSSL_sk_new_null solves the problem.
We do it in sql/net_serv.cc because this file is used by both
client and server and "embedded" versions of client and server.
Change-Id: I63e1a1bf5ecd29bf7c3f0697295c533c4401af41
Additional patch clarifying what "system" SSL means on Windows. Change-Id: I809847f1097d98062bfcc695b18c957f1eab899f (cherry picked from commit 19200be63510fa63f150cc591559062733c62b4a)
Problem: If we flush a table for EXPORT which has delete-marked rows. During flush, PageConverter read the pages and try to purge delete-marked records, only if it is possible to do so without re-organising the B+tree otherwise increase m_n_purge_failed counter in index stats. Later during IMPORT tablespace, we check m_n_purge_failed counter and purge any delete-marked records that couldn't be purged during the page conversion phase. To purge all delete-marked records, IndexPurge::garbage_collect() iterate over all the record, if a record is delete-marked then purge that record. If any page is corrupted and we position the cursor to that page then we hit an assert : ut_a(btr_page_get_prev(next_page, mtr) == btr_pcur_get_block(cursor)->page.id.page_no()) Instead of hitting any assert, we should terminate import tablespace operation and throw an error. Fix: Converted this assert check to if condition check for import operation. If a page is corrupted then we terminate import tablespace operation and throw an error. RB: 22895 Reviewed by: Mayank Prasad <mayank.prasad@oracle.com>
Problem and analysis ======================================================================== WL #12903 "Support compilation with OpenSSL 1.1.1 for MYSQL 5.6 & 5.7" introduces support for OpenSSL 1.1.1 in MySQL 5.7. Group Replication/XCom also uses OpenSSL, but the necessary changes for GR/XCom to properly support OpenSSL 1.1.1 were not included in the WL #12903 commit. Solution ======================================================================== Disable TLS 1.3 and its ciphersuites. RB: 22897 Reviewed-by: Tiago Jorge <tiago.jorge@oracle.com> Reviewed-by: Venkatesh Venugopal <venkatesh.venugopal@oracle.com>
… 5.6 & 5.7 Additional patch for 5.7 The 'mysqlxtest' client also needed the "-Wl,--undefined," trick. Change-Id: I94b5b6b3e765f5572ad6675e91ad5856486d4f65
This also involves removing COPYING and replacing it with the License Book. Packaging changes accordingly. Change-Id: I501aa36117634e915a12bc74a9bce351001ed446
This also involves removing COPYING and replacing it with the License Book. Packaging changes accordingly. Change-Id: I7280e1d4ffaaa5adfb887848a063531285190911
… 5.6 & 5.7
Patch for 5.6
Workaround for compiler bug
ld.so.1: mysqld: fatal: relocation error: file sql/mysqld:
symbol OPENSSL_sk_new_null: referenced symbol not found
openssl/safestack.h has lots of pragma weak <function>.
All are tagged as WEAK in generated binaries.
Taking the address of OPENSSL_sk_new_null solves the problem.
We do it in sql/net_serv.cc because this file is used by both
client and server and "embedded" versions of client and server.
Change-Id: I49b27dade048d8e27715ecfe3d6bd6c076cb7b55
(cherry picked from commit f85288135b54b021e2ba49e6583a07a9d8fe1aa6)
Additional patch clarifying what "system" SSL means on Windows. Change-Id: I809847f1097d98062bfcc695b18c957f1eab899f (cherry picked from commit 19200be63510fa63f150cc591559062733c62b4a)
Reviewed-by: Terje Rosten <terje.rosten@oracle.com> (cherry picked from commit d19e0699ea16ac33a3a7f3e29d899c7130fc9d90)
This also involves removing COPYING and replacing it with the License Book. Packaging changes accordingly. Change-Id: I501aa36117634e915a12bc74a9bce351001ed446 (cherry picked from commit cdbce8c9ce28e50e88924d4c1e5a821fedb89beb)
Analysis ======== SET READ_ONLY=1/SUPER_READ_ONLY = 1 under certain conditions does not block concurrent DDL executed by users without SUPER privileges. Consider the following scenario: connection 1(SUPER user) FLUSH TABLES WITH READ LOCK; connection 2:(Non SUPER user) CREATE TABLE t1 (fld1 INT); ---> Waits for GRL connection 3: (SUPER user) SET READ_ONLY=1; --> Succeeds connection 1(SUPER user) UNLOCK TABLES; --> Unblocks connection 2. CREATE TABLE tries to acquire IX in the GLOBAL namespace and is blocked because of the SHARED lock GLOBAL namespace held by FLUSH TABLES. SET READ_ONLY succeeds because the SHARED lock in GLOBAL and COMMIT namespace is granted. UNLOCK TABLES then releases the GRL held by FLUSH TABLES, which causes the the CREATE TABLE statement to resume without checking for the read_only state. Thus the CREATE TABLE statement succeeds even though the read_only has been enabled. This is a typical race condition situation where DDL checks 'read only' flag before acquiring IX lock in the GLOBAL namespace which protects from this flag being changed concurrently. This problem doesn't affect DML operations as there is an additional check for 'read_only' flag for such statements in mysql_lock_tables() (i.e. when IX lock in the GLOBAL namespace already has been acquired). Please note that this problem is less prominent in 8.0, where it affects only DDL which doesn't update any user or system table (for example, "DROP TABLE IF EXISTS no_such_table"). Fix === The read_only state is checked after the IX lock is acquired in the GLOBAL namespace in the following functions: lock_schema_name(), lock_tablespace_name() and lock_table_names() which provides protection against concurrent modification of the 'read_only' flag. Change-Id: I49a45aa627972d55e11b348bc2f5e5714215eae3
Analysis ======== Post push fix: FLUSH TABLES .. WITH READ LOCK and FLUSH TABLES .. FOR EXPORT which was blocked in read_only and super_read_only mode by the previous patch is now fixed i.e these operations are allowed in read_only/super_read_only mode. Change-Id: I026998bd4ac438d7b23c83bb6ebd8e8b4e16c4fa
RB : 23019 Reviewed by : Aditya A <aditya.a@oracle.com>
Approved By: Erlend Dahl <erlend.dahl@oracle.com>
Added tokudb_cache_size to the list of variables Verified link
PS-5521 tokudb_cache_size missing from documentation
PS-1673 LP #1520467:ps_tokudb_admin cannot install tokudb remotely
PS-4541 Documentation for maximum- and minimum- option specifiers can…
PS-4842 no default is listed for corrupt action
…TIALIZATIONS OF VARIABLES Issue ===== The testcases gr_autostart_on_install, gr_user and gr_readmode_on_autostart are failing Windows platform when build with with openssl-1.1.1d-static Group Replication started with 'group_replication_ssl_mode=REQUIRED' mode. The issue is because of variables plugin_is_being_uninstalled and plugin_is_waiting_to_set_server_read_mode are not getting reset in plugin_group_replication_init() (called when `INSTALL PLUGIN group_replication..` is executed) and their static initialization in plugin.cc. Solution ======== Remove static initialization of plugin variables plugin_is_being_uninstalled and plugin_is_waiting_to_set_server_read_mode in plugin.cc and reset them in plugin_group_replication_init(). RB: 23559
… SASL We have recently upgraded to build with OpenSSL 1.1.1 on several platforms. This means that the "system" versions of libraries like LDAP and SASL can no longer be used (they link with other versions of SSL) This patch is for 5.7: add cmake options WITH_LDAP and WITH_SASL, assuming these have been built with the same version of SSL libraries used for the server. Look up static (.a) libraries for LDAP, LBER, and SASL, and link these into appropriate plugins. We also enable some tests which were previously skipped because of bad feature tests. Change-Id: If5deba8ae593b2c666ce12fab6ac7827e41b4f99
…after the locks are taken Added sentence to opening paragraph based on the description in the Jira ticket.
PS-1761 LP #1640062: Document that QRT accounts query execution time …
PS-5952, PS-5956: Fix utility user bugs (5.6)
PS-1466 LP #1264306: Per Session innodb_flush_log_at_trx_commit poorl…
…_from_client_to_devel_5.6 PS-6750 obsolete mariadb-connector-c-devel
https://jira.percona.com/browse/PS-6104 *** Updated man pages from MySQL Server 5.6.47 source tarball. *** Updated 'scripts/fill_help_tables.sql' from MySQL Server 5.6.47 source tarball.
…olved) https://jira.percona.com/browse/PS-6104 *** VERSION raised to "5.6.47-87". PERCONA_INNODB_VERSION in univ.i raised to "87". *** Fixed `main.events_bugs` and `main.percona_innodb_version` *** Re-recorded `main.max_statement_time_func`, `innodb.xtradb_compressed_columns_mysqldump` and `innodb.xtradb_compressed_columns_with_dictionaries` after changes in `sql/share/errmsg-utf8.txt`
https://jira.percona.com/browse/PS-5937 *** Updated man pages from MySQL Server 5.7.29 source tarball. *** Updated 'scripts/fill_help_tables.sql' from MySQL Server 5.7.29 source tarball.
Implemented PS-6104 (Merge MySQL 5.6.47) part 1
inikep
force-pushed
the
merge-mysql-5.7.29-part2
branch
from
d8239df to
afb8882
Compare
…olved) https://jira.percona.com/browse/PS-5937 *** VERSION raised to "5.7.29-32". PERCONA_INNODB_VERSION in univ.i raised to "32". *** Kept our solution for PS-6052 and merged upstream comments. `PS-6052: gcc 9.2.1 reports -Werror=stringop-overflow in os_file_get_parent_dir` https://jira.percona.com/browse/PS-6052 (commit 5b83a01) https://bugs.mysql.com/bug.php?id=97466 *** Reverted changes at percona@36c77ad10d1 from https://jira.percona.com/browse/PS-5844 *** Reverted changes from https://jira.percona.com/browse/PS-1697 at percona@ee75a7f93ab in favor of mysql/mysql-server@bd24fc67e68 (https://bugs.mysql.com/bug.php?id=76142) *** Fixed `percona_extended_innodb_status`, `main.events_bugs`, `innodb.bug76142` and `encryption.innodb-bad-key-change3` *** Re-recorded `rpl_encryption.rpl_gtid_validate_slave_gtids`, `keyring_vault.table_encrypt_4_directory` and `keyring_vault.table_encrypt_4`
inikep
force-pushed
the
merge-mysql-5.7.29-part2
branch
from
afb8882 to
6d54f49
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://jira.percona.com/browse/PS-5937
Updated man pages from MySQL Server 5.7.29 source tarball.
Updated 'scripts/fill_help_tables.sql' from MySQL Server 5.7.29 source tarball.
VERSION raised to "5.7.29-32".
PERCONA_INNODB_VERSION in univ.i raised to "32".
Kept our solution for PS-6052 and merged upstream comments.
PS-6052: gcc 9.2.1 reports -Werror=stringop-overflow in os_file_get_parent_dirhttps://jira.percona.com/browse/PS-6052 (commit 5b83a01)
https://bugs.mysql.com/bug.php?id=97466
Reverted changes from https://jira.percona.com/browse/PS-1697 at ee75a7f93ab
in favor of mysql/mysql-server@bd24fc67e68 (https://bugs.mysql.com/bug.php?id=76142)
Reverted changes at 36c77ad10d1 from https://jira.percona.com/browse/PS-5844
Fixed
percona_extended_innodb_status,main.events_bugs,innodb.bug76142andencryption.innodb-bad-key-change3Re-recorded
rpl_encryption.rpl_gtid_validate_slave_gtids,keyring_vault.table_encrypt_4_directoryandkeyring_vault.table_encrypt_4