[pull] 5.7 from xelabs:5.7 #2
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
Additional patch clarifying what "system" SSL means on Windows. Change-Id: I809847f1097d98062bfcc695b18c957f1eab899f (cherry picked from commit 19200be63510fa63f150cc591559062733c62b4a)
… 5.6 & 5.7
Patch for 5.7
Workaround for compiler bug
ld.so.1: mysqld: fatal: relocation error: file sql/mysqld:
symbol OPENSSL_sk_new_null: referenced symbol not found
openssl/safestack.h has lots of pragma weak <function>.
All are tagged as WEAK in generated binaries.
Taking the address of OPENSSL_sk_new_null solves the problem.
We do it in sql/net_serv.cc because this file is used by both
client and server and "embedded" versions of client and server.
Change-Id: I63e1a1bf5ecd29bf7c3f0697295c533c4401af41
… 5.6 & 5.7 Additional patch for 5.7 The 'mysqlxtest' client also needed the "-Wl,--undefined," trick. Change-Id: I94b5b6b3e765f5572ad6675e91ad5856486d4f65
- Update the server compat version Reviewed-by: Terje Rosten <terje.rosten@oracle.com>
This also involves removing COPYING and replacing it with the License Book. Packaging changes accordingly. Change-Id: I7280e1d4ffaaa5adfb887848a063531285190911
Problem and analysis ======================================================================== WL #12903 "Support compilation with OpenSSL 1.1.1 for MYSQL 5.6 & 5.7" introduces support for OpenSSL 1.1.1 in MySQL 5.7. Group Replication/XCom also uses OpenSSL, but the necessary changes for GR/XCom to properly support OpenSSL 1.1.1 were not included in the WL #12903 commit. Solution ======================================================================== Disable TLS 1.3 and its ciphersuites. RB: 22897 Reviewed-by: Tiago Jorge <tiago.jorge@oracle.com> Reviewed-by: Venkatesh Venugopal <venkatesh.venugopal@oracle.com>
PROBLEM
-------
The root cause of the problem was that delete marked rows can acquire a
external read lock at the stage where partial rollback is not complete
In partial rollback when we try to convert implicit lock to explicit
we get a assert saying that already it is locked by a external read lock.
1. For Secondary Index: During rollback, we can remove delete marked key (that is ok
to purge) even if the transaction hasn't modified it. In such case it is not
right to convert to explicit lock since the transaction is not holding
implicit lock on the key.
2. For Cluster Index: If INSERT has modified an existing delete marked key, then
during rollback there are 2 steps.
A. Rollback the update replacing with previous key (delete mark)
B. Remove the delete mark on the row if it is ok to purged
The implicit lock is released at step A. Currently we were creating the
explicit lock in step-B which keeps a window when the key is not locked
allowing other transaction to lock the row. We must convert implicit to
explicit lock before step-A.
FIX
---
We are fixing the problem with these three steps
1) When reverting back the change done in cluster record we were
leaving the protection of the implicit lock when we did
a commit in the function row_undo_mod_clust() which enabled
other connection to acquire a lock on the row. So we try to
do a implicit to explicit conversion before the commit.
2) For secondary index records we don't allow the implicit to explicit conversion
if the record is delete marked.
3) Regression caused by IODKU patch (# Bug #29718243 MYQL SERVER CRASHING)
is fixed by not allowing temporary tables to do implicit to explicit conversions
since temporary tables are per session.
Reviewed by : Debarun Banerjee <debarun.banerjee@oracle.com>
…nodb_zip.wl5522_zip and innodb.alter_missing_tablespace Problem: As part of DISCARD TABLESPACE, MySQL deinitialize table statistics. Later if we need to access the table (MySQL allows DDL) those variables won't have been initialized again. This is a regression introduced by PS-3829 at [1]. Solution: Call dict_stats_init even if the table has been discarded. This will result in dict_stats_report_error been called, which as part of the error handling will init empty table statistics. As part of the fix, we need to suppress warnings that will be generated by dict_stats_report_error on related tests. [1]: commit c7f44ee Author: Robert Golebiowski <robert.golebiowski@percona.com> Date: Thu Nov 16 17:53:11 2017 +0100 PS-3829 : Innodb key rotation. ALPHA This PR implements Encryption threads for encryption/re-encryption/decryption of innodb tablespaces. It is based on MariaDB implementation ported from Google's patch. With this WL user can: 1) Change default KEYRING encryption key 2) Create a new table with KEYRING 3) Encrypt offline already existing tables with KEYRING (with alter) 4) Encrypt/re-encrypt online already existing tables with KEYRING (with innodb_online_encryption, innodb_online_encryption_rotate_key_age variables and innodb_online_encryption_threads) – including tables already encrypted with Master Key encryption. 5) Re-encrypt online already encrypted tables with newer version of encryption key (with variables innodb_online_encryption variable, innodb_online_encryption_threads, innodb_online_encryption_rotate_key_age). This WL also fixed the following bugs reported to MariaDB: (MDEV-17231) Encryption threads keep re-encrypting/encrypting corrupted pages (MDEV-17235) Data dictonary is not updated with encryption flags (MDEV-17234) In memory crypt_data is updated before page0 is updated (MDEV-17233) Page 0 is updated more than once when encryption completes (MDEV-17230) encryption_key_id from alter is ignored by encryption threads (MDEV-17229) Encryption threads ignore innodb_default_encryption_key_id
Analysis ======== SET READ_ONLY=1/SUPER_READ_ONLY = 1 under certain conditions does not block concurrent DDL executed by users without SUPER privileges. Consider the following scenario: connection 1(SUPER user) FLUSH TABLES WITH READ LOCK; connection 2:(Non SUPER user) CREATE TABLE t1 (fld1 INT); ---> Waits for GRL connection 3: (SUPER user) SET READ_ONLY=1; --> Succeeds connection 1(SUPER user) UNLOCK TABLES; --> Unblocks connection 2. CREATE TABLE tries to acquire IX in the GLOBAL namespace and is blocked because of the SHARED lock GLOBAL namespace held by FLUSH TABLES. SET READ_ONLY succeeds because the SHARED lock in GLOBAL and COMMIT namespace is granted. UNLOCK TABLES then releases the GRL held by FLUSH TABLES, which causes the the CREATE TABLE statement to resume without checking for the read_only state. Thus the CREATE TABLE statement succeeds even though the read_only has been enabled. This is a typical race condition situation where DDL checks 'read only' flag before acquiring IX lock in the GLOBAL namespace which protects from this flag being changed concurrently. This problem doesn't affect DML operations as there is an additional check for 'read_only' flag for such statements in mysql_lock_tables() (i.e. when IX lock in the GLOBAL namespace already has been acquired). Please note that this problem is less prominent in 8.0, where it affects only DDL which doesn't update any user or system table (for example, "DROP TABLE IF EXISTS no_such_table"). Fix === The read_only state is checked after the IX lock is acquired in the GLOBAL namespace in the following functions: lock_schema_name(), lock_tablespace_name() and lock_table_names() which provides protection against concurrent modification of the 'read_only' flag. Change-Id: I49a45aa627972d55e11b348bc2f5e5714215eae3
Also fixes the use of permission parameters where it is actually used now.
Fixed release date on 27-30 This change is specific to the 5.7 release notes and does not require any change in 8.0.
This could be useful when the server crashed with a corrupted rollback file, and is unable to start up. Specifying --tokudb--force-recovery=6 --super-read-only should start it up in a read-only, but usable state. Some data may be lost and unrecoverable. Starting the server without the read only option is NOT supported.
PS-5932: Update PerconaFT submodule (5.6)
PS-5932: Implementing --tokudb-force-recovery=6 to skip reading the logs
ps-5.7-5797 Correction on release note's release date
This fixes some 32-64 bit differences, and the 5.7->8.0 redo log encryption upgrade encryption issue. The new table_encrypt_upgrade test verifies that existing tablespaces using the V2 format will be upgraded to V3 when rebuilding, but not before. No change needed for 8.0: it already uses V3.
PS-5910: Support V3 encryption header
These variables make it possible to override reconds_in_range, effectively skipping its computation. This could be useful in some cases where reconds_in_range takes up most of the query cost.
Changed layout of tables to list-table Added items from spreadsheet made during the 8.0 GA Updated list based on comments This is a 5.7 ONLY update. There is no 8.0 merge.
PS-5584 Update PS-5.7 feature comparison
Analysis ======== Post push fix: FLUSH TABLES .. WITH READ LOCK and FLUSH TABLES .. FOR EXPORT which was blocked in read_only and super_read_only mode by the previous patch is now fixed i.e these operations are allowed in read_only/super_read_only mode. Change-Id: I026998bd4ac438d7b23c83bb6ebd8e8b4e16c4fa
Added a note below the innodb_undo_log_encrypt variable definition. Rewrote the note to fix the comment about the keyring.
…cona server Added distribution process.
…cona Server - 5.7 Merged with 5.6 version Added the distribution process
PS-5427 Document the correct distribution upgrade process for Percona…
Fix 3-way deadlock that can be achieved with 2 slave threads working and parallel and with 1 slave client that executes LOCK BINLOG FOR BACKUP. And the deadlock is: worker0: applying INSERT INTO t1 VALUES(11, NULL); worker1: applying INSERT INTO t1 VALUES(12, NULL); worker1: calls backup_binlog_lock.acquire_protection() worker1: waits for worker0 in wait_for_its_turn() client: executes LOCK BINLOG FOR BACKUP client: waits in backup_binlog_lock.acquire(), but protection is acquired by worker1 worker0: calls backup_binlog_lock.acquire_protection(), but it's blocked by client
…ite is disabled https://jira.percona.com/browse/PS-3411 Unnecessary xb_doublewrite file is no more created when innodb_doublewrite option is disabled.
PS-3411: Prevent creation of xb_doublewrite file when innodb_doublewrite is disabled (5.7)
…_from_client_to_devel_5.6 PS-6750 obsolete mariadb-connector-c-devel
…_from_client_to_devel_5.7 PS-6750 move mysql_config from client to devel pkg
https://jira.percona.com/browse/PS-6150 https://jira.percona.com/browse/PS-6093 Two related issues fixed: 1. When printing out locked mutexes (SHOW ENGINE INNODB STATUS), LATCH_ID_AUTOINC object was cast to wrong type in sync0arr.cc sync_array_cell_print(), because of wrong request_type stored in cell. That caused calling GenericPolicy<Mutex>::to_string() instead of AggregateMutexStatsPolicy<Mutex>::to_string(). As the result of wrong cast, m_id member of policy used inside to_string() contained random value. This value was used to index the array in sync_latch_get_meta() which caused memory access violation. 2. When locking of the mutex reaches OS level, sync_array cell is created in ut0mutex.ic TTASEventMutex<Policy>::wait(), LATCH_ID_BUF_POOL_ZIP is mapped to SYNC_BUF_BLOCK cell type. This causes proper cast of mutex in sync0arr.cc sync_array_cell_print() to AggregateMutexStatsPolicy<Mutex>::to_string(). However to_string() method did not service LATCH_ID_BUF_POOL_ZIP causing assertion. MTR test covering this case was not implemented, because the problem shows up when innodb internal threads lock zip_mutex (buf_flush_page_cleaner_coordinator thread and buf_lru_manager). Locking of zip_mutex on OS level can be reached by heavy insertion into the compressed table (and setting innodb_lru_scan_depth to high value) which will cause InnoDB to aggresively flush buffer pool pages. However, as locking threads are InnoDB internal threads, and DEBUG_SYNC facility is thread aware, there is no way to setup debug sync points from MTR test level.
PS-5840 - Memory leak in after 'main.threadpool_debug' - in 'Thread_p…
PS-6150: MySQL crash - sync_mutex_to_string
https://jira.percona.com/browse/PS-6104 *** Updated man pages from MySQL Server 5.6.47 source tarball. *** Updated 'scripts/fill_help_tables.sql' from MySQL Server 5.6.47 source tarball.
…alter_table_dict()' The problem If online alter fails to allocate undo log, heap memory allocated to index object won't be freed. This can be simulated by using debug point 'ib_create_table_fail_too_many_trx' prior to an alter table. Solution Prior to set index pointer to null, call dict_mem_index_free to correctly free memory objects allocated by the index.
PS-5844 Memory leak after 'innodb.alter_crash' - in 'prepare_inplace_…
…in sha256_password_authenticate The problem Valgrind is complaining about an uninitialised value of plain_text at sha256_password_authenticate. Solution Initialize plain_text with empty string.
…olved) https://jira.percona.com/browse/PS-6104 *** VERSION raised to "5.6.47-87". PERCONA_INNODB_VERSION in univ.i raised to "87". *** Fixed `main.events_bugs` and `main.percona_innodb_version` *** Re-recorded `main.max_statement_time_func`, `innodb.xtradb_compressed_columns_mysqldump` and `innodb.xtradb_compressed_columns_with_dictionaries` after changes in `sql/share/errmsg-utf8.txt`
https://jira.percona.com/browse/PS-5937 *** Updated man pages from MySQL Server 5.7.29 source tarball. *** Updated 'scripts/fill_help_tables.sql' from MySQL Server 5.7.29 source tarball.
Added a note to the docker run section of the installation/docker doc.
Implemented PS-6104 (Merge MySQL 5.6.47) part 1
https://jira.percona.com/browse/PS-6150 1. MTR test was failing on builds where DEBUG SYNC facility is not present. Added proper MTR test guard. 2. Fixed MTR test stability
Merge remote-tracking branch 'pb_remote/ps-5.6-4691' into ps-5.7-4691
5.7 PS-6773 - Conditional jump or move depends on uninitialised value(s) …
…h-fix-5.7 Post push fix: PS-6150: MySQL crash - sync_mutex_to_string 5.7
…olved) https://jira.percona.com/browse/PS-5937 *** VERSION raised to "5.7.29-32". PERCONA_INNODB_VERSION in univ.i raised to "32". *** Kept our solution for PS-6052 and merged upstream comments. `PS-6052: gcc 9.2.1 reports -Werror=stringop-overflow in os_file_get_parent_dir` https://jira.percona.com/browse/PS-6052 (commit 5b83a01) https://bugs.mysql.com/bug.php?id=97466 *** Reverted changes at percona@36c77ad10d1 from https://jira.percona.com/browse/PS-5844 *** Reverted changes from https://jira.percona.com/browse/PS-1697 at percona@ee75a7f93ab in favor of mysql/mysql-server@bd24fc67e68 (https://bugs.mysql.com/bug.php?id=76142) *** Fixed `percona_extended_innodb_status`, `main.events_bugs`, `innodb.bug76142` and `encryption.innodb-bad-key-change3` *** Re-recorded `rpl_encryption.rpl_gtid_validate_slave_gtids`, `keyring_vault.table_encrypt_4_directory` and `keyring_vault.table_encrypt_4`
PS-4691 add '--stop-timeout 600' option to 'docker run' command
Implemented PS-5937 (Merge MySQL 5.7.29) part 1
Implemented PS-5937 (Merge MySQL 5.7.29) part 2
Backport percona server 5.7.29-32
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )