PS-7332: Merge MySQL 5.7.32 part 2 #4117
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Approved By: Erlend Dahl <erlend.dahl@oracle.com>
Approved By: Erlend Dahl <erlend.dahl@oracle.com>
Approved By: Erlend Dahl <erlend.dahl@oracle.com>
Problem
-------
In case of MEMORY TABLES, the table content is deleted after every server
restart and to make sure that both master-slave are in sync we log an
implicit DELETE statement in the binary log of the server which just has
restarted.
There are some issues with the above implementation :
I1. DELETE statement is always be written in statement format even if the
binlog format is ROW.
I2. DELETE statement is always written to the binary log even if
(super_)read_only is set to 1.
Fix
---
1. To fix I1 we are changing DELETE to TRUNCATE because it is clearly
mentioned in MYSQL documentation that truncate statement is always written
in statement format in the binary log.
2. We do not address issue I2, We argue that the violation of read_only
mode happened already when the table data was erased. The write to the
binary log is only reflecting that. It should not be expected that
MEMORY tables obey the rules of read_only. If this induces problems for
any user, the user should disable the MEMORY engine.
RB#24592
The problem was that LOCK TABLES statement on a view did insufficient privilege checking for view's underlying tables. This patch solves the problem by ensuring that when we lock tables through a view we require the same SELECT and LOCK TABLES privileges on its underlying tables as would have been required if these tables were locked directly. To avoid breaking mysqldump/pump tools which need to be able to dump views with broken definer, which in default mode involves locking of such views, we accept both cases when: 1) User whose context will be used for view execution has necessary privileges on underlying tables. 2) User who originally inititated LOCK TABLES operation has them (which is typically for admin/backup users used for dumping). This is version of the patch for 5.6 branch. Reviewed-by: Sivert Sorumgaard <sivert.sorumgaard@oracle.com> Reviewed-by: Gopal Shankar <gopal.shankar@oracle.com>
The problem was that LOCK TABLES statement on a view did insufficient privilege checking for view's underlying tables. This patch solves the problem by ensuring that when we lock tables through a view we require the same SELECT and LOCK TABLES privileges on its underlying tables as would have been required if these tables were locked directly. To avoid breaking mysqldump/pump tools which need to be able to dump views with broken definer, which in default mode involves locking of such views, we accept both cases when: 1) User whose context will be used for view execution has necessary privileges on underlying tables. 2) User who originally inititated LOCK TABLES operation has them (which is typically for admin/backup users used for dumping). This is version of the patch for 5.7 branch. Reviewed-by: Sivert Sorumgaard <sivert.sorumgaard@oracle.com> Reviewed-by: Gopal Shankar <gopal.shankar@oracle.com>
Problem: 1. TABLE objects are cached in table cache and transaction handle (trx) objects are cached in trx pool. 2. When we close a connection we do not reset (innobase) file handler and row_prebuilt (named m_prebuilt) object. 3. m_prebuilt keep pointer to innodb trx (which might have released back to trx pool) 4. New connection (S1) get unused TABLE object from table cache and reuse innodb file handler, m_prebuilt and trx used by the old connection. New connection hold trx* in m_prebuilt, which is not allocated to this session. 5. If at the same time, this trx is allocated to another session (S2) then two session (S1 & S2) use same trx object at the same time. 6. S2 executes a query and calls ha_innobase::start_stmt(), at the same time, if S1 executes ha_innobase::end_stmt() then we make trx object inconsistent and might hit an assert . Fix: 1. If we load TABLE object from table cache, we reset some fields of the m_prebuilt and update the transaction fields (trx) in the m_prebuilt. RB: #24552 Reviewed by : Debarun Banerjee <debarun.banerjee@oracle.com> Reviewed by : Gopal Shankar <gopal.shankar@oracle.com>
Issue ===== We make use of prebuilt->m_end_range flag to check if end_range condition is not satisfied if the number of uncommitted rows we've passed during the scan is > 100 to avoid full table traversal. In the call to row_search_mvcc if we see that the flag is set we unset it and immediately return indicating that there are no more row to be fetched for the the given range. This is benefecial if we have prefetched rows that we need to return to the server. In such a case, every time the server calls row_search_mvcc, after fetching the first row, if there are any cached rows we return the cached row, and when we've returned all the cached rows we check if the end_range flag is set and if it's we unset it and return. Turns out that this flag is not reset at the beginning of every index read resulting in some reads being aborted because of this and resulting in missing rows. One such case is when optimiser decides to use Multi-Range-Read (MRR) optimisation as the range query specified by the user is split into multiple range queries. Fix === In row_search_mvv() reset the prebuilt->m_end_range cache at the start of every index read. RB: 24201 Reviewed-by: Debarun Bannerjee <debarun.bannerjee@oracle.com>
Issue here is that in read_only/super_read_only mode truncate operation on performance schema was prohibited even for users having appropriate privileges for truncating such table. While in 5.7.28 truncate operations on performance schema tables were allowed even in read_only/super_read_only mode Problem was caused by the fix for Bug28438114 which added an extra check of read only mode at the point where TRUNCATE TABLE operation acquires stronger metadata lock on the table to be truncated. To fix the issue, for truncate table operation read only state check is skipped if table belongs to performance schema. Change-Id: I70c6600ccc46ca64ca7951fbce90073ddb3065eb
Group_member_info_manager::get_group_member_info_by_member_id() returns a internal pointer to a Group_member_info entry, that entry can be removed without the pointer holder know. To solve the above issue, now the method does return a copy of the internal instance. RB: 24701
Description: Some of the certificates used for testing
CRL support are expired. This caused various
tests to failed.
Fix: - Added new set of certificates
- Updated read-me for CRL generation
RB: 24714
…ORT_LENGT SET Description: Assertion fialure for a query with ORDER By clause when server is started with SORT_BUFFER_SIZE = 1125899906842624 and MAX_SORT_LENGTH SET is set. ANALYSIS: If SORT_BUFFER_SIZE and MAX_SORT_LENGTH SET is set with some value because of which variable Sort_param.max_keys_per_buffer in filesort() can get value '0' then it asserts at statement "n < m_size" in "Bounds_checked_array<unsigned char*>::operator[]". So reason for this is we allow to create sort buffer of size '0' but we assert when it's size is '0'. SOLUTION: Changed the following statement in filesort(). - if (table_sort.get_sort_keys()) + if (table_sort.sort_buffer_size() > 0 && table_sort.get_sort_keys()) This change will ensure we continue to search and allocate the sort buffer if it's value is not greater than '0'. Change-Id: I87d6a19839a7476d09aca49b69e6a3feefdd8bde
Buffer pool may remove a page from memory, and even free the memory in which it resided unless it is marked as "buffer fixed". To be more precise: buf_block_t struct itself can become freed if the buffer pool is requested to shrink. Serveral places in our code dereferenced a buf_block_t pointer either after unfixing it, or before fixing it, which was not safe. This patch: - moves the accesses that happened after unfixing, to happen before - makes sure that places which store buf_block_t pointer as hint for future reference are structured in a way which prevents races with buffer pool resize. That is it introduces Block_hint class which stores a pointer, and allows accessing this pointer only by passing a functor which is run passing either the original pointer if it was still valid, or a nullptr if the pointer become stale in meantime - adds more assertions about buf_block_t's page is buffer fixed when accessed This patch also fixes Bug #31389823 BUF_BLOCK_IS_UNCOMPRESSED() RETURNS FALSE IN TOO MANY CASES, by simply removing this function and using buf_pointer_is_block_field_instance instead. It also backports important parts of Bug#20735882 REMOVE REDUNDANCY IN mtr0mtr.cc RELATED TO RELEASING MEMO SLOT fix from 8.0 RB: 24661 Reviewed-by: Marcin Babij <marcin.babij@oracle.com> Reviewed-by: Yasufumi Kinoshita <yasufumi.kinoshita@oracle.com>
IGNORING --EXCLUDE_DATABASES AND
--SINGLE-TRANSACTION OPTION
DESCRIPTION
===========
While trying to dump a single schema using mysqlpump, it
validates all database objects irrespective of the specific
database instructed to export using --exclude-databases
and --single-transaction.
ANALYSIS
========
According to the current mysqlpump design, it first
enumerates all the DB objects as part of the
Crawler module and later performs the filtering of the
objects before dumping them. This causes all the database
objects to be validated even if a single database was
instructed to be dumped.
FIX
===
The fix performs object filtering before enumeration
so that only the databases and their corresponding
objects that should be included in the dump are validated.
RB# 24340
PROBLEM ------- 1) We are trying to do a inplace update on a clustered index for an intrinsic table 2) This inplace upgrade is done under a innodb mini transaction (mtr). 3) Since this is an intrinsic table we turn the redo logging off for this mini transaction. We are correctly modifying the clustered index record ,but we need to add this modified pages in flush list ,otherwise because of redo log turned off ,we don't have the changes logged. 4) We were not adding these pages modified under mtr in the flush list, because of which if the buffer pool evicts this page we lose the modified changes and get wrong result. FIX --- Add the modified buffer pool pages in flush list. RB: #24796 Reviewed by : Debarun Banerjee <debarun.banerjee@oracle.com> Reviewed by : Annamalai Gurusami <annamalai.gurusami@oracle.com>
ROW_MYSQL_WHOLE_ROW" IN ROW0MYSQL.CC ANALYSIS: ========= Statements which called two stored routines, one of which tried to create temporary table in InnoDB SE and then to modify it and another routine simply read from the same table, resulted in assertion failure when statement or mixed binary log format were used. The assertion has been triggered because SQL-layer passed to InnoDB SE incorrect information about the operation type on the temporary table in such cases. The underlying issue lies in building the prelocking list in such cases. The presence of CREATE TEMPORARY TABLE inside the first of routines makes the server think that the table might not exist thus it does not add it to the prelocking list in this case. OTOH the usage of the same table in another routine causes its addition to prelocking list with a lock type indicating read access. As a result, at prelocking stage we only see one table instance in prelocking list which is marked as to be read. We pass this information to SE by calling handler::store_lock()/ external_lock() as a side-effect of calling mysql_lock_tables()). Later, when we try to access the same table for modification from the first of routines, InnoDB considers this as breach of contract and asserts. (The actual assertion is about whether the whole row is to be used for the operation or not and thus masked when --binlog_format=row as we have to use the whole row in this case anyway). FIX: ==== Do not call handler::store_lock()/external_lock() for the temporary tables in the extended prelocking list during prelocking stage. Delay informing SE about operation being perfomed (by calling handler::start_stmt()) till the point when the table is accessed during the execution of substatements. Approved by: Dmitry Lenev (dmitry.lenev@oracle.com) Sivert Sorumgaard (sivert.sorumgaard@oracle.com) Change-Id: I029f45662678f9fc5150f1ee2c7db33a11a59b67
Change-Id: I87a151cc280d7e2bdb8016c4f21c371a7891213f
IGNORING --EXCLUDE_DATABASES AND
--SINGLE-TRANSACTION OPTION
Post push patch to fix test case failures.
RB# 24340
Code used std::string without including <string>, and this broke the build on mac. Change-Id: Ib0acbbf38e3b2e57225bf192adf313bf1c9b6a8e
Approved By: Erlend Dahl <erlend.dahl@oracle.com>
Between btr_pcur_store_position() and sel_restore_position_for_mysql(), prev_rec might be invalid. If not needed, prev_rec should be null. If needed, should confirm the prev_rec is valid. RB: 24349 Reviewed-by: Debarun Banerjee <debarun.banerjee@oracle.com>
Change-Id: Ie388d199b8ac7d85119334c5a609d7a8492ad628
….49-part2 PS-7012: Merge MySQL 5.6.49 part 2
…AGE SIZE
Issue :
It was a regression (and issue caught) from patch for bug#31313533
in which an assert was added to confirm the input data length for
encryption is of expected size.
FIx:
Made sure the input data length for encryption is sufficient.
RB : 25037
Reviewed by : Darshan M N <darshan.m.n@oracle.com>
Debarun Banerjee <debarun.banerjee@oracle.com>
…STAMP PROBLEM ------- The tests are failing with the result difference in explain output FIX --- We will call analyze on the table before calling explain to stabilize the result. Reviewed by : Chaithra Gopalareddy <chaithra.gopalareddy@oracle.com>
…STAMP Post push fix to modify a result file which was missed in the previous push Reviewed by : Chaithra Gopalareddy <chaithra.gopalareddy@oracle.com>
modified file: yum_repo added a downgrade section
Approved-by: Bjorn Munch <bjorn.munch@oracle.com>
PS-7213 Add jenkins file for autobuilds
5.6 - Fixed PS-7346 - Huge memory usage for audit plugin when large queries…
PS-7374: Travis CI and Azure: add clang-11 testing
https://jira.percona.com/browse/PS-7200 *** Updated man pages from MySQL Server 5.6.50 source tarball. *** Updated 'scripts/fill_help_tables.sql' from MySQL Server 5.6.50 source tarball.
…olved) https://jira.percona.com/browse/PS-7200 *** VERSION raised to "5.6.50-90.0". PERCONA_INNODB_VERSION in univ.i raised to "90.0".
PS-7379: fix compilation issues with clang-11 (5.6)
….50-part1 PS-7200: Merge MySQL 5.6.50 part 1 (up to e453966)
oleksandr-kachan
force-pushed
the
merge-mysql-5.7.32-part2
branch
from
7c4735c to
1fd519f
Compare
https://jira.percona.com/browse/PS-7332 *** Updated man pages from MySQL Server 5.7.32 source tarball. *** Updated 'scripts/fill_help_tables.sql' from MySQL Server 5.7.32 source tarball.
oleksandr-kachan
force-pushed
the
merge-mysql-5.7.32-part2
branch
2 times, most recently
from
c81a47d to
1d25038
Compare
…olved) https://jira.percona.com/browse/PS-7332 *** VERSION raised to "5.7.32-35.0". PERCONA_INNODB_VERSION in univ.i raised to "35.0". *** New error code is added in github.com/mysql/mysql-server/commit/05de35a6c17c862bddaa91c98671e33b3a39cbd8. This lead to shifted MyRocks error codes. Some MyRocks tests were updated. *** Fixed assertion in Partition_base::extra() which appeared after fix from github.com/mysql/mysql-server/commit/27094f693d7ba27f372918817c20e2135e5cf3c0. *** Fixed ASAN detected memory leak in keyring::Buffered_file_io. *** Skip some tests requiring TLSv1 or TLSv1.1 support on systems where default min supported TLS version is higher. *** Disabled encryption.create_or_replace test as encryption threads are not stable in 5.7. *** Fixed innodb.undo_encrypt_basic test. *** The coredumper conflicts with ASAN. As a result main.coredump test hungs. Disable this test in case ASAN is enabled. *** Update PerconaFT submodule to latest version. It contains a fix for tokudb assertion failure when code is built with ASAN support.
Analysis -------- We have observed that when the call to getaddrinfo with an unresolvable hostname ends up using systemd-resolved, the call always returns EAI_AGAIN. When the call to getaddrinfo with an unresolvable hostname ends up *not* using systemd-resolved, the call returns EAI_NONAME. Fix --- Limit the number of repeated calls to getaddrinfo while it returns EAI_AGAIN. (cherry picked from commit e1b1ab8)
oleksandr-kachan
force-pushed
the
merge-mysql-5.7.32-part2
branch
from
1d25038 to
0982ed3
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
https://jira.percona.com/browse/PS-7332
VERSION raised to "5.7.32-35.0".
PERCONA_INNODB_VERSION in univ.i raised to "35.0".
New error code is added in
github.com/mysql/mysql-server/commit/05de35a6c17c862bddaa91c98671e33b3a39cbd8.
This lead to shifted MyRocks error codes. Some MyRocks tests were updated.
Fixed assertion in Partition_base::extra() which appeared after fix from
github.com/mysql/mysql-server/commit/27094f693d7ba27f372918817c20e2135e5cf3c0.
Fixed ASAN detected memory leak in keyring::Buffered_file_io.
Skip some tests requiring TLSv1 or TLSv1.1 support on systems where
default min supported TLS version is higher.
Disabled encryption.create_or_replace test as encryption threads are not stable in 5.7.
Fixed innodb.undo_encrypt_basic test.
The coredumper conflicts with ASAN. As a result main.coredump test hungs.
Skip this test in case ASAN is enabled.
Update PerconaFT submodule to latest version. It contains a fix for
tokudb assertion failure when code is built with ASAN support.