-
Notifications
You must be signed in to change notification settings - Fork 482
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PS 7798 [5.7] - Merge fb-prod202102, fb-prod202103, fb-prod202104 #4401
Conversation
Upstream commit ID : fb-mysql-5.6.35/64ba7e29cdc2d444e21e100acdbe2351e1c88384 PS-7798 : Merge fb-prod202102 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/0fb83bb7569838026b912af23de52077244693b3 PS-7798 : Merge fb-prod202102 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/f896415fa0d31ca956fbe2e63ee0eed544879f96 PS-7798 : Merge fb-prod202102 Summary: This re-introduces Luqun's get_rocksdb_files.sh (which was reverted due to RocksDB perf regression in 6.17) to support building RocksDB 6.17+ and updates RocksDB to 6.19. The regression in 6.17 was reverted in RocksDB 6.18 so we should be able to proceed with new version. update-submodule: rocksdb Reviewed By: Pushapgl Differential Revision: D27378086 fbshipit-source-id: dd0e0eb7ba3
Upstream commit ID : fb-mysql-5.6.35/f12eea3370f5c147305afdf0727b355bcd2cf4ec PS-7798 : Merge fb-prod202103 Summary: In `inplace_populate_sk`, `ha_index_init` can return an error if the query has been killed, and if that happens, it's not safe to continue with calling `index_first`/`index_next`. The fix is to check for errors and return early if needed. Reviewed By: luqun Differential Revision: D26960027 fbshipit-source-id: aaf7bd24f9d
--disable_warnings | ||
--source include/have_rocksdb.inc | ||
--source include/have_binlog_format_row.inc | ||
--source include/master-slave.inc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
master-slave
should be the last include
--rocksdb_max_subcompactions=1 | ||
--rocksdb_default_cf_options=target_file_size_base=100k;max_bytes_for_level_multiplier=1;max_bytes_for_level_base=1m;target_file_size_multiplier=1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--rocksdb_
should be always --loose_rocksdb_
in case RocksDB is not compiled
--disable_warnings | ||
DROP TABLE IF EXISTS t1; | ||
--enable_warnings |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be removed as it it no longer required.
select * from t1 order by id; | ||
source include/sync_slave_sql_with_master.inc; | ||
|
||
connection slave; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--source include/rpl_connection_slave.inc
select * from t1 order by id; | ||
source include/sync_slave_sql_with_master.inc; | ||
|
||
connection slave; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--source include/rpl_connection_slave.inc
@@ -0,0 +1,53 @@ | |||
connection master; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--source include/rpl_connection_master.inc
select * from t1 order by id; | ||
|
||
|
||
connection master; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--source include/rpl_connection_master.inc
select * from t1 order by id; | ||
|
||
|
||
connection master; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--source include/rpl_connection_master.inc
… MyRocks Manual Compaction Upstream commit ID : fb-mysql-5.6.35/e00196f49460de6eeb91ee6f1e761da576b93ceb PS-7798 : Merge fb-prod202103 Summary: Currently BottommostLevelCompaction is set to default kForceOptimized for manual compaction. This is fine usually. However, for privacy triggered compactions, it does not make much sense, because it compacts Lmax SST files that do not have any overlapping key updates at all. This adds a session variable to specify the option for bottommost level compaction before triggering manual compaction. Porting from 8.0.20 branch Reviewed By: yoshinorim Differential Revision: D27780545 fbshipit-source-id: e5b9e7c6012
Upstream commit ID : fb-mysql-5.6.35/997d25216c3f968fa0439bcba90d25ffd2bbb695 PS-7798 : Merge fb-prod202102 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/e4f83712bced1e13a4a25da545719677d20890ab PS-7798 : Merge fb-prod202104 Summary: Port of D27419569. **Porting Notes** Core Data Demand Efficiency team wants to track physical size of the assoc types and fbtypes, as well as existing logical size. Tracking physical size of the specific assoc/fb types is not available as of now. ***Change*** Added the new function GET_INDEX_SIZE_BY_PREFIX which returns the index size by prefix. Currently we support only numeric keys. Reviewed By: yizhang82 Differential Revision: D27919708 fbshipit-source-id: c1ba5976cbe
Upstream commit ID : fb-mysql-5.6.35/ccfbfe798da49c4ea49483048647b9128850ef5d PS-7798 : Merge fb-prod202104 Summary: As title, stay on the latest rocksdb branch for each full feature as per Yoshi's recommendation. update-submodule: rocksdb Reviewed By: yoshinorim Differential Revision: D28049296 fbshipit-source-id: b74b75b101f
Upstream commit ID : fb-mysql-5.6.35/dcf392a57223ff46cefe6bf07130a3bfd8c40cc9 PS-7798 : Merge fb-prod202104 Summary: see D28049296 (facebook/mysql-5.6@ccfbfe7) discussion, there is a rocksdb change which cause this memtable size difference Reviewed By: yizhang82 Differential Revision: D28155396 fbshipit-source-id: 3267cd53d68
Upstream commit ID : fb-mysql-5.6.35/d45c30966752a04ca931c86552e00df3ef933bd7 PS-7798 : Merge fb-prod202104 Summary: Previously the `mysql_ldb` commands could not operate on a MyRocks instance due to wrong comparator provided. In particular, MyRocks column families can use a mix of "RocksDB_SE_v3.10" and "rev:RocksDB_SE_v3.10"; however, the provided comparator was always "RocksDB_SE_v3.10". This diff registers both possible comparators, which allows each column family's proper comparator to be deserialized from the OPTIONS file. The user must pass `--try_load_options` to their `ldb` command. Reviewed By: yizhang82 Differential Revision: D28313731 fbshipit-source-id: e9219a1199d
Upstream commit ID : fb-mysql-5.6.35/ba9d330448aa82009df5303bef21f952e49d2547 PS-7798 : Merge fb-prod202104 Summary: When hitting some I/O errors in RocksDB (e.g. out of space), current RocksDB may return error codes other than kIOError at engine commit phase, while not returning any error at prepare phase, depending on how it hits errors. Our expectation is any engine commit failure of the two phase commit protocol should abort the process, since binlog commit is already done. This diff makes sure to call abort() to terminate the process on RDB_IO_ERROR_TX_COMMIT at commit phase, not just RDB_IO_ERROR_DICT_COMMIT. This prevents from a case that a transaction gets written to binlog (and replicated) but it is rolled back in MyRocks engine due to commit failure, which is an inconsistent state. With this diff, binlog and engine will be consistent based on binlog state (either rolled forward or rolled back, aligned to binlog state). This diff makes another change that it no longer aborts on systems errors on MyRocks Prepare. Prior to this diff, kIOError at Prepare() terminated the process, but it was not necessary since transactions could be safely rolled back on Prepare() error. Clients hitting errors at Prepare() will get ER_INTERNAL_ERROR with RocksDB error status. Default ER_ERROR_DURING_COMMIT is not returned. This was because ER_ERROR_DURING_COMMIT does not take string arguments so it is difficult for users to understand why it failed. Reviewed By: hermanlee Differential Revision: D28576723 fbshipit-source-id: 1b0b6893f30
Upstream commit ID : fb-mysql-5.6.35/6e53ec0ef23f6a16f966669bd58b72a7a9ced698 PS-7798 : Merge fb-prod202104 Summary: pull in the latest rocksdb version for next full feature (prod202104) update-submodule: rocksdb Reviewed By: yoshinorim Differential Revision: D28689774 fbshipit-source-id: 455d6d93e54
Upstream commit ID : fb-mysql-5.6.35/2451c64901e59e10be92843da298733710cb2bd7 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/623780af3ee8295a98ce407ab46a48e865b9e95e PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/c607a5a9104ee50b87efd2326f741bded737f70e PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/e40167380581697596b398e832d2d35d132f5bc6 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/0b215f8c7eb777e6e4d2202b621c40519c03b0c3 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/e4a49506500c0b8c46b2340b8c15afe3e64e8ed5 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/6b1db2cfa9c1c8fb60e01d09c6b7c2d8cdc2dcae PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/6ab1c7c1ac92693062ccfbcb1de1552cbf0cb162 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/d8d5fed47c5509ee5fab4eb596f3378e877f46e3 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/4cd2cff73fa4d821c1f7bd2ecfcd9f17f98e9fe1 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/9486d32143a49416e399cfa1526d7f9a7c738e31 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/50a8e298885ba5ddf0415d11eee528cd31ccf90b PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/c018587221e551162ac587eab2fcbbd0960fe372 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/77b9160bcaa58d5e2cb9ce262efc6064ca7642b3 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/9d01043b1df6de6537cf223a7b62987d8649de5f PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/acd99073327aaa7c4121696d1996a135caf75555 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/f9fd0ce3069b0a95bcdc3f6efc823b8fb56845a8 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
…covery Upstream commit ID : fb-mysql-5.6.35/d4626033dd8a8e46bee8be0ec0c338994d1cea48 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/2901d9a9463c89e6bd24e918fe02844850f8d981 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/05007850d65b3a69cbaa3e9442aea33abe9250a8 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
…appened Upstream commit ID : fb-mysql-5.6.35/354ea91e0d9ccf471ea08d8724eef7d44291db04 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/e19d09073012954be372d32ba93b38bd497f0685 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/7cdcea348f9ff0c0d3c3b0e8729476229f090819 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/077b8d254efd92cc7c055c8b915ba6262e18cb6e PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/2378c1e8c1e59d55c0659e97316cae5ac787e81b PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/ddda374d0a518c9e13e9fa4563a55ba67e321e24 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/01e535506ca5c4565a9f51cadec9b84d2a748e29 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/73b7ec9184c70acf7199b217778fffb23563e7b2 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/1b80de18f6b009aa75efcb128f26d3f89912f3dd PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/05a8c91dcb787bc3a423765ba05c1b2b858d33de PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/bae825b5b8f332b16ba9e8065d43e1fa7a364867 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/47fc3c7282e0726761c50431f9e046daa533e2aa PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/39ce50d9bffe13c8e89c279ef0810e9ecb16f2dc PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/baaf5dbe0ab754d87a9721e3cb9ecbc876efe464 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/c807ef482fe4e998a804622165264eedf18d62d6 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/0f1f275a0255b88b43b4e20b23b9498558fd9b60 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/124deb25ff19049b287d824273ed1a072ca3fccc PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
…mit ordering Upstream commit ID : fb-mysql-5.6.35/ce8c4740a0c0dfed0d219b7f1d30d05da8ffb40c PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/f43c69d0e5b4bb9c655698933f14fc017306de53 PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
Upstream commit ID : fb-mysql-5.6.35/4edc56e8b7f473ea35bc6629335221c51a5aac2d PS-7798 : Merge fb-prod202104 This is a NULL cherry-pick to Percona Server to create the commit placeholder for the corresponding upstream commit. Reason: Patch not taken into Percona Server
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
https://jira.percona.com/browse/PS-7798
https://github.com/facebook/mysql-5.6/tree/fb-prod202102
https://github.com/facebook/mysql-5.6/tree/fb-prod202103
https://github.com/facebook/mysql-5.6/tree/fb-prod202104
https://ps57.cd.percona.com/view/5.7/job/percona-server-5.7-param/488/
Update rocksdb submodule to c7f8ae9f17743774ac48e2ffc6e150eafcfae416 SequenceIterWrapper should use internal comparator (#8328) (v6.21.0)
Add new MyRocks variable
rocksdb_manual_compaction_bottommost_level
.Cherry-pick the following commits from
fb-prod202104
: