Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLASH-270] Delete Range Data At First while Applying Snapshot #75

Merged
merged 35 commits into from
May 30, 2019
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
350cd73
change getGCSafePoint in MockPDClient.
solotzg May 20, 2019
65c3037
Add del col to secondary sorting expression.
solotzg May 20, 2019
c92453a
Complete operation in memory.
solotzg May 20, 2019
9fbd896
check all table while applying snapshot.
solotzg May 20, 2019
6e9ded9
!!!tmp
solotzg May 21, 2019
3e34280
Merge remote-tracking branch 'pingcap/master' into FLASH-270
solotzg May 21, 2019
9abf554
Revert "!!!tmp"
solotzg May 21, 2019
93f7d9d
!!!tmp
solotzg May 21, 2019
3e85a92
Small fix.
solotzg May 22, 2019
f3569e6
rollback operation about deleting record.
solotzg May 22, 2019
0b10c65
Optimize executor.
solotzg May 22, 2019
7210284
optimize ReplacingTMTSortedBlockInputStream.
solotzg May 22, 2019
6db5de9
optimize for only need handle, tso, delmark.
solotzg May 23, 2019
acf76b6
import TMTSortCursor.
solotzg May 23, 2019
2a27711
finish.
solotzg May 23, 2019
b0c8399
fix.
solotzg May 23, 2019
7759b63
fix.
solotzg May 23, 2019
1019e44
fix.
solotzg May 23, 2019
146690b
Optimize for count(*).
solotzg May 23, 2019
c860f47
fix bug.
solotzg May 23, 2019
d3a77a5
Fix
solotzg May 24, 2019
dacb64c
optimize TMTSortedBlockInputStream.
solotzg May 24, 2019
3a13a98
Optimize log.
solotzg May 24, 2019
2ef4eaa
no need to report after persisted.
solotzg May 25, 2019
b4d3c12
Optimize memory copy in Region::onCommand.
solotzg May 25, 2019
acea86c
more mock test.
solotzg May 25, 2019
189f1b1
optimize snapshot.
solotzg May 26, 2019
92ee816
Fix Region::compareAndCompleteSnapshot.
solotzg May 26, 2019
e73b8c5
fix compile
solotzg May 26, 2019
2bca1d2
Optimize when pk is int64 or uint64.
solotzg May 27, 2019
68b450d
fix compile.
solotzg May 27, 2019
457c872
Small fix: add const to parameter.
solotzg May 29, 2019
5e0d2c8
optimize performance.
solotzg May 29, 2019
3270582
Format.
solotzg May 29, 2019
f2e51a8
address comment.
solotzg May 30, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
optimize TMTSortedBlockInputStream.
solotzg committed May 24, 2019
commit dacb64c4cec26ee4ad29cac13704d311731460b0
13 changes: 6 additions & 7 deletions dbms/src/DataStreams/TMTSortedBlockInputStream.cpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
#include <Columns/ColumnsNumber.h>
#include <DataStreams/TMTSortedBlockInputStream.h>
#include <Storages/MutableSupport.h>


namespace DB
{
@@ -130,20 +128,22 @@ void TMTSortedBlockInputStream::merge_optimized(MutableColumns & merged_columns,

setPrimaryKeyRefOptimized(next_key, current);

if (next_key != current_key)
const auto key_differs = TMTSortCursorPK::cmp(*current_key.columns, current_key.row_num, *next_key.columns, next_key.row_num);

if (key_differs.all)
{
by_row++;

max_version = 0;
/// Write the data for the previous primary key.
if (!MutableSupport::DelMark::isDel(UInt8(max_delmark)))
if (!max_delmark)
insertRow(merged_columns, merged_rows);

if (is_clean_top)
{
/// Delete current cache and return.
/// We will come back later and use current block's data directly.
max_delmark = MutableSupport::DelMark::genDelMark(true);
max_delmark = (UInt8)1;
current_key.reset();
selected_row.reset();
current_row_sources.resize(0);
@@ -211,7 +211,7 @@ void TMTSortedBlockInputStream::merge_optimized(MutableColumns & merged_columns,
}

/// We will write the data for the last primary key.
if (!MutableSupport::DelMark::isDel(UInt8(max_delmark)) && !current_key.empty())
if (!max_delmark && !current_key.empty())
insertRow(merged_columns, merged_rows);

if (cur_block)
@@ -224,7 +224,6 @@ void TMTSortedBlockInputStream::merge_optimized(MutableColumns & merged_columns,
finished = true;
}

// TODO: use MutableSupport::DelMark here to check and generate del-mark
bool TMTSortedBlockInputStream::insertByColumn(TMTSortCursorPK current, size_t & merged_rows, MutableColumns & merged_columns)
{
if (current.notSame(cur_block_cursor))