-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ttl: submit ttl scan task to the system table #40422
ttl: submit ttl scan task to the system table #40422
Conversation
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
0f1b28b
to
519a625
Compare
1352e51
to
525d0b8
Compare
The failed test is really wired. The This problem occurs in column "last_job_start_time", which should be NULL and default NULL. However, when selecting it, the |
As the debugger told me, the |
525d0b8
to
71635ea
Compare
reqCtx.buf, err = encodeFromOldRow(m.Value, reqCtx.buf) | ||
if err != nil { | ||
log.Error("encode data failed", zap.Binary("value", m.Value), zap.Binary("key", m.Key), zap.Stringer("op", m.Op), zap.Error(err)) | ||
return nil, err | ||
} | ||
|
||
lock.Value = make([]byte, len(reqCtx.buf)) | ||
copy(lock.Value, reqCtx.buf) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I finally find the bug! Reuse the reqCtx.buf
could make the lock.Value
be overwritten by another row.
It seems that this bug requires 1PC, and set multiple rows in one transaction (if these rows are in different tables, it will panic), so this problem only occurs in the TTL test, in which we set the session variables to enable 1PC explicitly.
It takes me a lot of time to reach here 😢 .
a8770e5
to
f2fe4bd
Compare
@@ -935,16 +935,16 @@ func (store *MVCCStore) buildPrewriteLock(reqCtx *requestCtx, m *kvrpcpb.Mutatio | |||
lock.Op = uint8(kvrpcpb.Op_Put) | |||
} | |||
if rowcodec.IsRowKey(m.Key) && lock.Op == uint8(kvrpcpb.Op_Put) { | |||
if rowcodec.IsNewFormat(m.Value) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When rowcodec.IsNewFormat(m.Value)
, the previous code are deleted?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here it sets reqCtx.buf = m.Value
and later lock.Value = reqCtx.buf
. The main purpose is to set lock.Value = m.Value
. In https://github.com/pingcap/tidb/pull/40422/files/f2fe4bde5508cad89d582145709fa286a24bec18#diff-c8058e505bb9be555aca4f1a51fbf20e712ea4d6d89812a09f3ebf02ff99e2f2R896, the lock.Value
has been initialized with m.Value
, so there is no need to assign again.
/retest |
f2fe4bd
to
6bd6ef6
Compare
/hold I forgot to add "expireTime" column for ttl task! |
6bd6ef6
to
da85ab8
Compare
/unhold |
/retest |
d5ff369
to
93d9c89
Compare
Signed-off-by: YangKeao <yangkeao@chunibyo.icu>
93d9c89
to
b1b5e2c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
/merge |
This pull request has been accepted and is ready to merge. Commit hash: b1b5e2c
|
Signed-off-by: YangKeao yangkeao@chunibyo.icu
What problem does this PR solve?
Issue Number: close #40362, #40363
Problem Summary:
This PR creates a system table to track the TTL scan tasks, and add some utilities to insert / delete the scan tasks.
This PR also submits and removes the related ttl scan tasks when the job is finished.
Check List
Tests