-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge master about 6.6(7ec73fd) #262
Merge master about 6.6(7ec73fd) #262
Conversation
ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Signed-off-by: Zwb <ethercflow@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 a few panic fix 1) update_approximate_raft_log_size may run into divid by zero error 2) appy_delete may have None write_batch 3) StoreMeta::set_region may run into region corruption error if it's destroyed and re-created. 4) TabletSnapManager's snapshot size calculation may throw Other error. Signed-off-by: qi.xu <tonxuqi@outlook.com> Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: qi.xu <tonxuqi@outlook.com> Co-authored-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
…13995) ref tikv#12842 Whenever timeout, the peer will check for unapplied logs whether there are pending conf change and trigger heavy reads. So we wait till most logs are applied before ticking. It also fix following issues: - PersistenceListener is not installed - implementation of persisted_apply_index is wrong - parse tablet name is wrong Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 Signed-off-by: bufferflies <1045931706@qq.com>
close tikv#13997 Support to use evict_entry_cache when restart node. Signed-off-by: tabokie <xy.tao@outlook.com> Signed-off-by: hongyunyan <649330952@qq.com> Signed-off-by: Xinye Tao <xy.tao@outlook.com> Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Signed-off-by: Zwb <ethercflow@gmail.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com> Co-authored-by: Jay <BusyJay@users.noreply.github.com> Co-authored-by: Zwb <ethercflow@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 The API is supposed to be used with `append` but nowhere can we find the clue. This PR merges `cut_logs` and `append` to reduce confusion and mistakes. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 Publish tablet in apply thread is unsafe. This PR moves the operation to raftstore. It also fixes the issues that applying two splits at a time can cause panic. It also makes sure cache will be cleared after tablet is published. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 These two are helpers to utilize the static KV pairs in logger. In the past, we use `logger.list()` to try to format the configured KV pairs, but it will not work as values are omitted. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#13730 Introduce priority-based channel Signed-off-by: Connor1996 <zbk602423539@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 When the tablet contains dirty data right after split, generating snapshot may just a waste. On the other hand, split usually happens on all peers, so delay it a bit actually makes all peers more likely to be initialized by split. So this PR rejects generating snapshot when it detects it still has dirty data. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 1. store heartbeat should add snapshot and kv engine used size Signed-off-by: bufferflies <1045931706@qq.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
close tikv#13977 Signed-off-by: tabokie <xy.tao@outlook.com>
ref tikv#12842 Make apply adaptive to reduce high tail latency. Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 - add water metrics - fix potential panic when destroying a peer - fix incorrect store size Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
* util: Fix incorrect memory capacity Signed-off-by: Wish <breezewish@outlook.com> * Fix lints Signed-off-by: Wish <breezewish@outlook.com> * Check capacity with /proc/meminfo Signed-off-by: Wish <breezewish@outlook.com> Signed-off-by: Wish <breezewish@outlook.com>
* hotfix kvproto for global config Signed-off-by: husharp <jinhao.hu@pingcap.com> * make format happy Signed-off-by: husharp <jinhao.hu@pingcap.com> Signed-off-by: husharp <jinhao.hu@pingcap.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#13730 Signed-off-by: glorv <glorvs@163.com>
…ock request (tikv#14037) close tikv#14038, close pingcap/tidb#40114 Fixes the problem that when handling repeated acquire_pessimistic_lock requests is recevied, should_not_exist is ignored. TiKV provides idempotency for these RPC requests, but for acquire_pessimistic_lock, it ignored the possibility that the client may expect a pessimistic_rollback between two acquire_pessimistic_lock request on the same key. In this case the second request may come from another statement and carries `should_not_exist` that wasn't set in the previously finished pessimistic lock request. If the first request successfully acquired the lock and the pessimistic_rollback failed, TiKV may return a sucessful response, making the client believe that the key doesn't exist before. In some rare cases, this has risk to cause data inconsistency. Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 This PR fixes several bugs and metrics: - Now waterfall timer will be reset in before_write, the goal is to solve the confusion that stall writes can pollute the whole waterfall metrics. - Perf context is changed not to be associated with engine instance. Perf context is thread local and instance independent under the hook. - Fix flushed index advance failure due to suspicious flush. - Support print long uncommitted logs and fix incorrect commit time Signed-off-by: Jay Lee <BusyJayLee@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#12842 Move transaction related code to txn_ext.rs. Fix the bug that snapshot doesn't set term and extra_op. Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
ref tikv#12842 1) add snapshot apply metrics 2) disable bloomfilter for raftkv-v2 for now until a proper ratio is found 3) disable rocksdb write stall for raftkv-v2 until the tablet flow control is fully verified. Signed-off-by: Qi Xu <tonyxuqqi@outlook.com> Co-authored-by: Qi Xu <tonyxuqqi@outlook.com>
ref tikv#12876 fix witness raft log gc panic and refactor Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
ref tikv#12999 copr: support handling keyspace request Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
ref tikv#13730 Support priority-based scheduling for the scheduler worker pool. Signed-off-by: Connor1996 <zbk602423539@gmail.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
ref tikv#11312 Fix `make docker`. Signed-off-by: tabokie <xy.tao@outlook.com>
close tikv#14063 make sure kvproto compatibility Signed-off-by: husharp <jinhao.hu@pingcap.com>
ref tikv#12842 Signed-off-by: tabokie <xy.tao@outlook.com>
close tikv#14071 Log backup would aware TLS certifications changing. Signed-off-by: hillium <yujuncen@pingcap.com>
close tikv#14069 Signed-off-by: YangKeao <yangkeao@chunibyo.icu> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
* support switch witness ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * add switch witness api for test_pd_client ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * pd heartbeat resp support switch witness ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * update region epoch ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * fix write apply state race ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * remove unnecessary code ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * add back test_witness_conf_change ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * add some tests ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * avoid test failures ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * a few refactor ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * add witness election priority and address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * clean code ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * fix tests failed caused by cfg ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * fix test failed caused by mistake modify ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * adjust priority after snapshot persisted ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * address comments ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * notify pd after switch witness as region changed ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * define a new backoff error for witness ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * fix panic caused by applygap ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * forbid transfer leader to non-witness waiting data ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * update kvproto ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * fix two panics ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * retry request snapshot ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * retry to request snaphost after term change ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> * update kvproto comment ref tikv#12876 Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Signed-off-by: Wenbo Zhang <ethercflow@gmail.com> Signed-off-by: Zwb <ethercflow@gmail.com> Co-authored-by: Xinye Tao <xy.tao@outlook.com>
close tikv#13983 - add etcd mock for pd - add service for resource group Signed-off-by: husharp <jinhao.hu@pingcap.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
ref tikv#13730 Support priority-based scheduling for the apply batch system. Signed-off-by: Connor1996 <zbk602423539@gmail.com>
|
close tikv#13586, close tikv#13752, ref tikv#14017 Signed-off-by: Andrei Dragus <andrei.dragus@databricks.com>
…#14087) close tikv#14086 PD_REQUEST_HISTOGRAM_VEC can be changed to use static metrics to improve performance. Signed-off-by: husharp <jinhao.hu@pingcap.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
ref tikv#13936 Signed-off-by: Zhi Qi <qizhi@pingcap.com>
…ikv#14088) close tikv#14045 - Store the flashback `start_ts` in region meta. - Allow the read request with a smaller ts during flashback. Signed-off-by: JmPotato <ghzpotato@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
…ata. (tikv#14046) ref tikv#12999 import: sst_importer support download SST and rewrite into keyspace data. Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
What problem does this PR solve?
Issue Number: close #xxx
Problem Summary:
This PR is to adopt Flashback fix:
Origin 453b9df.
What is changed and how it works?
Check List
Tests
Side effects
Documentation
Release note