Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add end_key_exclude field to SstMeta and distinguish restoring of RawKv #581

Merged
merged 5 commits into from
May 9, 2020

Conversation

MyonKeminta
Copy link
Contributor

Copy link
Contributor

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backup.pb.go should not need to be changed.

Rest LGTM.

@MyonKeminta
Copy link
Contributor Author

@kennytm It's generated. Should I remove those changes?

@kennytm
Copy link
Contributor

kennytm commented Mar 23, 2020

Yeah please revert it. gogoprotoc creates a random name every time it is called, and thus all *.pb.go will be changed even if the *.proto is unchanged.

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Copy link
Contributor

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kennytm kennytm merged commit 6a4d5c2 into pingcap:master May 9, 2020
@MyonKeminta MyonKeminta deleted the m/sst-meta-exclude-end-key branch May 9, 2020 06:53
MyonKeminta added a commit to MyonKeminta/kvproto that referenced this pull request May 18, 2020
…Kv (pingcap#581)

* Add end_key_exclusive field to SSTMeta

* Add is_raw_kv field to download request

* Revert changes of unrelated go files

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* build

Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
MyonKeminta added a commit to MyonKeminta/kvproto that referenced this pull request May 18, 2020
…Kv (pingcap#581)

* Add end_key_exclusive field to SSTMeta

* Add is_raw_kv field to download request

* Revert changes of unrelated go files

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* build

Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
5kbpers pushed a commit to 5kbpers/kvproto that referenced this pull request Aug 17, 2020
…Kv (pingcap#581)

* Add end_key_exclusive field to SSTMeta

* Add is_raw_kv field to download request

* Revert changes of unrelated go files

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* build

Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
overvenus pushed a commit that referenced this pull request Aug 18, 2020
* cdc: add regions field for reducing message of resolved ts

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* pdpb: add replicate_mode (#584)

Signed-off-by: disksing <i@disksing.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* replicate_mode: change recover-id to state-id (#594)

Signed-off-by: disksing <i@disksing.com>

* replicate: add unknown region state (#595)

Signed-off-by: disksing <i@disksing.com>

* *: update name and script (#597)

Signed-off-by: Jay Lee <BusyJayLee@gmail.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* support batch coprocessor (#586)

Add batch coprocessor function for TiFlash Engine.

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* pd:  add UpdateServiceGCSafePoint method (#603)

* pd: add UpdateServiceGCSafePoint method

Signed-off-by: Shafreeck Sea <shafreeck@gmail.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Add end_key_exclude field to SstMeta and distinguish restoring of RawKv (#581)

* Add end_key_exclusive field to SSTMeta

* Add is_raw_kv field to download request

* Revert changes of unrelated go files

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* build

Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>

* Revert "cdc: add regions field for reducing message of resolved ts"

This reverts commit c382b16.

Signed-off-by: 5kbpers <tangminghua@pingcap.com>

Co-authored-by: disksing <i@disksing.com>
Co-authored-by: Jay <BusyJay@users.noreply.github.com>
Co-authored-by: Han Fei <hanfei19910905@gmail.com>
Co-authored-by: Shafreeck Sea <shafreeck@users.noreply.github.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
ti-chi-bot pushed a commit to tikv/tikv that referenced this pull request Jan 28, 2021
…ive (#7196) (#9583)


cherry-pick #7196 to release-4.0

---

<!--
Thank you for contributing to TiKV!

If you haven't already, please read TiKV's [CONTRIBUTING](https://github.com/tikv/tikv/blob/master/CONTRIBUTING.md) document.

If you're unsure about anything, just ask; somebody should be along to answer within a day or two.
-->

###  What have you changed?

Partially fixes #7163 , but it still doesn't work for restoring plain transactional kv (without TiDB).

This PR adds is_raw_kv field to DownloadRequest to distinguish rawkv restore requests, and adds end_key_exclusive field to SstMeta so that TiKV knows whether the endkey need to be included.l

###  What is the type of the changes?

- Bugfix (a change which fixes an issue)

###  How is the PR tested?

- Unit test
- Manual test (add detailed scripts or steps below)

###  Does this PR affect documentation (docs) or should it be mentioned in the release notes?

(I'm not sure if it's necessary)
* Fix the issue that when restoring partial of backed up raw kv data, TiKV may write out-of-range keys.

###  Does this PR affect `tidb-ansible`?

###  Refer to a related PR or issue link (optional)

- [x] kvproto pingcap/kvproto#581
- [ ] Corresponding PR of BR

### Release note

* No release note
gengliqi pushed a commit to gengliqi/tikv that referenced this pull request Feb 20, 2021
…ive (tikv#7196) (tikv#9583)


cherry-pick tikv#7196 to release-4.0

---

<!--
Thank you for contributing to TiKV!

If you haven't already, please read TiKV's [CONTRIBUTING](https://github.com/tikv/tikv/blob/master/CONTRIBUTING.md) document.

If you're unsure about anything, just ask; somebody should be along to answer within a day or two.
-->

###  What have you changed?

Partially fixes tikv#7163 , but it still doesn't work for restoring plain transactional kv (without TiDB).

This PR adds is_raw_kv field to DownloadRequest to distinguish rawkv restore requests, and adds end_key_exclusive field to SstMeta so that TiKV knows whether the endkey need to be included.l

###  What is the type of the changes?

- Bugfix (a change which fixes an issue)

###  How is the PR tested?

- Unit test
- Manual test (add detailed scripts or steps below)

###  Does this PR affect documentation (docs) or should it be mentioned in the release notes?

(I'm not sure if it's necessary)
* Fix the issue that when restoring partial of backed up raw kv data, TiKV may write out-of-range keys.

###  Does this PR affect `tidb-ansible`?

###  Refer to a related PR or issue link (optional)

- [x] kvproto pingcap/kvproto#581
- [ ] Corresponding PR of BR

### Release note

* No release note
daimashusheng pushed a commit to daimashusheng/kvproto that referenced this pull request Sep 2, 2021
…Kv (pingcap#581)

* Add end_key_exclusive field to SSTMeta

* Add is_raw_kv field to download request

* Revert changes of unrelated go files

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* build

Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants