Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

Detect duplicate data from TiKV #1144

Merged
merged 26 commits into from
Aug 3, 2021
Merged

Detect duplicate data from TiKV #1144

merged 26 commits into from
Aug 3, 2021
+1,245 −131

Commits on Jul 14, 2021

  1. Squashed commit of the following: 

    commit 1ed2b5b
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Wed Jul 14 12:11:19 2021 +0800

    fix kvproto

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit 3187304
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Tue Jun 8 16:27:42 2021 +0800

    update getValues to public

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit b8a36b4
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Tue Jun 8 16:12:25 2021 +0800

    refactor duplicate index

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit bab4072
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Mon May 31 17:58:40 2021 +0800

    fix fmt

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit 2dfebb0
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Mon May 31 17:30:40 2021 +0800

    use io.EOF to judge end

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit b9b6a31
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Thu May 27 17:08:25 2021 +0800

    add some node

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit 34b2150
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Tue May 25 20:37:12 2021 +0800

    duplicate data before checksum

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit 4341adc
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Tue May 25 17:44:38 2021 +0800

    support decode kv

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit 18edf13
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Mon May 24 23:58:33 2021 +0800

    retry region error

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

commit bdd64be
Author: Little-Wallace <bupt2013211450@gmail.com>
Date:   Mon May 24 20:02:40 2021 +0800

    add duplicate manager

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    9657cef View commit details
    Browse the repository at this point in the history

  2. fix conflict 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    7d57e17 View commit details
    Browse the repository at this point in the history

  3. fic fmt 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    1f03ab1 View commit details
    Browse the repository at this point in the history

  4. fix test compile 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    beab74a View commit details
    Browse the repository at this point in the history

  5. move duplicate to local 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    abbf24b View commit details
    Browse the repository at this point in the history

  6. add method 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    42f5fac View commit details
    Browse the repository at this point in the history

  7. fix get values bug 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    14ac67e View commit details
    Browse the repository at this point in the history

  8. fix read remote tikv 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 14, 2021
    Configuration menu
    Copy the full SHA
    abe67b9 View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2021

  1. Report duplicate keys 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 15, 2021
    Configuration menu
    Copy the full SHA
    b8b9dbe View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2021

  1. add test for cross engine duplicate detection

    @sleepymole
    sleepymole committed Jul 19, 2021
    Configuration menu
    Copy the full SHA
    9ccde16 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2021

  1. Merge pull request #1 from gozssky/dup-test 

    tests: add test for cross engine duplicate detection
    @Little-Wallace
    Little-Wallace authored Jul 20, 2021
    Configuration menu
    Copy the full SHA
    62a271f View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2021

  1. fix range bug 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 25, 2021
    Configuration menu
    Copy the full SHA
    dd6c16a View commit details
    Browse the repository at this point in the history

  2. fix fmt 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 25, 2021
    Configuration menu
    Copy the full SHA
    0af24e9 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2021

  1. Merge branch 'master' into duplicate

    @Little-Wallace
    Little-Wallace authored Jul 27, 2021
    Configuration menu
    Copy the full SHA
    3318945 View commit details
    Browse the repository at this point in the history

  2. fix conflict 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 27, 2021
    Configuration menu
    Copy the full SHA
    b600a75 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2021

  1. fix comment 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 29, 2021
    Configuration menu
    Copy the full SHA
    0dce624 View commit details
    Browse the repository at this point in the history

  2. fix buf 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 29, 2021
    Configuration menu
    Copy the full SHA
    2e13497 View commit details
    Browse the repository at this point in the history

Commits on Jul 30, 2021

  1. address comment 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Jul 30, 2021
    Configuration menu
    Copy the full SHA
    d9e2de4 View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2021

  1. Merge branch 'master' into duplicate

    @Little-Wallace
    Little-Wallace authored Aug 2, 2021
    Configuration menu
    Copy the full SHA
    30e0978 View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2021

  1. address comment 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    3339bd5 View commit details
    Browse the repository at this point in the history

  2. fix duplicaterequest field 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    7cb54b3 View commit details
    Browse the repository at this point in the history

  3. fix warn 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    6010c07 View commit details
    Browse the repository at this point in the history

  4. split local and remote keys 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    618b406 View commit details
    Browse the repository at this point in the history

  5. fix fmt 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    366204e View commit details
    Browse the repository at this point in the history

  6. fix fmt 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    ddb54ba View commit details
    Browse the repository at this point in the history

  7. fix fmt 

    Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>
    @Little-Wallace
    Little-Wallace committed Aug 3, 2021
    Configuration menu
    Copy the full SHA
    fb64eaa View commit details
    Browse the repository at this point in the history