Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fine grained partition writer optimization #6173

Merged
merged 29 commits into from
Oct 31, 2022
Merged

Fine grained partition writer optimization #6173

merged 29 commits into from
Oct 31, 2022

Commits on Oct 19, 2022

  1. Add reuse memory scatterTo api for IColumn 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 19, 2022
    Configuration menu
    Copy the full SHA
    9e4d3cf View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2022

  1. Implement set,nothing,const,tuple columns scatterTo, and add gtest 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    0ff883a View commit details
    Browse the repository at this point in the history

  2. Partition Opt 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    b06df19 View commit details
    Browse the repository at this point in the history

  3. Apply FineGrainedShuffleWriter optimization 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    9588d49 View commit details
    Browse the repository at this point in the history

  4. Fix gtest compilation failures 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    12b7c41 View commit details
    Browse the repository at this point in the history

  5. Little refact 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    e62857e View commit details
    Browse the repository at this point in the history

  6. Little refact 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    a83c7af View commit details
    Browse the repository at this point in the history

  7. Little refact 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 20, 2022
    Configuration menu
    Copy the full SHA
    545bbb0 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2022

  1. Extract flush method from write for DAGResponseWriter interface 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 21, 2022
    Configuration menu
    Copy the full SHA
    f833565 View commit details
    Browse the repository at this point in the history

  2. Fix after refactor 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 21, 2022
    Configuration menu
    Copy the full SHA
    0f87acd View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2022

  1. Update to comments and fix format issue 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 28, 2022
    Configuration menu
    Copy the full SHA
    af4b9b5 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into fine_grained_partition_opt

    @yibin87
    yibin87 authored Oct 28, 2022
    Configuration menu
    Copy the full SHA
    2b682af View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into fine_grained_partition_opt

    @yibin87
    yibin87 authored Oct 28, 2022
    Configuration menu
    Copy the full SHA
    a593f6c View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into fine_grained_partition_opt

    @yibin87
    yibin87 authored Oct 28, 2022
    Configuration menu
    Copy the full SHA
    a2cb6f7 View commit details
    Browse the repository at this point in the history

  5. Fix format issue 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 28, 2022
    Configuration menu
    Copy the full SHA
    df4f0b7 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'fine_grained_partition_opt' of github.com:yibin87/tifla… 

    …sh into fine_grained_partition_opt
    @yibin87
    yibin87 committed Oct 28, 2022
    Configuration menu
    Copy the full SHA
    6d538d2 View commit details
    Browse the repository at this point in the history

  7. Fix clang tidy issues 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 28, 2022
    Configuration menu
    Copy the full SHA
    e812ecf View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2022

  1. Change to review comments 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    456d1a5 View commit details
    Browse the repository at this point in the history

  2. Add prepared flag to ensure FineGrainedShuffleWriter is prepared befo… 

    …re writing

Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    517c3b9 View commit details
    Browse the repository at this point in the history

  3. Delete useless file 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    8f6072b View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into fine_grained_partition_opt

    @yibin87
    yibin87 authored Oct 31, 2022
    Configuration menu
    Copy the full SHA
    a5f8c66 View commit details
    Browse the repository at this point in the history

  5. Convert constant columns for sample block 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    8b6d17e View commit details
    Browse the repository at this point in the history

  6. Merge branch 'fine_grained_partition_opt' of github.com:yibin87/tifla… 

    …sh into fine_grained_partition_opt
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    e5c6f63 View commit details
    Browse the repository at this point in the history

  7. Update dbms/src/Flash/Mpp/FineGrainedShuffleWriter.cpp 

    Co-authored-by: SeaRise <hhssearise@foxmail.com>
    @yibin87 @SeaRise
    yibin87 and SeaRise authored Oct 31, 2022
    Configuration menu
    Copy the full SHA
    9dbec38 View commit details
    Browse the repository at this point in the history

  8. Update dbms/src/Columns/tests/gtest_column_scatterTo.cpp 

    Co-authored-by: SeaRise <hhssearise@foxmail.com>
    @yibin87 @SeaRise
    yibin87 and SeaRise authored Oct 31, 2022
    Configuration menu
    Copy the full SHA
    7d636ad View commit details
    Browse the repository at this point in the history

  9. Fix compilation issue 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    8f54189 View commit details
    Browse the repository at this point in the history

  10. Fix compilation issue 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    35c1357 View commit details
    Browse the repository at this point in the history

  11. Handle potential null column issue 

    Signed-off-by: yibin <huyibin@pingcap.com>
    @yibin87
    yibin87 committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    6ef1faf View commit details
    Browse the repository at this point in the history

  12. Merge branch 'master' into fine_grained_partition_opt

    @ti-chi-bot
    ti-chi-bot authored Oct 31, 2022
    Configuration menu
    Copy the full SHA
    b774dd0 View commit details
    Browse the repository at this point in the history