Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pick exception handle logic to 2.1 #40547

Closed
wants to merge 7 commits into from

Conversation

yiguolei
Copy link
Contributor

@yiguolei yiguolei commented Sep 9, 2024

Proposed changes

pick
#39536
#39913
#39964
#40020
#38960
#40092

yiguolei and others added 7 commits September 9, 2024 13:46
…pache#39536)

It is needed to pick to branch 21, because we will depend on it to do
spill disk.

Co-authored-by: yiguolei <yiguolei@gmail.com>
…pache#39913)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
…age builder allocate memory failed (apache#39964)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
…ate memory failed (apache#40020)

1. pageio method should return error when allocate memory failed just
for point query.
2. variant parsing logic should return error when allocate memory
failed.

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
… exception safe (apache#38960)

At present, the memory allocation of `ByteBuffer` is done through `new
char[capacity_]`. Now, it is uniformly allocated by `Allocator` for the
following purposes:
1. Better memory statistics
2. Better support for memory limit check
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@yiguolei
Copy link
Contributor Author

yiguolei commented Sep 9, 2024

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

@@ -111,8 +111,8 @@ Status PageIO::write_page(io::FileWriter* writer, const std::vector<Slice>& body
return Status::OK();
}

Status PageIO::read_and_decompress_page(const PageReadOptions& opts, PageHandle* handle,
Slice* body, PageFooterPB* footer) {
Status PageIO::read_and_decompress_page_(const PageReadOptions& opts, PageHandle* handle,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'read_and_decompress_page_' exceeds recommended size/complexity thresholds [readability-function-size]

Status PageIO::read_and_decompress_page_(const PageReadOptions& opts, PageHandle* handle,
               ^
Additional context

be/src/olap/rowset/segment_v2/page_io.cpp:113: 115 lines including whitespace and comments (threshold 80)

Status PageIO::read_and_decompress_page_(const PageReadOptions& opts, PageHandle* handle,
               ^

}

~ByteBuffer() { delete[] ptr; }
~ByteBuffer() { Allocator<false>::free(ptr, capacity); }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: use '= default' to define a trivial destructor [modernize-use-equals-default]

    ~ByteBuffer() { Allocator<false>::free(ptr, capacity); }
    ^

fmt::memory_buffer& error_prefix,
const uint32_t row_count,
vectorized::IColumn::Permutation* rows) {
Status OlapTableBlockConvertor::_internal_validate_column(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function '_internal_validate_column' has cognitive complexity of 331 (threshold 50) [readability-function-cognitive-complexity]

Status OlapTableBlockConvertor::_internal_validate_column(
                                ^
Additional context

be/src/vec/sink/vtablet_block_convertor.cpp:189: +1

    DCHECK((rows == nullptr) || (rows->size() == row_count));
                             ^

be/src/vec/sink/vtablet_block_convertor.cpp:191: nesting level increased to 1

    auto set_invalid_and_append_error_msg = [&](int row) {
                                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:205: +1, including nesting penalty of 0, nesting level increased to 1

            column_ptr == nullptr ? column : (column_ptr->get_nested_column_ptr());
                                  ^

be/src/vec/sink/vtablet_block_convertor.cpp:206: +1, including nesting penalty of 0, nesting level increased to 1

    const auto* null_map = column_ptr == nullptr ? nullptr : column_ptr->get_null_map_data().data();
                                                 ^

be/src/vec/sink/vtablet_block_convertor.cpp:207: nesting level increased to 1

    const auto need_to_validate = [&null_map, this](size_t j, size_t row) {
                                  ^

be/src/vec/sink/vtablet_block_convertor.cpp:211: +1, including nesting penalty of 0, nesting level increased to 1

    switch (type.type) {
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:220: +2, including nesting penalty of 1, nesting level increased to 2

        if (type.len > 0) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:226: +2, including nesting penalty of 1, nesting level increased to 2

        for (int j = 0; j < row_count; ++j) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:230: nesting level increased to 2

        auto check_column_type = [&]() {
                                 ^

be/src/vec/sink/vtablet_block_convertor.cpp:232: +3, including nesting penalty of 2, nesting level increased to 3

            if (nullptr == dynamic_cast<const vectorized::ColumnString*>(&real_column)) {
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:240: +2, including nesting penalty of 1, nesting level increased to 2

        if (invalid_count) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:241: +3, including nesting penalty of 2, nesting level increased to 3

            for (size_t j = 0; j < row_count; ++j) {
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:242: +4, including nesting penalty of 3, nesting level increased to 4

                auto row = rows ? (*rows)[j] : j;
                                ^

be/src/vec/sink/vtablet_block_convertor.cpp:243: +4, including nesting penalty of 3, nesting level increased to 4

                if (need_to_validate(j, row)) {
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:246: +5, including nesting penalty of 4, nesting level increased to 5

                    if (invalid) {
                    ^

be/src/vec/sink/vtablet_block_convertor.cpp:248: +6, including nesting penalty of 5, nesting level increased to 6

                        if (!st.ok()) {
                        ^

be/src/vec/sink/vtablet_block_convertor.cpp:252: +6, including nesting penalty of 5, nesting level increased to 6

                        if (str_val.size > type.len) {
                        ^

be/src/vec/sink/vtablet_block_convertor.cpp:259: +1, nesting level increased to 6

                        } else if (str_val.size > limit) {
                               ^

be/src/vec/sink/vtablet_block_convertor.cpp:269: +6, including nesting penalty of 5, nesting level increased to 6

                        RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));
                        ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:269: +7, including nesting penalty of 6, nesting level increased to 7

                        RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));
                        ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:279: +2, including nesting penalty of 1, nesting level increased to 2

        for (size_t j = 0; j < row_count; ++j) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:280: +3, including nesting penalty of 2, nesting level increased to 3

            if (!_filter_map[j]) {
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:281: +4, including nesting penalty of 3, nesting level increased to 4

                if (is_nullable && column_ptr && column_ptr->is_null_at(j)) {
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:281: +1

                if (is_nullable && column_ptr && column_ptr->is_null_at(j)) {
                                              ^

be/src/vec/sink/vtablet_block_convertor.cpp:286: +4, including nesting penalty of 3, nesting level increased to 4

                if (invalid) {
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:289: +5, including nesting penalty of 4, nesting level increased to 5

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(j));
                    ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:289: +6, including nesting penalty of 5, nesting level increased to 6

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(j));
                    ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:301: +2, including nesting penalty of 1, nesting level increased to 2

        for (size_t j = 0; j < row_count; ++j) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:302: +3, including nesting penalty of 2, nesting level increased to 3

            auto row = rows ? (*rows)[j] : j;
                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:303: +3, including nesting penalty of 2, nesting level increased to 3

            if (need_to_validate(j, row)) {
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:308: +4, including nesting penalty of 3, nesting level increased to 4

                if (dec_val.greater_than_scale(type.scale)) {
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:312: +5, including nesting penalty of 4, nesting level increased to 5

                    if (code != E_DEC_OK) {
                    ^

be/src/vec/sink/vtablet_block_convertor.cpp:318: +4, including nesting penalty of 3, nesting level increased to 4

                if (dec_val > max_decimalv2 || dec_val < min_decimalv2) {
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:318: +1

                if (dec_val > max_decimalv2 || dec_val < min_decimalv2) {
                                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:328: +4, including nesting penalty of 3, nesting level increased to 4

                if (invalid) {
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:329: +5, including nesting penalty of 4, nesting level increased to 5

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));
                    ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:329: +6, including nesting penalty of 5, nesting level increased to 6

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));
                    ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:343: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    for (int j = 0; j < row_count; ++j) {                                                         \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:345: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        invalid_count += dec_val > max_decimal || dec_val < min_decimal;                          \
                                               ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:347: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    if (invalid_count) {                                                                          \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +3, including nesting penalty of 2, nesting level increased to 3

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:348: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        for (size_t j = 0; j < row_count; ++j) {                                                  \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:349: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            auto row = rows ? (*rows)[j] : j;                                                     \
                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:350: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            if (need_to_validate(j, row)) {                                                       \
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                                          ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:361: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (invalid) {                                                                    \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +6, including nesting penalty of 5, nesting level increased to 6

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:367: +7, including nesting penalty of 6, nesting level increased to 7

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal32);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:343: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    for (int j = 0; j < row_count; ++j) {                                                         \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:345: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        invalid_count += dec_val > max_decimal || dec_val < min_decimal;                          \
                                               ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:347: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    if (invalid_count) {                                                                          \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +3, including nesting penalty of 2, nesting level increased to 3

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:348: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        for (size_t j = 0; j < row_count; ++j) {                                                  \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:349: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            auto row = rows ? (*rows)[j] : j;                                                     \
                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:350: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            if (need_to_validate(j, row)) {                                                       \
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                                          ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:361: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (invalid) {                                                                    \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +6, including nesting penalty of 5, nesting level increased to 6

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:371: +7, including nesting penalty of 6, nesting level increased to 7

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal64);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:343: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    for (int j = 0; j < row_count; ++j) {                                                         \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:345: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        invalid_count += dec_val > max_decimal || dec_val < min_decimal;                          \
                                               ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:347: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    if (invalid_count) {                                                                          \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +3, including nesting penalty of 2, nesting level increased to 3

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:348: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        for (size_t j = 0; j < row_count; ++j) {                                                  \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:349: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            auto row = rows ? (*rows)[j] : j;                                                     \
                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:350: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            if (need_to_validate(j, row)) {                                                       \
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                                          ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:361: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (invalid) {                                                                    \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +6, including nesting penalty of 5, nesting level increased to 6

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:375: +7, including nesting penalty of 6, nesting level increased to 7

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal128V3);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:343: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    for (int j = 0; j < row_count; ++j) {                                                         \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:345: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        invalid_count += dec_val > max_decimal || dec_val < min_decimal;                          \
                                               ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +2, including nesting penalty of 1, nesting level increased to 2

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:347: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

    if (invalid_count) {                                                                          \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +3, including nesting penalty of 2, nesting level increased to 3

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:348: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

        for (size_t j = 0; j < row_count; ++j) {                                                  \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:349: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            auto row = rows ? (*rows)[j] : j;                                                     \
                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +4, including nesting penalty of 3, nesting level increased to 4

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:350: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

            if (need_to_validate(j, row)) {                                                       \
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +1

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:353: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (dec_val > max_decimal || dec_val < min_decimal) {                             \
                                          ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +5, including nesting penalty of 4, nesting level increased to 5

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:361: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                if (invalid) {                                                                    \
                ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +6, including nesting penalty of 5, nesting level increased to 6

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:379: +7, including nesting penalty of 6, nesting level increased to 7

        CHECK_VALIDATION_FOR_DECIMALV3(vectorized::Decimal256);
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:362: expanded from macro 'CHECK_VALIDATION_FOR_DECIMALV3'

                    RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));                       \
                    ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:390: +2, including nesting penalty of 1, nesting level increased to 2

        for (size_t r = 0; r < row_count; ++r) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:391: +3, including nesting penalty of 2, nesting level increased to 3

            for (size_t c = offsets[r - 1]; c < offsets[r]; ++c) {
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:396: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(_validate_column(state, nested_type, type.contains_nulls[0],
        ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:396: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(_validate_column(state, nested_type, type.contains_nulls[0],
        ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:408: +2, including nesting penalty of 1, nesting level increased to 2

        for (size_t r = 0; r < row_count; ++r) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:409: +3, including nesting penalty of 2, nesting level increased to 3

            for (size_t c = offsets[r - 1]; c < offsets[r]; ++c) {
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:414: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(_validate_column(state, key_type, type.contains_nulls[0],
        ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:414: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(_validate_column(state, key_type, type.contains_nulls[0],
        ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:417: +2, including nesting penalty of 1, nesting level increased to 2

        RETURN_IF_ERROR(_validate_column(state, val_type, type.contains_nulls[1],
        ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:417: +3, including nesting penalty of 2, nesting level increased to 3

        RETURN_IF_ERROR(_validate_column(state, val_type, type.contains_nulls[1],
        ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:427: +2, including nesting penalty of 1, nesting level increased to 2

        for (size_t sc = 0; sc < column_struct->tuple_size(); ++sc) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:428: +3, including nesting penalty of 2, nesting level increased to 3

            RETURN_IF_ERROR(_validate_column(state, type.children[sc], type.contains_nulls[sc],
            ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:428: +4, including nesting penalty of 3, nesting level increased to 4

            RETURN_IF_ERROR(_validate_column(state, type.children[sc], type.contains_nulls[sc],
            ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:443: +1, including nesting penalty of 0, nesting level increased to 1

    if ((!is_nullable || type == TYPE_OBJECT) && column_ptr) {
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:443: +1

    if ((!is_nullable || type == TYPE_OBJECT) && column_ptr) {
                                              ^

be/src/vec/sink/vtablet_block_convertor.cpp:443: +1

    if ((!is_nullable || type == TYPE_OBJECT) && column_ptr) {
                      ^

be/src/vec/sink/vtablet_block_convertor.cpp:444: +2, including nesting penalty of 1, nesting level increased to 2

        for (int j = 0; j < row_count; ++j) {
        ^

be/src/vec/sink/vtablet_block_convertor.cpp:445: +3, including nesting penalty of 2, nesting level increased to 3

            auto row = rows ? (*rows)[j] : j;
                            ^

be/src/vec/sink/vtablet_block_convertor.cpp:446: +3, including nesting penalty of 2, nesting level increased to 3

            if (null_map[j] && !_filter_map[row]) {
            ^

be/src/vec/sink/vtablet_block_convertor.cpp:449: +4, including nesting penalty of 3, nesting level increased to 4

                RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));
                ^

be/src/common/status.h:619: expanded from macro 'RETURN_IF_ERROR'

    do {                                \
    ^

be/src/vec/sink/vtablet_block_convertor.cpp:449: +5, including nesting penalty of 4, nesting level increased to 5

                RETURN_IF_ERROR(set_invalid_and_append_error_msg(row));
                ^

be/src/common/status.h:621: expanded from macro 'RETURN_IF_ERROR'

        if (UNLIKELY(!_status_.ok())) { \
        ^

fmt::memory_buffer& error_prefix,
const uint32_t row_count,
vectorized::IColumn::Permutation* rows) {
Status OlapTableBlockConvertor::_internal_validate_column(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function '_internal_validate_column' exceeds recommended size/complexity thresholds [readability-function-size]

Status OlapTableBlockConvertor::_internal_validate_column(
                                ^
Additional context

be/src/vec/sink/vtablet_block_convertor.cpp:184: 267 lines including whitespace and comments (threshold 80)

Status OlapTableBlockConvertor::_internal_validate_column(
                                ^

@yiguolei yiguolei closed this Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants