Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segment v2 use string real length #1943

Closed
wangbo opened this issue Oct 9, 2019 · 0 comments
Closed

Segment v2 use string real length #1943

wangbo opened this issue Oct 9, 2019 · 0 comments

Comments

@wangbo
Copy link
Contributor

wangbo commented Oct 9, 2019

Current segment v2 use fixed length to allocate string field,using real length of string field is better

major process

  1. init a Field by type in SegmentWriter.init,then ColumnWriter and ColumnZoneMapBuilder holds it instead of holding a typeinfo,this is because field contains whole info of a specific type Field,while typeInfo only contains method
  2. using polymorphism to realize string type allocate value with length while other type allocate value with default method.
    Char/VarcharField override allocate_value_from_arena,and Char/VarcharField contains string real length

Other question

  1. bug of set_to_max;
    current set_to_max of varchar,hard code setting size of varchar length to 1,but it should use user define varchar length.
    we can use field.length(come from tabletColumn.length) to set_to_max
  2. problem of using field.length in set_to_max
    segment v1 think field.length=sizeof(StringLengthType) + slice.size
    segment v2 think field.length=slice.size
    so confict happends
    I defined a new method signature (set_to_max_v1)to let segment v1 code using it.
    the logic of set_to_max_v1 not changed,it is still hard code.
    segment v2 using set_to_max method, and it set string using string'real length
  3. current code refers field.set_to_max
    wrapper_field.cpp, change to set_to_max_v1
    stream_index_common.cpp,no change,string and hll skipped
    row_cursor.cpp, change to set_to_max_v1
wangbo added a commit to wangbo/incubator-doris that referenced this issue Oct 9, 2019
wangbo added a commit to wangbo/incubator-doris that referenced this issue Oct 9, 2019
wangbo added a commit to wangbo/incubator-doris that referenced this issue Oct 9, 2019
wuyunfeng pushed a commit to wuyunfeng/incubator-doris that referenced this issue Oct 22, 2019
@wangbo wangbo closed this as completed Oct 22, 2019
SWJTU-ZhangLei pushed a commit to SWJTU-ZhangLei/incubator-doris that referenced this issue Jul 25, 2023
If putting server encryption_key fails, the process will exit, including the scenario of transaction conflict. 

In the k8s, multiple processes may start at the same time, so potential transaction conflicts will cause some processes to exit. This PR avoids the above problem by simply retrying.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant