-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[Doris On ES] Add docvalue limitation for doc_values scan and enable doc_values scan default #4055
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
BabySid
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add come comment
BabySid
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
BabySid
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
morningman
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
…doc_values scan default (apache#4055)
apache#4055) …ndant tablets (apache#50200) secondary BE in cloud mode, temporarily stores the be tablet mapping relationship of the be abnormal rehash. If it is not cleaned up, report the redundant tablet cleaning diff logic, which does not work as expected.
apache#4123) …… (apache#4055) …ndant tablets (apache#50200) secondary BE in cloud mode, temporarily stores the be tablet mapping relationship of the be abnormal rehash. If it is not cleaned up, report the redundant tablet cleaning diff logic, which does not work as expected.
#3306
When reading more column values such as above 20 (default, can be configured), switch from column(
doc_value)reading to row(_sourceorstored_fields) reading, And we setenable_docvalue_scanto true for default.It is possible to notice that retrieving an high number of fields leads to a sensible worsening of performance if DocValues are used(referenced from Solr performance-smackdown)

Doris On ES would provide 20(default value) for max doc_value fields, if exceeding this value, would use
_source(_stored_fields)BTW: after ES 6.x, ES introduce a dynamic index level setting
index.max_docvalue_fields_searchwhich default value is100