Skip to content

Conversation

@wuyunfeng
Copy link
Member

@wuyunfeng wuyunfeng commented Jul 9, 2020

#3306
When reading more column values such as above 20 (default, can be configured), switch from column(doc_value)reading to row(_source or stored_fields) reading, And we set enable_docvalue_scan to true for default.

It is possible to notice that retrieving an high number of fields leads to a sensible worsening of performance if DocValues are used(referenced from Solr performance-smackdown)
image

Doris On ES would provide 20(default value) for max doc_value fields, if exceeding this value, would use _source(_stored_fields)

BTW: after ES 6.x, ES introduce a dynamic index level setting index.max_docvalue_fields_search which default value is 100

@wuyunfeng wuyunfeng changed the title [Doris On ES] Add docvalue limitation for doc_values scan [Doris On ES] Add docvalue limitation for doc_values scan and enable doc_values scan default Jul 9, 2020
Copy link
Contributor

@BabySid BabySid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add come comment

BabySid
BabySid previously approved these changes Jul 9, 2020
Copy link
Contributor

@BabySid BabySid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Copy link
Contributor

@BabySid BabySid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman added approved Indicates a PR has been approved by one committer. area/doris-on-es Issues or PRs related to Doris on ElasticSearch labels Jul 9, 2020
Copy link
Contributor

@morningman morningman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@morningman morningman merged commit 265c26f into apache:master Jul 10, 2020
@wuyunfeng wuyunfeng deleted the docvalue-opt branch July 10, 2020 11:06
morningman pushed a commit to morningman/doris that referenced this pull request Jul 16, 2020
@EmmyMiao87 EmmyMiao87 mentioned this pull request Aug 18, 2020
Hastyshell pushed a commit to Hastyshell/doris that referenced this pull request Jul 21, 2025
apache#4055)

…ndant tablets (apache#50200)

secondary BE in cloud mode, temporarily stores the be tablet mapping
relationship of the be abnormal rehash. If it is not cleaned up, report
the redundant tablet cleaning diff logic, which does not work as
expected.
csun5285 pushed a commit to csun5285/doris that referenced this pull request Sep 23, 2025
apache#4123)

…… (apache#4055)

…ndant tablets (apache#50200)

secondary BE in cloud mode, temporarily stores the be tablet mapping
relationship of the be abnormal rehash. If it is not cleaned up, report
the redundant tablet cleaning diff logic, which does not work as
expected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/doris-on-es Issues or PRs related to Doris on ElasticSearch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants