You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Merge gives inaccurate results when there are deleted documents
Related component
Search:Performance
To Reproduce
Perform refresh and merge operations with deleted documents.
Expected behavior
Refresh / Lucene flush should work with star tree index irrespective of deleted documents. In order to do this, fix is idenitfied to include SegmentSuffix as part of custom write state and read state used during star tree file formats
For merge , there are few cases where SegmentDocValuesProducer is present as DocValuesProducer in a segment instead of the CodecDocValuesProducer.
So one change is to pick DocValuesProducer in the DVP list where star tree index is present - each field is associated with only one of the DocValuesProducer - even SegmentDocValuesProducer has a map of FieldInfo to DocValuesProducer which confirms the same. So StarTreeValues composite field too will be present in only one of the DocValuesProducers
Also, star tree index will not account for deleted documents except for cases where complete segments are dropped during segment merge. [ In which case star tree associated with that segment is also dropped ]
During merge, out of all the segments which does have star tree , we will merge based on the values already present in star tree index - hence deletes will not be accounted.
This issue fix handles the bug around enforcing the above behavior.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
OS: [e.g. iOS]
Version [e.g. 22]
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
Describe the bug
Related component
Search:Performance
To Reproduce
Perform
refresh
andmerge
operations with deleted documents.Expected behavior
SegmentSuffix
as part of custom write state and read state used during star tree file formatsSegmentDocValuesProducer
is present asDocValuesProducer
in a segment instead of theCodecDocValuesProducer
.So one change is to pick DocValuesProducer in the DVP list where star tree index is present - each field is associated with only one of the DocValuesProducer - even SegmentDocValuesProducer has a map of FieldInfo to DocValuesProducer which confirms the same. So
StarTreeValues
composite field too will be present in only one of theDocValuesProducers
Also, star tree index will not account for deleted documents except for cases where complete segments are dropped during segment merge. [ In which case star tree associated with that segment is also dropped ]
During merge, out of all the segments which does have star tree , we will merge based on the values already present in star tree index - hence deletes will not be accounted.
This issue fix handles the bug around enforcing the above behavior.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: