Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LUCENE-8585: Index-time jump-tables for DocValues #525

Closed
wants to merge 24 commits into from
Closed

LUCENE-8585: Index-time jump-tables for DocValues #525

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
1a6891f
LUCENE-8585: Index-time jump-tables for DocValues
tokee Dec 12, 2018
951d302
LUCENE-8585: Documentation fix (escaping of < and >) + links to moved…
tokee Dec 12, 2018
a7fca36
LUCENE-8585: Multiple minor changes after review from Adrien Grand (l…
tokee Dec 17, 2018
5f425d3
LUCENE-8585: Moved jumpTableEntryCount to meta (saves a seek on Index…
tokee Dec 17, 2018
fc217ab
LUCENE-8585: Loads DENSE rank up-front on block open and thereby spar…
tokee Dec 18, 2018
6f5eb29
LUCENE-8585: Fixed DENSE rank problem (caused by wrong bit shift). DV…
tokee Dec 19, 2018
5d199ae
LUCENE-8585: Fixed Lucene80NormsProducer problem where the slice for …
tokee Dec 20, 2018
bf1d6b3
LUCENE-8585: Expanded the IndexedDISI data format to support custom r…
tokee Dec 27, 2018
3988785
LUCENE-8585: Made merging Lucene80NormsProducer re-use the jumpTable …
tokee Dec 27, 2018
f0aa4c5
LUCENE-8585: Multiple smaller changes as per pull request review
tokee Dec 28, 2018
b988e25
LUCENE-8585: Forgot to remove debug line
tokee Dec 28, 2018
b7dec8b
LUCENE-8585: Adds NO_MORE_DOCS-block jump-table entry to IndexedDISI …
tokee Jan 3, 2019
4524add
LUCENE-8585: Made the block slice for IndexedDISI cover only blocks a…
tokee Jan 7, 2019
b89809e
LUCENE-8585: Fixed IndexedDISI index when advancing beyond maxDocs
tokee Jan 8, 2019
75dea0f
LUCENE-8585: Fixed precommit warnings (unclosed streams, unused import)
tokee Jan 8, 2019
11f761f
LUCENE-8585: Made IndexedDISI thron an exception on illegal denseRank…
tokee Jan 14, 2019
d613455
LUCENE-8585: Bugfix: Disabling DENSE rank cache (not used in producti…
tokee Jan 14, 2019
9fa6cc1
LUCENE-8585: Moved explicit skip-tests from BasedocValuesFormatTestCa…
tokee Jan 17, 2019
8147840
LUCENE-8585: Bugfix: The old offset-compensation for the vBPV jump-ta…
tokee Jan 17, 2019
e239a93
LUCENE-8585: Bugfix of simple oversight from introducing jumpTables R…
tokee Jan 17, 2019
d0ee060
Changed the DV-codec from Lucene70 to Lucene80 to match the default c…
tokee Jan 17, 2019
21e68f1
LUCENE-8585: Reduced initial size after feedback from Adrien Grand
tokee Jan 18, 2019
eddb765
Merged from master
tokee Jan 18, 2019
5c7e546
Merged from master
tokee Jan 18, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions lucene/CHANGES.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,9 @@ Optimizations
* LUCENE-8607: MatchAllDocsQuery can shortcut when total hit count is not
required (Alan Woodward, Adrien Grand)

* LUCENE-8585: Index-time jump-tables for DocValues, for O(1) advance when retrieving doc values.
(Toke Eskildsen, Adrien Grand)

======================= Lucene 7.7.0 =======================

Changes in Runtime Behavior
Expand Down
0 ...e/lucene/codecs/lucene70/IndexedDISI.java → ...e/lucene/codecs/lucene70/IndexedDISI.java
View file
Open in desktop
File renamed without changes.
0 ...s/lucene70/Lucene70DocValuesConsumer.java → ...s/lucene70/Lucene70DocValuesConsumer.java
View file
Open in desktop
File renamed without changes.
0 ...ecs/lucene70/Lucene70DocValuesFormat.java → ...ecs/lucene70/Lucene70DocValuesFormat.java
View file
Open in desktop
File renamed without changes.
0 ...s/lucene70/Lucene70DocValuesProducer.java → ...s/lucene70/Lucene70DocValuesProducer.java
View file
Open in desktop
File renamed without changes.
0 ...odecs/lucene70/Lucene70NormsConsumer.java → ...odecs/lucene70/Lucene70NormsConsumer.java
View file
Open in desktop
File renamed without changes.
0 .../codecs/lucene70/Lucene70NormsFormat.java → .../codecs/lucene70/Lucene70NormsFormat.java
View file
Open in desktop
File renamed without changes.
0 ...odecs/lucene70/Lucene70NormsProducer.java → ...odecs/lucene70/Lucene70NormsProducer.java
View file
Open in desktop
File renamed without changes.
1 change: 1 addition & 0 deletions .../backward-codecs/src/resources/META-INF/services/org.apache.lucene.codecs.DocValuesFormat
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@
# See the License for the specific language governing permissions and
# limitations under the License.

org.apache.lucene.codecs.lucene70.Lucene70DocValuesFormat
0 ...cene/codecs/lucene70/TestIndexedDISI.java → ...cene/codecs/lucene70/TestIndexedDISI.java
View file
Open in desktop
File renamed without changes.
0 ...lucene70/TestLucene70DocValuesFormat.java → ...lucene70/TestLucene70DocValuesFormat.java
View file
Open in desktop
File renamed without changes.
0 ...ecs/lucene70/TestLucene70NormsFormat.java → ...ecs/lucene70/TestLucene70NormsFormat.java
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion lucene/core/src/java/org/apache/lucene/codecs/lucene50/package-info.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

/**
* Components from the Lucene 5.0 index format
* See {@link org.apache.lucene.codecs.lucene50} for an overview
* See {@link org.apache.lucene.codecs.lucene80} for an overview
* of the index format.
*/
package org.apache.lucene.codecs.lucene50;
2 changes: 1 addition & 1 deletion lucene/core/src/java/org/apache/lucene/codecs/lucene60/package-info.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
*/

/**
* Components from the Lucene 6.0 index format. See {@link org.apache.lucene.codecs.lucene70}
* Components from the Lucene 6.0 index format. See {@link org.apache.lucene.codecs.lucene80}
* for an overview of the current index format.
*/
package org.apache.lucene.codecs.lucene60;
387 changes: 2 additions & 385 deletions lucene/core/src/java/org/apache/lucene/codecs/lucene70/package-info.java
View file
Open in desktop

Large diffs are not rendered by default.

632 changes: 632 additions & 0 deletions lucene/core/src/java/org/apache/lucene/codecs/lucene80/IndexedDISI.java
View file
Open in desktop

Large diffs are not rendered by default.

7 changes: 3 additions & 4 deletions lucene/core/src/java/org/apache/lucene/codecs/lucene80/Lucene80Codec.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@
import org.apache.lucene.codecs.lucene50.Lucene50TermVectorsFormat;
import org.apache.lucene.codecs.lucene60.Lucene60FieldInfosFormat;
import org.apache.lucene.codecs.lucene60.Lucene60PointsFormat;
import org.apache.lucene.codecs.lucene70.Lucene70NormsFormat;
import org.apache.lucene.codecs.lucene70.Lucene70SegmentInfoFormat;
import org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat;
import org.apache.lucene.codecs.perfield.PerFieldPostingsFormat;
Expand Down Expand Up @@ -150,7 +149,7 @@ public PostingsFormat getPostingsFormatForField(String field) {
/** Returns the docvalues format that should be used for writing
* new segments of <code>field</code>.
*
* The default implementation always returns "Lucene70".
* The default implementation always returns "Lucene80".
* <p>
* <b>WARNING:</b> if you subclass, you are responsible for index
* backwards compatibility: future version of Lucene are only
Expand All @@ -166,9 +165,9 @@ public final DocValuesFormat docValuesFormat() {
}

private final PostingsFormat defaultFormat = PostingsFormat.forName("Lucene50");
private final DocValuesFormat defaultDVFormat = DocValuesFormat.forName("Lucene70");
private final DocValuesFormat defaultDVFormat = DocValuesFormat.forName("Lucene80");

private final NormsFormat normsFormat = new Lucene70NormsFormat();
private final NormsFormat normsFormat = new Lucene80NormsFormat();

@Override
public final NormsFormat normsFormat() {
Expand Down
Loading