Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree #13521

Open
wants to merge 63 commits into
base: main
Choose a base branch
from
+1,140 −299
Open

Introduce new encoding of BPV 21 for DocIdsWriter used in BKD Tree #13521

Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
11e41b4
BPV 21 Encoding in DocIdsWriter and it's micro benchmark
expani Jun 21, 2024
2376766
Deleted single loop encoders
expani Jun 21, 2024
886810b
Using 2 step bpv 21 encoder in docIdsWriter
expani Jun 26, 2024
9e20958
Minor refactor
expani Jun 26, 2024
5034e04
Initial commit - DocId Benchmark
expani Aug 6, 2024
e06bc3b
Explicit use of NIOFSDirectory
expani Aug 6, 2024
d2122cd
Increasing iterations to 10
expani Aug 6, 2024
d09e79a
Added a new encoder
expani Aug 6, 2024
1ddd597
Removed older benchmarks
expani Aug 6, 2024
9f3792b
Added License
expani Aug 6, 2024
4e8a1f2
Refactoring
expani Aug 6, 2024
04361d3
Gradle Tidy
expani Aug 6, 2024
f35605a
Added teardown to properly cleanup files
expani Aug 6, 2024
4a02e64
Added comments
expani Aug 6, 2024
c01fb9b
Reading docId sequences from files
expani Aug 6, 2024
d751e60
Using charset while reading file
expani Aug 6, 2024
b833e73
Avoid write/read to files per unique sequence
expani Aug 6, 2024
db1ed1d
Fixing build failure for distro test
expani Aug 6, 2024
42eac06
Added output verification code and cleanup
expani Aug 20, 2024
5fb5b25
Added sequence smaller than 512 elements
expani Aug 20, 2024
5af1e60
Added Bit32Encoder and moved the unrolled version of Bit21Encoder to …
expani Aug 29, 2024
0a07019
Merge branch 'apache:main' into bpv21_main
expani Aug 29, 2024
1cb896a
Merge branch 'apache:main' into bpv21_main
expani Sep 6, 2024
0b59eb3
Separated encode and decode benchmarks - initial commit
expani Sep 11, 2024
bd04aaf
Separated encode and decode benchmarks - initial commit
expani Sep 11, 2024
ad7b749
Taking input file as a system property other than default
expani Sep 11, 2024
5479ac2
Added input scale factor as a sys property for input to benchmark
expani Sep 11, 2024
a3a5c58
Replaced usage of java.io.File and other cleanup
expani Sep 11, 2024
4085679
Refactor
expani Sep 12, 2024
e309031
Trimming numbers to avoid NFE
expani Sep 12, 2024
7dd5d80
Used reflection for generating singleton instances and addressed PR C…
expani Oct 6, 2024
0289c6b
Fixed gradle check errors
expani Oct 7, 2024
e20be32
Fixed gradle check errors
expani Oct 7, 2024
16c6c2b
Added hybrid bit21 encoder
expani Oct 14, 2024
fa34d4e
Added versions.lock for unit testing dependency in lucene-jmh module
expani Oct 14, 2024
a090196
Changed decoder for os.arch amd64 ( x86 processor )
expani Oct 14, 2024
dad37c1
Merge branch 'apache:main' into bpv21_main
expani Oct 15, 2024
fd862b7
Added unit tests to randomly generate docIds for all encoders
expani Oct 15, 2024
68d0bbf
Added unit tests to randomly generate docIds for all encoders
expani Oct 15, 2024
33232e5
Removed resource file with hardcoded docIds
expani Oct 15, 2024
5853093
Removed resource file with hardcoded docIds
expani Oct 15, 2024
8d6f7ca
Refactoring
expani Oct 15, 2024
336308e
Merge branch 'apache:main' into bpv21_main
expani Oct 15, 2024
4fb0a87
Fixed all gradle failures
expani Oct 15, 2024
33fd619
Auto inferring docIdProvider
expani Oct 16, 2024
295531c
Made encoding conditional based on architecture
expani Oct 16, 2024
7312d91
Merge branch 'apache:main' into bpv21_main
expani Oct 16, 2024
359440a
Fixed file cleanup for unit tests
expani Oct 16, 2024
4406d75
Benchmarking effects of only using writeLong instead of writeInt
expani Oct 18, 2024
dc3d6fc
Closing IndexInput after every use
expani Oct 19, 2024
0d1d879
Temporarily added debug loggers to track for larger inputs
expani Oct 19, 2024
5b6b26d
Disabling encode benchmark temporarily
expani Oct 19, 2024
fdca13f
Added extra loggers
expani Oct 19, 2024
b2c45e5
Removed print statements for debugging stages
expani Oct 20, 2024
ecf53d9
Made loading docIdSequences parallel to reduce benchmark time
expani Oct 20, 2024
7f95cd3
Excluding lines with comments
expani Oct 22, 2024
af8b914
Changed readLong to readInt for trailing docIds in leaf block
expani Oct 22, 2024
4d10d9d
Adding new encoder with only readLong instead of readInt to easily co…
expani Oct 24, 2024
da2325c
Adding new encoder with only readLong instead of readInt to easily co…
expani Oct 24, 2024
06dd48f
Added new hybrid encoder based on readlong vs readint tests
expani Oct 25, 2024
775fb38
Added a variation of BPV32 using only r/w long
expani Oct 26, 2024
702580a
Added a variation of BPV32 using only r/w long
expani Oct 26, 2024
1182119
Added arch specific write int/long
expani Dec 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading