-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add ChunkKV #51
add ChunkKV #51
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for opening this PR!
The overall code looks good, thanks for adding it. There are several points that I'd ask to address before merging:
- We prefer to have a new press class (
ChunkKVPress
) associated with your paper, as your method (selecting whole chunks) differs fromChunkPress
's idea (compress each chunk independently). Both methods also have different default parameters (e.g.chunk_length
). - After introducing a new class, you can add a new test (similar to
test_chunk_press
). Please do not modify the existing test that usesExpectedAttentionPress
. - Please add your class in the corresponding README section (available presses).
- Most style errors can be fixed my running
make format
. You can test style locally bymake style
. Feel free to add# noqa:
flags if necessary; e.g. sometimes mypy has issues to resolve tensor attributes correctly. - Please don't forget to sign off your commit. See https://github.com/NVIDIA/kvpress/blob/main/CONTRIBUTING.md for additional information.
Feel free to reach out with any questions or comments!
Hi! Thanks for the review and suggestions! I've made the following changes as requested:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the updates, code looks good!
Could you shorten your description of ChunkKVPress
in the README (see also other descriptions)?
Also, there's a minor issue with style checks, it should be easy to fix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for integrating ChunkKVPress, LGTM!
@Dominic789654
|
3f4332f
to
4fb6d1b
Compare
Signed-off-by: Dominic789654 <xliu29@gmu.edu>
Signed-off-by: Dominic789654 <xliu29@gmu.edu>
Signed-off-by: Dominic789654 <xliu29@gmu.edu>
Signed-off-by: Dominic789654 <xliu29@gmu.edu>
Signed-off-by: Dominic789654 <xliu29@gmu.edu>
PR Description
Add ChunkKV paper implementation (https://arxiv.org/abs/2502.00299)
Benchmark Results
Results on Ruler (4096) dataset under different compression ratios (0.1, 0.25, 0.5):
New press checklist
mypress_press.py
in thepresses
directoryMyPress
in__init__.py
README.md
with a 1 liner about my new press in the Available presses sectiondefault_presses
list intests/default_presses.py