Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ Docs ] Update main readme #77

Merged
merged 51 commits into from
Aug 12, 2024
Merged

[ Docs ] Update main readme #77

merged 51 commits into from
Aug 12, 2024

Conversation

robertgshaw2-redhat
Copy link
Collaborator

@robertgshaw2-redhat robertgshaw2-redhat commented Aug 11, 2024 •
edited
Loading

SUMMARY:

  • flip order to put examples more front and center
  • update install instructions to suggest pypi install
  • update example to use w8a8 instead of w4a16
  • update example to have pointer to using custom calibration data

TEST PLAN:

  • manually run example

Satrat
Satrat approved these changes Aug 12, 2024
View reviewed changes
Copy link
Contributor

@Satrat Satrat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great!

@mgoin mgoin merged commit 919a0a1 into main Aug 12, 2024
7 of 12 checks passed
markmc pushed a commit to markmc/llm-compressor that referenced this pull request Nov 13, 2024
@dbogunowicz
Marlin24 Compressor (vllm-project#77)
14b1db1 
* add marlin compressor

* full implementation running

* save meta matrix

* fix type

* change axis we pack on

* debug

* fix offset

* fix scale shape

* cast dtype

* cleanup

* fix

* avoid extra transpose

* add uncompressed weights

* unit tests

* cleanup

* Update src/compressed_tensors/compressors/marlin_24.py

Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

* Update src/compressed_tensors/compressors/marlin_24.py

Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>

* style

* move helper fn

---------

Co-authored-by: dbogunowicz <97082108+dbogunowicz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Satrat Satrat Satrat approved these changes

@mgoin mgoin mgoin approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@robertgshaw2-redhat @mgoin @Satrat