Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Preserve sparsity SPARSEGPT #2282

Merged
merged 4 commits into from
May 17, 2024
Merged

Preserve sparsity SPARSEGPT #2282

merged 4 commits into from
May 17, 2024

Conversation

rahul-tuli
Copy link
Member

This PR incorporates changes from @abhinavnmagic's PR #2222 into new modifier UX

We introduce a new argument preserve_sparsity_mask in SparseGPTModifier which can be used to extend or ignore the base weight mask during SparseGPT application

Test Recipe:

test_stage:
  obcq_modifiers:
    SparseGPTModifier:
      sparsity: 0.5
      block_size: 128
      percdamp: 0.01
      mask_structure: "2:4"
      targets: ["re:model.layers.\\d*$"]
      preserve_sparsity_mask: true

@rahul-tuli rahul-tuli changed the base branch from main to preserve-sparsity-gptq May 13, 2024 14:03
@rahul-tuli rahul-tuli mentioned this pull request May 13, 2024
Merged
7 tasks
Base automatically changed from preserve-sparsity-gptq to gptq-ux-config-groups May 17, 2024 16:15
@rahul-tuli rahul-tuli merged commit 446555f into gptq-ux-config-groups May 17, 2024
@rahul-tuli rahul-tuli deleted the preserve-sparsity-sparsegpt branch May 17, 2024 16:15
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
2 more reviewers

@Satrat Satrat Satrat approved these changes

@bfineran bfineran bfineran approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rahul-tuli @Satrat @bfineran