Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Conversation

abhinavnmagic
Copy link
Contributor

Motivation

If we run SparseGPT on a base model at some sparsity, the sparsity mask after SparseGPT could be very different from the initial one. In other words, SparseGPT doesn't preserve the initial sparsities of the base model. This PR fixes that.
If preserve_sparsity_mask=True in the recipe, the sparsity mask of the base model being pruned is preserve. This can be useful for multiple scenarios: one-shot from 50->70% sparsity, going from unstructured to structured sparsity or vice versa, etc.

Testing

The implementation was tested by applying 2:4 sparsity on a 70% sparse model. The resulting model was checked if it preserves the 70% sparsity masks (ie. weights which were pruned out in the base model remain pruned), and if it has 2:4 sparsity pattern.

Recipe

test_stage:
  obcq_modifiers:
    SparseGPTModifier:
      sparsity: 0.5
      block_size: 128
      sequential_update: false
      quantize: false
      percdamp: 0.01
      mask_structure: "2:4"
      targets: ["re:model.layers.\\d*$"]
      preserve_sparsity_mask: true

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
abhinavnmagic and others added 2 commits April 5, 2024 11:48

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
@jeanniefinks
Copy link
Member

Per the main README announcement, SparseML is being deprecated by June 2, 2025. Closing the PR as work has been suspended; thank you for the inputs and support!

@jeanniefinks jeanniefinks deleted the abhinav/preserve_sparsity_mask_sgpt branch May 29, 2025 23:38
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants