Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[OBCQ][Recipe UX change] Prunen-prunem-to-mask-structure #1802

Merged
merged 8 commits into from
Nov 2, 2023
Merged

[OBCQ][Recipe UX change] Prunen-prunem-to-mask-structure #1802

merged 8 commits into from
Nov 2, 2023

Conversation

rahul-tuli
Copy link
Member

@rahul-tuli rahul-tuli commented Oct 31, 2023 •
edited
Loading

Before this PR OBCQ recipes needed prunen/prunem to be specified individually. This PR makes a UX change such that now, a mask_structure of the form N:M can be defined; the where both N, M are integers (similar to prunen and prunem)

Test command:

python3 src/sparseml/transformers/sparsification/obcq/obcq.py faceb
ook/opt-350m c4 --recipe src/sparseml/transformers/sparsification/obcq/example.yaml --eval wikitext2

Output:

.
.
.
2023-10-31 07:03:54 sparseml.modifiers.quantization.pytorch INFO     Running quantization calibration using calibration_dataloader
2023-10-31 07:04:02 sparseml.modifiers.obcq.utils.helpers INFO     Evaluating perplexity...
2023-10-31 07:04:03 sparseml.modifiers.obcq.utils.helpers INFO     tensor(34.7602, device='cuda:0')
2023-10-31 07:04:05 sparseml.modifiers.obcq.utils.helpers INFO     tensor(36.3635, device='cuda:0')
2023-10-31 07:04:06 sparseml.modifiers.obcq.utils.helpers INFO     tensor(37.2799, device='cuda:0')
2023-10-31 07:04:08 sparseml.modifiers.obcq.utils.helpers INFO     tensor(36.4086, device='cuda:0')
2023-10-31 07:04:09 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.7602, device='cuda:0')
2023-10-31 07:04:11 sparseml.modifiers.obcq.utils.helpers INFO     tensor(35.3937, device='cuda:0')
2023-10-31 07:04:12 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.7370, device='cuda:0')
2023-10-31 07:04:14 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.7335, device='cuda:0')
2023-10-31 07:04:16 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.4511, device='cuda:0')
2023-10-31 07:04:17 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.1673, device='cuda:0')
2023-10-31 07:04:19 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.1939, device='cuda:0')
2023-10-31 07:04:20 sparseml.modifiers.obcq.utils.helpers INFO     tensor(34.0454, device='cuda:0')
2023-10-31 07:04:22 sparseml.modifiers.obcq.utils.helpers INFO     tensor(34.4664, device='cuda:0')
2023-10-31 07:04:23 sparseml.modifiers.obcq.utils.helpers INFO     tensor(34.3553, device='cuda:0')
2023-10-31 07:04:25 sparseml.modifiers.obcq.utils.helpers INFO     tensor(34.0684, device='cuda:0')
2023-10-31 07:04:26 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.9638, device='cuda:0')
2023-10-31 07:04:28 sparseml.modifiers.obcq.utils.helpers INFO     tensor(33.9543, device='cuda:0')
2023-10-31 07:04:29 sparseml.modifiers.obcq.utils.helpers INFO     tensor(34.1352, device='cuda:0')
2023-10-31 07:04:29 sparseml.modifiers.obcq.utils.helpers INFO     Perplexity: 34.135204

@rahul-tuli rahul-tuli marked this pull request as ready for review October 31, 2023 11:11
@rahul-tuli rahul-tuli self-assigned this Oct 31, 2023
@mgoin
Copy link
Member

mgoin commented Oct 31, 2023

@rahul-tuli could you put an example of this in a recipe? especially so if this will be a breaking change to existing recipes

dsikka
dsikka previously approved these changes Oct 31, 2023
View reviewed changes
Copy link
Contributor

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Just two questions since unfamiliar with this work

src/sparseml/modifiers/obcq/base.py Show resolved Hide resolved
src/sparseml/transformers/sparsification/obcq/example_llama.yaml Show resolved Hide resolved
src/sparseml/transformers/sparsification/obcq/example.yaml Show resolved Hide resolved
@dsikka dsikka self-requested a review November 1, 2023 14:28
@rahul-tuli rahul-tuli merged commit f1626b7 into main Nov 2, 2023
11 checks passed
@rahul-tuli rahul-tuli deleted the prunen-prunem-to-mask-structure branch November 2, 2023 14:22
bfineran pushed a commit that referenced this pull request Nov 16, 2023
@rahul-tuli
[OBCQ][Recipe UX change] Prunen-prunem-to-mask-structure (#1802)
754fbd3 
* Make docstring consistent

* Rename `_validate_layerwise_sparisity()`
         --> `_validate_layerwise_sparsity()`

* Propagate mask_structure

* Update recipes
bfineran pushed a commit that referenced this pull request Nov 16, 2023
@rahul-tuli @bfineran
[OBCQ][Recipe UX change] Prunen-prunem-to-mask-structure (#1802)
e81f14d 
* Make docstring consistent

* Rename `_validate_layerwise_sparisity()`
         --> `_validate_layerwise_sparsity()`

* Propagate mask_structure

* Update recipes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dsikka dsikka dsikka approved these changes

@mgoin mgoin mgoin approved these changes

@Satrat Satrat Awaiting requested review from Satrat

@bfineran bfineran Awaiting requested review from bfineran

@dbogunowicz dbogunowicz Awaiting requested review from dbogunowicz

Assignees

@rahul-tuli rahul-tuli

Labels
mle-team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rahul-tuli @mgoin @dsikka