is_linear fix for MHA #1141
Merged
is_linear fix for MHA #1141
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1141
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 4f45e03 with merge base 4b563f2 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
jerryzh168
reviewed
Oct 23, 2024
| @@ -234,6 +234,7 @@ def _is_linear(mod, *args): | |||
| and not isinstance(mod.weight, AffineQuantizedTensor) | |||
| and not isinstance(mod.weight, LinearActivationQuantizedTensor) | |||
| and not isinstance(mod.weight, AffineFakeQuantizedTensor) | |||
| and not isinstance(mod, nn.modules.linear.NonDynamicallyQuantizableLinear) | |||
There was a problem hiding this comment.
nit: maybe add a Note here, to say that if MHA is refactored to not use this module we'd need a different solution
jerryzh168
reviewed
Oct 23, 2024
| @@ -1278,6 +1278,32 @@ def test_autoquant_compile(self, device, dtype, m1, m2, k, n): | |||
| sqnr = SQNR(out, out2) | |||
| self.assertTrue(sqnr >= 30) | |||
|
|
|||
| @parameterized.expand(COMMON_DEVICE_DTYPE) | |||
| @unittest.skipIf(not TORCH_VERSION_AT_LEAST_2_5, "autoquant requires 2.5+.") | |||
There was a problem hiding this comment.
according to CI, probably need to add skip if cuda is not available, or change this
ao/test/integration/test_integration.py
Line 97 in 629aee1
Summary: filter fn may need access to parent types of module as in the case with mha Test Plan: TODO Reviewers: Subscribers: Tasks: Tags:
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags:
HDCharles
force-pushed
the
047_fix_mha
branch
from
2b08079
to
4f45e03
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary: filter fn may need access to parent types of module as in the case with mha
Test Plan: python /home/cdhernandez/local/ao/test/integration/test_integration.py -k "test_autoquant_mha"
Reviewers:
Subscribers:
Tasks:
Tags: