Use torch.uint1 to torch.uint7 for Uintx tensor subclass
#672
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/672
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 3a0d131 with merge base 9860194 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
jerryzh168
force-pushed
the
enable-uintx
branch
7 times, most recently
from
54b5569
to
9d12e5e
Compare
test/dtypes/test_uintx.py
Outdated
| @@ -6,7 +6,10 @@ | |||
|
|
|||
| from torchao.dtypes.uintx.Uintx import to_uintx | |||
| from torchao.quantization.quant_api import quantize_, uintx_weight_only | |||
| from torchao.utils import TORCH_VERSION_AFTER_2_5 | |||
| from torchao.utils import ( | |||
| TORCH_VERSION_AFTER_2_3, | |||
There was a problem hiding this comment.
This import makes sense, but also is kind of useless since tests don't run unless Torch version is nightly. Maybe we can change tests to running if the version is after 2.3, and only calling torch.compile() if the torch version is nightly.
There was a problem hiding this comment.
sorry I don't quite follow, this is to guard against reference to torch.uintx, which is added in torch 2.3, so the tests will only run if torch is 2.3+
are you saying to add the version guard for tests as well?
also I didn't see a call for torch.compile in the test, are you saying we should add that
There was a problem hiding this comment.
Oh interesting, I didn't catch that we weren't using torch compile. That is intended usage so it should be tested.
I was trying to say the guard for version 2.5 was for torch compile since the nested sub-classes issue was fixed only recently. Currently tests only run if version after 2.5 so the version after 2.3 guard is not really doing anything.
Now I see it guards the import which is used in a statement before the tests are executed so it is still needed
There was a problem hiding this comment.
right that's correct, 2_3 is for torch.uintx references. actually torch.compile is tested now in test_uintx_target_dtype
jerryzh168
force-pushed
the
enable-uintx
branch
from
9d12e5e
to
fca3674
Compare
jerryzh168
force-pushed
the
enable-uintx
branch
3 times, most recently
from
7d8cdce
to
e06c05a
Compare
Summary: Previously we are using bit_width for uintx quantization, but we can actually use `dtype` directly. But there are still some workaround to convert from torch dtype to bit_width right now, if we want to remove all the hacks, we'd need to support Uintx tensor subclass properly and have `torch.uintx` dispatch to the tensor subclass this is probably not the highest priority for now since good perf is more important. Test Plan: python test/dtypes/test_affine_quantized.py pytest test/dtypes/test_uintx.py Reviewers: Subscribers: Tasks: Tags:
jerryzh168
force-pushed
the
enable-uintx
branch
from
e06c05a
to
3a0d131
Compare
Summary:
Previously we are using bit_width for uintx quantization, but we can actually use
dtypedirectly.But there are still some workaround to convert from torch dtype to bit_width right now, if we want to remove all the hacks, we'd need to support Uintx tensor subclass properly and have
torch.uintxdispatch to the tensor subclass. this is probably not the highest priority for now since good perf is more important.Test Plan:
python test/dtypes/test_affine_quantized.py
pytest test/dtypes/test_uintx.py
Reviewers:
Subscribers:
Tasks:
Tags: