Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[KNOWN BUG] gguf + GPU AOTI Inference bug due to PT version, fix in progress #1423

Closed
Jack-Khuu opened this issue Dec 14, 2024 · 0 comments
Closed
Assignees
Labels
bug Something isn't working CI Infra Issues related to CI infrastructure and setup Compile / AOTI Issues related to AOT Inductor and torch compile Known Gaps These are known Gaps/Issues/Bug items in torchchat Quantization Issues related to Quantization or torchao triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@Jack-Khuu
Copy link
Contributor

🐛 Describe the bug

As of #1367, torchchat/main is failing 3-5 CI jobs related to GPU AOTI inference and GGUF inference

GPU AOTI inference will be fixed with a pinbump to pytorch/pytorch#143236
GGUF AO bug is being addressed in #1404

Versions

bb72b09

@Jack-Khuu Jack-Khuu added bug Something isn't working Known Gaps These are known Gaps/Issues/Bug items in torchchat Compile / AOTI Issues related to AOT Inductor and torch compile Quantization Issues related to Quantization or torchao CI Infra Issues related to CI infrastructure and setup labels Dec 14, 2024
@Jack-Khuu Jack-Khuu self-assigned this Dec 14, 2024
@Jack-Khuu Jack-Khuu added Triage review Triage assistance requested triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module and removed Triage review Triage assistance requested labels Dec 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working CI Infra Issues related to CI infrastructure and setup Compile / AOTI Issues related to AOT Inductor and torch compile Known Gaps These are known Gaps/Issues/Bug items in torchchat Quantization Issues related to Quantization or torchao triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests

1 participant