Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

does trt-llm support w4a8 (int4 * int8)? #1189

Closed
yyfcc17 opened this issue Feb 29, 2024 · 2 comments
Closed

does trt-llm support w4a8 (int4 * int8)? #1189

yyfcc17 opened this issue Feb 29, 2024 · 2 comments
Assignees
@ncomly-nvidia
Labels
feature request New feature or request

Comments

@yyfcc17
Copy link

yyfcc17 commented Feb 29, 2024

for gpus like a30 and a100, they don't support fp8, is there a plan to support w4a8 (int4 * int8)?
@byshiue
Copy link
Collaborator

byshiue commented Mar 6, 2024

Currently, we don't have such plan in our roadmap. If you are interested, you could create a feature request ticket to help tracking, and we will consider its priority.

@byshiue byshiue added the feature request New feature or request label Mar 6, 2024
@yyfcc17
Copy link
Author

yyfcc17 commented Mar 25, 2024

FYI, there is a pr in cutlass, i will close this.

NVIDIA/cutlass#1413

@yyfcc17 yyfcc17 closed this as completed Mar 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ncomly-nvidia ncomly-nvidia

Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@byshiue @yyfcc17 @ncomly-nvidia