Skip to content

✨[Feature] Is there a plan to support to convert quantized PT2 to trt ? #3471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
MaltoseFlower opened this issue Apr 14, 2025 · 1 comment
Open

✨[Feature] Is there a plan to support to convert quantized PT2 to trt ? #3471

MaltoseFlower opened this issue Apr 14, 2025 · 1 comment
Assignees
@narendasan @lanluo-nvidia
Labels
feature request New feature or request

Comments

@MaltoseFlower
Copy link

when i use torch-tensorrt 2.4.0 to convert a quantized PT2 to trt, i got this error blow.

Image

I wonder whether it will support in ther future? Or, i am able to do this based on the current version (maybe through converting the torch.ops.quantized_decomposed.dequantize_per_tensor.default operator to other quantization operator? ).
@MaltoseFlower MaltoseFlower added the feature request New feature or request label Apr 14, 2025
@narendasan
Copy link
Collaborator

@lanluo-nvidia Can you take a look at this post FP4?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@narendasan narendasan

@lanluo-nvidia lanluo-nvidia

Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@narendasan @MaltoseFlower @lanluo-nvidia