New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GDINO - separate BERT into a standalone encoder stage for easy quantization #21

Open

rhysdg opened this issue Aug 2, 2024 · 0 comments

Assignees

Labels

Owner

rhysdg commented Aug 2, 2024 •

edited

Loading

One thing we do know for certain is that that onnxruntime now have BERT support via their transformer optimizer, it;d be great to pull this out and treat it for either mixed precision or quant.
The rest of the network can then become a self contained optimisation problem or remain as a torch eager execution with an AMP context

rhysdg added the enhancement label

rhysdg self-assigned this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment