Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GPU] Fixes for hybrid quantization #27127

Conversation

vladimir-paramuzov
Copy link
Contributor

@vladimir-paramuzov vladimir-paramuzov commented Oct 18, 2024

Details:

  • set LPT callbacks to handle compression and avoid constant folding for it (taken from [CPU] Decompression support for hybrid models #20973)
  • Allow u8/i8 output data type for compressed onednn FC
  • Disable Dequantize propagation through Transpose if it's a dependency of SDPA to keep Transpose+SDPA fusion

@github-actions github-actions bot added the category: GPU OpenVINO GPU plugin label Oct 18, 2024
@vladimir-paramuzov vladimir-paramuzov added this to the 2024.5 milestone Oct 24, 2024
@vladimir-paramuzov vladimir-paramuzov added this pull request to the merge queue Oct 25, 2024
Merged via the queue into openvinotoolkit:master with commit c21f572 Oct 25, 2024
142 of 144 checks passed
@vladimir-paramuzov vladimir-paramuzov deleted the hybrid_quantization_fixes branch October 25, 2024 07:21
vladimir-paramuzov added a commit to vladimir-paramuzov/openvino that referenced this pull request Oct 29, 2024
This reverts commit c21f572.

Signed-off-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
github-merge-queue bot pushed a commit that referenced this pull request Oct 30, 2024
This reverts commit c21f572.

Signed-off-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
CuriousPanCake pushed a commit to CuriousPanCake/openvino that referenced this pull request Nov 6, 2024
### Details:
- set LPT callbacks to handle compression and avoid constant folding for
it (taken from openvinotoolkit#20973)
 - Allow u8/i8 output data type for compressed onednn FC
- Disable Dequantize propagation through Transpose if it's a dependency
of SDPA to keep Transpose+SDPA fusion
CuriousPanCake pushed a commit to CuriousPanCake/openvino that referenced this pull request Nov 6, 2024
…penvinotoolkit#27308)

This reverts commit c21f572.

Signed-off-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: GPU OpenVINO GPU plugin
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants