Details about LPBQ Quantization in QNN Backend

[chenghuaWang]

I'm currently learning about QNN and its LPBQ quantization in Executorch. My environment consists of a Snapdragon SM8850 chipset with HTP architecture v81 and QNN version 2.41.0.251128. When I use the low-level API(qnn_interface.graphAddNode()) to add a FullyConnected node, QNN reports the error: "FullyConnected: Block expansion encoding not supported." However, the op validation succeeds.

I observed that in Executorch, all linear layers are rewritten to conv2d layers. This raises the question: Is LPBQ quantization only supported for Conv2D and not for Linear layers? Or could this be a limitation specific to my QNN version?

[cccclai]

@haowhsu-quic @winskuo-quic @winskuo-quic @shewu-quic thoughts on this?

[shewu-quic]

Hi @chenghuaWang ,

Based on QNN documentation, LPBQ quantization support both of Conv2D and Linear operation.
You can apply this patch to run unit test for Linear with LPBQ.

Reproduce Command:

python3 backends/qualcomm/tests/test_qnn_delegate.py TestQNNQuantizedOperator.test_qnn_backend_16a4w_per_block_linear -b build-android  -H ${HOST} -s ${DEVICE} -m SM8850 -r {EXECUTORCH_ROOT} -a unit_test

Patch:

diff --git a/backends/qualcomm/tests/test_qnn_delegate.py b/backends/qualcomm/tests/test_qnn_delegate.py
index c57dbbcc33..7267efbb36 100644
--- a/backends/qualcomm/tests/test_qnn_delegate.py
+++ b/backends/qualcomm/tests/test_qnn_delegate.py
@@ -2186,6 +2186,18 @@ class TestQNNQuantizedOperator(TestQNN):
             quant_dtype=QuantDtype.use_16a4w,
         )
         self.lower_module_and_test_output(module, sample_input)
+        
+    def test_qnn_backend_16a4w_per_block_linear(self):
+        module = Linear(use_bias=False)  # noqa: F405
+        sample_input = (torch.randn([3, 512]),)
+        module = self.get_qdq_module(
+            module,
+            sample_input,
+            is_linear_per_channel=True,
+            quant_dtype=QuantDtype.use_16a4w_block,
+            block_size_map={"linear": (1, 32)},
+        )
+        self.lower_module_and_test_output(module, sample_input)
 
     def test_qnn_backend_16a4w_per_channel_linear_with_bias(self):
         module = Linear()  # noqa: F405

[chenghuaWang]

@shewu-quic Thanks! Your patch works fine for me. Is there a way to get QNN log info during ExecuTorch processing?

[shewu-quic]

Glad to hear the good news. Yes, you can set debug as True to get more QNN related info. By default, the log_level is set to QnnExecuTorchLogLevel.kLogLevelError, which means only error messages are displayed. If you're using the example script in examples/qualcomm, you can update it here to enable debug mode.