AveragePooling2D layer is not mapped to DPUCZDX8G

[takashi-imagawa]

I want to implement my tensorflow2 model, which includes some AveragePooling2D layers, to DPUCZDX8G (DPUCZDX8G_ISA1_B4096).
The user guide of DPUCZDX8G (PG338, v4.0) says that average pooling is supported.

However, the inspector says, "`AveragePooling2D` is not supported by target".

Why are the average pooling layers not mapped to the DPU, and what should I do to map them to the DPUCZDX8G?

The docker image version and workflow are as follows.

• Docker Image Version: 2.5.0.1260 (GPU)
• Vitis AI Git Hash: fec2756
• using TensorFlow 2.8 Workflow with conda activate vitis-ai-tensorflow2

The followings are a simplified code and an execution log.

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, AveragePooling2D
from tensorflow_model_optimization.quantization.keras import vitis_inspect

input_layer = Input([32, 16, 8], name='input')
net = AveragePooling2D(pool_size=2, strides=1, name='avg')(input_layer)
model = Model(inputs=input_layer, outputs=net)

inspector = vitis_inspect.VitisInspector(target='0x101000017010407', # 'DPUCZDX8G_ISA1_B4096',
                                         target_type='fingerprint')
inspector.inspect_model(model,
                        plot=True,
                        plot_file="model.svg",
                        dump_results=True,
                        dump_results_file="inspect_results.txt",
                        verbose=2,
                        )

[VAI INFO] Update include_bias_corr: False
[VAI INFO] Update include_fast_ft: False
[VAI INFO] Update include_cle: False
[VAI INFO] VitisPof2SOptimizeTransformsPipeline configs:
[VAI INFO] - remove_dropout: True
[VAI INFO] - separate_conv_act: True
[VAI INFO] - fold_conv_bn: True
[VAI INFO] - convert_bn_to_dwconv: True
[VAI INFO] - convert_relu6_to_relu: False
[VAI INFO] - convert_tf_op_to_keras: True
[VAI INFO] - include_cle: False
[VAI INFO] - cle_to_relu6: False
[VAI INFO] - cle_steps: 10
[VAI INFO] - cle_balance_method: max
[VAI INFO] - cle_weight_threshold: 0.1
[VAI INFO] - train_with_bn: False
[VAI INFO] user_quantize_config:
[VAI INFO] - input_bit: None
[VAI INFO] - input_method: None
[VAI INFO] - input_symmetry: None
[VAI INFO] - input_per_channel: None
[VAI INFO] - input_round_mode: None
[VAI INFO] - weight_bit: None
[VAI INFO] - weight_method: None
[VAI INFO] - weight_symmetry: None
[VAI INFO] - weight_per_channel: None
[VAI INFO] - weight_round_mode: None
[VAI INFO] - bias_bit: None
[VAI INFO] - bias_method: None
[VAI INFO] - bias_symmetry: None
[VAI INFO] - bias_per_channel: None
[VAI INFO] - bias_round_mode: None
[VAI INFO] - activation_bit: None
[VAI INFO] - activation_method: None
[VAI INFO] - activation_symmetry: None
[VAI INFO] - activation_per_channel: None
[VAI INFO] - activation_round_mode: None
[VAI INFO] - use_framework_quant: None
[VAI INFO] - input_layers: []
[VAI INFO] - output_layers: []
[VAI INFO] - ignore_layers: []
[VAI INFO] VitisPof2SQuantizeTransformsPipeline configs:
[VAI INFO] - convert_sigmoid_to_hard_sigmoid: True
[VAI INFO] - convert_hard_sigmoid_to_dpu_version: True
[VAI INFO] - convert_average_pooling2d_to_dpu_version: True
[VAI INFO] - convert_leaky_relu_to_dpu_version: True
[VAI INFO] - freeze_bn_delay: -1
[VAI INFO] VitisPof2SRefineTransformsPipeline configs:
[VAI INFO] - adjust_dpu_sigmoid: True
[VAI INFO] - adjust_dpu_shift_cut: True
[VAI INFO] - adjust_dpu_shift_bias: True
[VAI INFO] - adjust_dpu_shift_read: True
[VAI INFO] - include_fast_ft: False
[VAI INFO] - fast_ft_epochs: 10
[VAI INFO] - include_bias_corr: False
[VAI INFO] - convert_to_fs_quantize_strategy: False
[VAI WARNING] Layer avg(VitisAveragePooling2D)'s `AveragePooling2D` is not supported by target
[VAI INFO] Inspect Results:
[MODEL INFO]:
________________________________________________________________________________________________________________________
Model Name: model
________________________________________________________________________________________________________________________
ID          Name                    Type                    Device      Notes                                           
========================================================================================================================
0/1         input                   InputLayer              INPUT                                                       
------------------------------------------------------------------------------------------------------------------------
1/1         avg                     AveragePooling2D        CPU         Converted to VitisAveragePooling2D;             
                                                                        `AveragePooling2D` is not supported by target   
------------------------------------------------------------------------------------------------------------------------
========================================================================================================================
[SUMMARY INFO]:
- [Target Name]: DPUCZDX8G_ISA1_B4096_0101000017010407
- [Target Type]: DPUCZDX8G
- [Total Layers]: 2
- [Layer Types]: InputLayer(1) AveragePooling2D(1) 
- [Partition Results]: INPUT(1) CPU(1) 
========================================================================================================================
[NOTES INFO]:
- [1/1] Layer avg (Type:AveragePooling2D, Device:CPU):
    * Converted to VitisAveragePooling2D
    * `AveragePooling2D` is not supported by target
========================================================================================================================
[VAI INFO] Start plotting model to model.svg
[VAI INFO] Inspected model has been plotted in: model.svg.
[VAI INFO] Start dumping inspected results to inspect_results.txt
[VAI INFO] Inspected results has been dumped in: inspect_results.txt.
[VAI INFO] Inspect Finished.

[Fabuloso-debug]

Hi, I have the same issue, did you solve it?

[foolmarks]

AveragePooling support needs to be enabled as part of the DPU configuration - if its not enabled, the compiler will map it to a CPU subgraph.

[Fabuloso-debug]

Thank you for your reply, but unfortunately I think that averagePooling is already enabled
My DPU is DPUCZDX8G_ISA1_B4096, {"fingerprint":"0x101000016010407"}

The user guide of DPUCZDX8G (PG338, v4.0) says:
"The AveragePool option determines whether the average pooling operation will be performed on the DPUCZDX8G or not. This feature is always enabled, and supports rectangle sizes."

Anyway, I added POOL_AVG_ENABLED to my dpu_conf.vh, I rebuilt all and the json file hasn't changed, I still have {"fingerprint":"0x101000016010407"}

[omgitsmj24]

I'm also encoutering a similar problem regarding DepthwiseConv2D layer which said to be supported by DPUCZDX8G_ISA1_B4096 in the document UG1414. Have you been able to fix this issue yet? My system is the same with op's.

I've attached below the same code with op's code with the change in the layer from AveragePooling to DepthwiseConv2D and still doesn't seem to map to DPU.

I've also included output from inspect.

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'

from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, AveragePooling2D, DepthwiseConv2D
from tensorflow_model_optimization.quantization.keras import vitis_inspect

input_layer = Input([960, 960, 3], name='input')
net = DepthwiseConv2D(kernel_size=(3, 3), padding='same', name='DepthwiseConv2D')(input_layer)
model = Model(inputs=input_layer, outputs=net)

inspector = vitis_inspect.VitisInspector(target='DPUCZDX8G_ISA1_B4096')
inspector.inspect_model(model,
                        plot=True,
                        plot_file="model.svg",
                        dump_results=True,
                        dump_results_file="inspect_results.txt",
                        verbose=2,
                        )

[VAI INFO] Update include_bias_corr: False
[VAI INFO] Update include_fast_ft: False
[VAI INFO] Update include_cle: False
[VAI INFO] VitisPof2SOptimizeTransformsPipeline configs:
[VAI INFO] - remove_dropout: True
[VAI INFO] - separate_conv_act: True
[VAI INFO] - fold_conv_bn: True
[VAI INFO] - convert_bn_to_dwconv: True
[VAI INFO] - convert_relu6_to_relu: False
[VAI INFO] - convert_tf_op_to_keras: True
[VAI INFO] - include_cle: False
[VAI INFO] - cle_to_relu6: False
[VAI INFO] - cle_steps: 10
[VAI INFO] - cle_balance_method: max
[VAI INFO] - cle_weight_threshold: 0.1
[VAI INFO] - train_with_bn: False
[VAI INFO] user_quantize_config:
[VAI INFO] - input_bit: None
[VAI INFO] - input_method: None
[VAI INFO] - input_symmetry: None
[VAI INFO] - input_per_channel: None
[VAI INFO] - input_round_mode: None
[VAI INFO] - weight_bit: None
[VAI INFO] - weight_method: None
[VAI INFO] - weight_symmetry: None
[VAI INFO] - weight_per_channel: None
[VAI INFO] - weight_round_mode: None
[VAI INFO] - bias_bit: None
[VAI INFO] - bias_method: None
[VAI INFO] - bias_symmetry: None
[VAI INFO] - bias_per_channel: None
[VAI INFO] - bias_round_mode: None
[VAI INFO] - activation_bit: None
[VAI INFO] - activation_method: None
[VAI INFO] - activation_symmetry: None
[VAI INFO] - activation_per_channel: None
[VAI INFO] - activation_round_mode: None
[VAI INFO] - use_framework_quant: None
[VAI INFO] - input_layers: []
[VAI INFO] - output_layers: []
[VAI INFO] - ignore_layers: []
[VAI INFO] VitisPof2SQuantizeTransformsPipeline configs:
[VAI INFO] - convert_sigmoid_to_hard_sigmoid: True
[VAI INFO] - convert_hard_sigmoid_to_dpu_version: True
[VAI INFO] - convert_average_pooling2d_to_dpu_version: True
[VAI INFO] - convert_leaky_relu_to_dpu_version: True
[VAI INFO] - freeze_bn_delay: -1
[VAI INFO] VitisPof2SRefineTransformsPipeline configs:
[VAI INFO] - adjust_dpu_sigmoid: True
[VAI INFO] - adjust_dpu_shift_cut: True
[VAI INFO] - adjust_dpu_shift_bias: True
[VAI INFO] - adjust_dpu_shift_read: True
[VAI INFO] - include_fast_ft: False
[VAI INFO] - fast_ft_epochs: 10
[VAI INFO] - include_bias_corr: False
[VAI INFO] - convert_to_fs_quantize_strategy: False
[VAI WARNING] Layer DepthwiseConv2D(DepthwiseConv2D)'s `DepthwiseConv2D` is not supported by target
[VAI INFO] Inspect Results:
[MODEL INFO]:
________________________________________________________________________________________________________________________
Model Name: model_1
________________________________________________________________________________________________________________________
ID          Name                    Type                    Device      Notes                                           
========================================================================================================================
0/1         input                   InputLayer              INPUT                                                       
------------------------------------------------------------------------------------------------------------------------
1/1         DepthwiseConv2D         DepthwiseConv2D<linear> CPU         `DepthwiseConv2D` is not supported by target    
------------------------------------------------------------------------------------------------------------------------
========================================================================================================================
[SUMMARY INFO]:
- [Target Name]: DPUCZDX8G_ISA1_B4096
- [Target Type]: DPUCZDX8G
- [Total Layers]: 2
- [Layer Types]: InputLayer(1) DepthwiseConv2D<linear>(1) 
- [Partition Results]: INPUT(1) CPU(1) 
========================================================================================================================
[NOTES INFO]:
- [1/1] Layer DepthwiseConv2D (Type:DepthwiseConv2D<linear>, Device:CPU):
    * `DepthwiseConv2D` is not supported by target
========================================================================================================================
[VAI INFO] Start plotting model to model.svg
[VAI INFO] Inspected model has been plotted in: model.svg.
[VAI INFO] Start dumping inspected results to inspect_results.txt
[VAI INFO] Inspected results has been dumped in: inspect_results.txt.
[VAI INFO] Inspect Finished.

• Edit: DepthwiseConv2D is a part of the block consist of DepthwiseConv2D + BN + Relu/Leaky Relu, all of which have been mapped to CPU instead of DPU

[zhenzhen-AMD]

Hi @takashi-imagawa @foolmarks @omgitsmj24 ，
The issue has been fixed in the vitis-ai-3.0 Docker image. Please use the latest released Docker version.
On DPUCZDX8G_ISA1_B4096, the AveragePooling2D and DepthwiseConv2D layers will be allocated to the DPU.

[zhenzhen-AMD]

Hi @takashi-imagawa @foolmarks @omgitsmj24 ,
I will close this issue. If you have any new questions in the future, please feel free to open a new issue. Thank you.