[VL] When performance degradation，What factors may affect the performance？

[rhh777]

Description

Did the community find performance degradation when testing tpcds? According to my own test, Overall performance improved significantly，most of the SQLs have been improved, but some SQLs have significantly decreased performance, such as query11. I checked the execution plan and found no fallback. What factors may affect the performance?

Spark: 3.2.2
Gluten：0.5.0
TPCDS：1T Parquet

command

spark-sql \
--master yarn \
--deploy-mode client \
--conf spark.plugins=io.glutenproject.GlutenPlugin \
--conf spark.gluten.loadLibFromJar=true \
--conf spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager \
--driver-cores 1 \
--driver-memory 5G \
--executor-cores 5 \
--executor-memory 10G \
--num-executors 10 \
--conf spark.memory.offHeap.enabled=true \
--conf spark.memory.offHeap.size=11G \

== Physical Plan ==
AdaptiveSparkPlan (196)
+- == Final Plan ==
   GlutenColumnarToRowExec (122)
   +- TakeOrderedAndProjectExecTransformer (121)
      +- ^ ProjectExecTransformer (119)
         +- ^ GlutenBroadcastHashJoinExecTransformer Inner (118)
            :- ^ ProjectExecTransformer (91)
            :  +- ^ GlutenBroadcastHashJoinExecTransformer Inner (90)
            :     :- ^ GlutenBroadcastHashJoinExecTransformer Inner (61)
            :     :  :- BroadcastQueryStage (33)
            :     :  :  +- ColumnarBroadcastExchange (32)
            :     :  :     +- CoalesceBatches (31)
            :     :  :        +- ColumnarAQEShuffleRead (30)
            :     :  :           +- ShuffleQueryStage (29)
            :     :  :              +- ColumnarExchange (28)
            :     :  :                 +- ^ ProjectExecTransformer (26)
            :     :  :                    +- ^ GlutenFilterExecTransformer (25)
            :     :  :                       +- ^ GlutenHashAggregateExecTransformer (24)
            :     :  :                          +- CoalesceBatches (23)
            :     :  :                             +- ColumnarAQEShuffleRead (22)
            :     :  :                                +- ShuffleQueryStage (21)
            :     :  :                                   +- ColumnarExchange (20)
            :     :  :                                      +- ^ ProjectExecTransformer (18)
            :     :  :                                         +- ^ GlutenHashAggregateExecTransformer (17)
            :     :  :                                            +- ^ ProjectExecTransformer (16)
            :     :  :                                               +- ^ GlutenBroadcastHashJoinExecTransformer Inner (15)
            :     :  :                                                  :- ^ ProjectExecTransformer (9)
            :     :  :                                                  :  +- ^ GlutenBroadcastHashJoinExecTransformer Inner (8)
            :     :  :                                                  :     :- BroadcastQueryStage (5)
            :     :  :                                                  :     :  +- ColumnarBroadcastExchange (4)
            :     :  :                                                  :     :     +- ^ GlutenFilterExecTransformer (2)
            :     :  :                                                  :     :        +- ^ Scan parquet tpcds_bin_partitioned_parquet_1024.customer (1)
            :     :  :                                                  :     +- ^ GlutenFilterExecTransformer (7)
            :     :  :                                                  :        +- ^ Scan parquet tpcds_bin_partitioned_parquet_1024.store_sales (6)
            :     :  :                                                  +- BroadcastQueryStage (14)
            :     :  :                                                     +- ColumnarBroadcastExchange (13)
            :     :  :                                                        +- ^ GlutenFilterExecTransformer (11)
            :     :  :                                                           +- ^ Scan parquet tpcds_bin_partitioned_parquet_1024.date_dim (10)
            :     :  +- CoalesceBatches (60)
            :     :     +- ColumnarAQEShuffleRead (59)
            :     :        +- ShuffleQueryStage (58)
            :     :           +- ColumnarExchange (57)
            :     :              +- ^ ProjectExecTransformer (55)
            :     :                 +- ^ GlutenHashAggregateExecTransformer (54)
            :     :                    +- CoalesceBatches (53)
            :     :                       +- ColumnarAQEShuffleRead (52)
            :     :                          +- ShuffleQueryStage (51)
            :     :                             +- ColumnarExchange (50)
            :     :                                +- ^ ProjectExecTransformer (48)
            :     :                                   +- ^ GlutenHashAggregateExecTransformer (47)
            :     :                                      +- ^ ProjectExecTransformer (46)
            :     :                                         +- ^ GlutenBroadcastHashJoinExecTransformer Inner (45)
            :     :                                            :- ^ ProjectExecTransformer (39)
            :     :                                            :  +- ^ GlutenBroadcastHashJoinExecTransformer Inner (38)
            :     :                                            :     :- BroadcastQueryStage (35)
            :     :                                            :     :  +- ReusedExchange (34)
            :     :                                            :     +- ^ GlutenFilterExecTransformer (37)
            :     :                                            :        +- ^ Scan parquet tpcds_bin_partitioned_parquet_1024.store_sales (36)
            :     :                                            +- BroadcastQueryStage (44)
            :     :                                               +- ColumnarBroadcastExchange (43)
            :     :                                                  +- ^ GlutenFilterExecTransformer (41)
            :     :                                                     +- ^ Scan parquet tpcds_bin_partitioned_parquet_1024.date_dim (40)
            :     +- BroadcastQueryStage (89)
            :        +- ColumnarBroadcastExchange (88)
            :           +- CoalesceBatches (87)
            :              +- ColumnarAQEShuffleRead (86)
            :                 +- ShuffleQueryStage (85)
            :                    +- ColumnarExchange (84)
            :                       +- ^ ProjectExecTransformer (82)
            :                          +- ^ ProjectExecTransformer (81)
            :                             +- ^ GlutenFilterExecTransformer (80)
            :                                +- ^ GlutenHashAggregateExecTransformer (79)
            :                                   +- CoalesceBatches (78)
            :                                      +- ColumnarAQEShuffleRead (77)
            :                                         +- ShuffleQueryStage (76)
            :                                            +- ColumnarExchange (75)
            :                                               +- ^ ProjectExecTransformer (73)
            :                                                  +- ^ GlutenHashAggregateExecTransformer (72)
            :                                                     +- ^ ProjectExecTransformer (71)
            :                                                        +- ^ GlutenBroadcastHashJoinExecTransformer Inner (70)
            :                                                           :- ^ ProjectExecTransformer (67)
            :                                                           :  +- ^ GlutenBroadcastHashJoinExecTransformer Inner (66)
            :                                                           :     :- BroadcastQueryStage (63)
            :                                                           :     :  +- ReusedExchange (62)
            :                                                           :     +- ^ GlutenFilterExecTransformer (65)
            :                                                           :        +- ^ Scan parquet tpcds_bin_partitioned_parquet_1024.web_sales (64)
            :                                                           +- BroadcastQueryStage (69)
            :                                                              +- ReusedExchange (68)
            +- BroadcastQueryStage (117)
               +- ColumnarBroadcastExchange (116)
                  +- CoalesceBatches (115)
                     +- ColumnarAQEShuffleRead (114)
                        +- ShuffleQueryStage (113)
                           +- ColumnarExchange (112)
                              +- ^ ProjectExecTransformer (110)
                                 +- ^ GlutenHashAggregateExecTransformer (109)
                                    +- CoalesceBatches (108)
                                       +- ColumnarAQEShuffleRead (107)
                                          +- ShuffleQueryStage (106)
                                             +- ColumnarExchange (105)
                                                +- ^ ProjectExecTransformer (103)
                                                   +- ^ GlutenHashAggregateExecTransformer (102)
                                                      +- ^ ProjectExecTransformer (101)
                                                         +- ^ GlutenBroadcastHashJoinExecTransformer Inner (100)
                                                            :- ^ ProjectExecTransformer (97)
                                                            :  +- ^ GlutenBroadcastHashJoinExecTransformer Inner (96)
                                                            :     :- BroadcastQueryStage (93)
                                                            :     :  +- ReusedExchange (92)
                                                            :     +- ^ GlutenFilterExecTransformer (95)
                                                            :        +- ^ Scan parquet tpcds_bin_partitioned_parquet_1024.web_sales (94)
                                                            +- BroadcastQueryStage (99)
                                                               +- ReusedExchange (98)