`BlockInputStream->dumpTree()` is very slow if the input stream tree contains multiple `SharedQueryBlockInputStream` #4494

windtalker · 2022-03-30T04:08:29Z

Bug Report

Please answer these questions before submitting your issue. Thanks!
For a query pipeline:

Union
 ExchangeSender × 36
  Expression
   SharedQuery
    ParallelAggregating
     Expression × 36
      Expression
       Expression
        Expression
         Expression
          Expression
           Expression
            Expression
             Expression
              Expression
               Expression
                Expression
                 Expression
                  Expression
                   SharedQuery
                    ParallelAggregating
                     Expression × 36
                      Expression
                       Expression
                        Expression
                         Squashing
                          TiRemoteBlockInputStream(ExchangeReceiver)

It took more than 3 second with intel i710700F for function call DumpTree.

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

The text was updated successfully, but these errors were encountered:

…ace stringstream by FmtBuffer (#4506) close #4494

…ace stringstream by FmtBuffer (pingcap#4506) close pingcap#4494

JaySon-Huang · 2022-06-29T04:00:20Z

Mark that this problem is more noticeable on the arm64 platform. This is because the efficiency of the IBlockInputStream::dumpTree function is O(n^2) where n is the number of CPU cores and arm64 platforms generally have more CPU cores than x86 platforms.

The performance of TPC-DS query 75 on arm64 platforms is greatly affected by this problem.

Query pipeline:
Union
 ExchangeSender × 64
  Expression
   Expression
    Expression
     SharedQuery
      Expression
       Expression
        ParallelAggregating
         Expression × 64
          Squashing
           TiRemoteBlockInputStream(ExchangeReceiver)

mysql> -- start query 75 in stream 0 using template query75.tpl
MySQL> explain analyze WITH all_sales 
     AS (SELECT d_year, 
                i_brand_id, 
                i_class_id, 
                i_category_id, 
                i_manufact_id, 
                Sum(sales_cnt) AS sales_cnt, 
                Sum(sales_amt) AS sales_amt 
         FROM   (SELECT d_year, 
                        i_brand_id, 
                        i_class_id, 
                        i_category_id, 
                        i_manufact_id, 
                        cs_quantity - COALESCE(cr_return_quantity, 0)        AS 
                        sales_cnt, 
                        cs_ext_sales_price - COALESCE(cr_return_amount, 0.0) AS 
                        sales_amt 
                 FROM   catalog_sales 
                        JOIN item 
                          ON i_item_sk = cs_item_sk 
                        JOIN date_dim 
                          ON d_date_sk = cs_sold_date_sk 
                        LEFT JOIN catalog_returns 
                               ON ( cs_order_number = cr_order_number 
                                    AND cs_item_sk = cr_item_sk ) 
                 WHERE  i_category = 'Men' 
                 UNION 
                 SELECT d_year, 
                        i_brand_id, 
                        i_class_id, 
                        i_category_id, 
                        i_manufact_id, 
                        ss_quantity - COALESCE(sr_return_quantity, 0)     AS 
                        sales_cnt, 
                        ss_ext_sales_price - COALESCE(sr_return_amt, 0.0) AS 
                        sales_amt 
                 FROM   store_sales 
                        JOIN item 
                          ON i_item_sk = ss_item_sk 
                        JOIN date_dim 
                          ON d_date_sk = ss_sold_date_sk 
                        LEFT JOIN store_returns 
                               ON ( ss_ticket_number = sr_ticket_number 
                                    AND ss_item_sk = sr_item_sk ) 
                 WHERE  i_category = 'Men' 
                 UNION 
                 SELECT d_year, 
                        i_brand_id, 
                        i_class_id, 
                        i_category_id, 
                        i_manufact_id, 
                        ws_quantity - COALESCE(wr_return_quantity, 0)     AS 
                        sales_cnt, 
                        ws_ext_sales_price - COALESCE(wr_return_amt, 0.0) AS 
                        sales_amt 
                 FROM   web_sales 
                        JOIN item 
                          ON i_item_sk = ws_item_sk 
                        JOIN date_dim 
                          ON d_date_sk = ws_sold_date_sk 
                        LEFT JOIN web_returns 
                               ON ( ws_order_number = wr_order_number 
                                    AND ws_item_sk = wr_item_sk ) 
                 WHERE  i_category = 'Men') sales_detail 
         GROUP  BY d_year, 
                   i_brand_id, 
                   i_class_id, 
                   i_category_id, 
                   i_manufact_id) 
SELECT prev_yr.d_year                        AS prev_year, 
               curr_yr.d_year                        AS year1, 
               curr_yr.i_brand_id, 
               curr_yr.i_class_id, 
               curr_yr.i_category_id, 
               curr_yr.i_manufact_id, 
               prev_yr.sales_cnt                     AS prev_yr_cnt, 
               curr_yr.sales_cnt                     AS curr_yr_cnt, 
               curr_yr.sales_cnt - prev_yr.sales_cnt AS sales_cnt_diff, 
               curr_yr.sales_amt - prev_yr.sales_amt AS sales_amt_diff 
FROM   all_sales curr_yr, 
       all_sales prev_yr 
WHERE  curr_yr.i_brand_id = prev_yr.i_brand_id 
       AND curr_yr.i_class_id = prev_yr.i_class_id 
       AND curr_yr.i_category_id = prev_yr.i_category_id 
       AND curr_yr.i_manufact_id = prev_yr.i_manufact_id 
       AND curr_yr.d_year = 2002 
       AND prev_yr.d_year = 2002 - 1 
       AND Cast(curr_yr.sales_cnt AS DECIMAL(17, 2)) / Cast(prev_yr.sales_cnt AS 
                                                                DECIMAL(17, 2)) 
           < 0.9 
ORDER  BY sales_cnt_diff
LIMIT 100; 
+-----------------------------------------------------------------------------------+--------------+-----------+--------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+
| id                                                                                | estRows      | actRows   | task         | access object         | execution info                                                                                                                                                     | operator info                                                                                                                                                                                                                                                                                                                                                                       | memory   | disk    |
+-----------------------------------------------------------------------------------+--------------+-----------+--------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+
| Projection_584                                                                    | 100.00       | 100       | root         |                       | time:2m3.2s, loops:2, Concurrency:OFF                                                                                                                              | Column#341, Column#334, Column#335, Column#336, Column#337, Column#338, Column#346, Column#339, minus(Column#339, Column#346)->Column#348, minus(Column#340, Column#347)->Column#349                                                                                                                                                                                                | 23.4 KB  | N/A     |
| └─Projection_595                                                                  | 100.00       | 100       | root         |                       | time:2m3.2s, loops:2, Concurrency:OFF                                                                                                                              | Column#334, Column#335, Column#336, Column#337, Column#338, Column#339, Column#340, Column#341, Column#346, Column#347                                                                                                                                                                                                                                                              | 27.5 KB  | N/A     |
|   └─TopN_587                                                                      | 100.00       | 100       | root         |                       | time:2m3.2s, loops:2                                                                                                                                               | Column#448, offset:0, count:100                                                                                                                                                                                                                                                                                                                                                     | 38.7 KB  | N/A     |
|     └─Projection_596                                                              | 2008823.87   | 113       | root         |                       | time:2m3.2s, loops:4, Concurrency:5                                                                                                                                | Column#334, Column#335, Column#336, Column#337, Column#338, Column#339, Column#340, Column#341, Column#346, Column#347, minus(Column#339, Column#346)->Column#448                                                                                                                                                                                                                   | 90.1 KB  | N/A     |
|       └─HashJoin_589                                                              | 2008823.87   | 113       | root         |                       | time:2m3.2s, loops:4, build_hash_table:{total:2m3.2s, fetch:2m3.2s, build:2.45ms}, probe:{concurrency:5, total:10m15.9s, max:2m3.2s, probe:29.5ms, fetch:10m15.9s} | inner join, equal:[eq(Column#335, Column#342) eq(Column#336, Column#343) eq(Column#337, Column#344) eq(Column#338, Column#345)], other cond:lt(div(cast(Column#339, decimal(17,2) BINARY), cast(Column#346, decimal(17,2) BINARY)), 0.9)                                                                                                                                            | 365.1 KB | 0 Bytes |
|         ├─Selection_593(Build)                                                    | 2008823.87   | 2664      | root         |                       | time:2m3.2s, loops:4                                                                                                                                               | eq(Column#341, 2001), not(isnull(Column#342)), not(isnull(Column#343)), not(isnull(Column#344)), not(isnull(Column#345))                                                                                                                                                                                                                                                            | 121.7 KB | N/A     |
|         │ └─CTEFullScan_594                                                       | 2511029.84   | 16011     | root         | CTE:prev_yr           | time:2m3.2s, loops:18                                                                                                                                              | data:CTE_0                                                                                                                                                                                                                                                                                                                                                                          | 0 Bytes  | 0 Bytes |
|         └─Selection_591(Probe)                                                    | 2008823.87   | 2752      | root         |                       | time:2m3.2s, loops:4                                                                                                                                               | eq(Column#334, 2002), not(isnull(Column#335)), not(isnull(Column#336)), not(isnull(Column#337)), not(isnull(Column#338))                                                                                                                                                                                                                                                            | 121.7 KB | N/A     |
|           └─CTEFullScan_592                                                       | 2511029.84   | 16011     | root         | CTE:curr_yr           | time:2m3.2s, loops:18                                                                                                                                              | data:CTE_0                                                                                                                                                                                                                                                                                                                                                                          | 3.72 MB  | 0 Bytes |
| CTE_0                                                                             | 2511029.84   | 16011     | root         |                       | time:2m3.2s, loops:18                                                                                                                                              | Non-Recursive CTE                                                                                                                                                                                                                                                                                                                                                                   | 0 Bytes  | 0 Bytes |
| └─Projection_60(Seed Part)                                                        | 2511029.84   | 16011     | root         |                       | time:2m3.2s, loops:17, Concurrency:5                                                                                                                               | Column#325, Column#326, Column#327, Column#328, Column#329, Column#332, Column#333                                                                                                                                                                                                                                                                                                  | 631.5 KB | N/A     |
|   └─TableReader_578                                                               | 2511029.84   | 16011     | root         |                       | time:2m3.2s, loops:17, cop_task: {num: 2, max: 0s, min: 0s, avg: 0s, p95: 0s, copr_cache_hit_ratio: 0.00}                                                          | data:ExchangeSender_577                                                                                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|     └─ExchangeSender_577                                                          | 2511029.84   | 16011     | cop[tiflash] |                       | tiflash_task:{time:2m3.1s, loops:1, threads:64}                                                                                                                    | ExchangeType: PassThrough                                                                                                                                                                                                                                                                                                                                                           | N/A      | N/A     |
|       └─Projection_573                                                            | 2511029.84   | 16011     | cop[tiflash] |                       | tiflash_task:{time:2m3.1s, loops:1, threads:64}                                                                                                                    | Column#332, Column#333, Column#325, Column#326, Column#327, Column#328, Column#329                                                                                                                                                                                                                                                                                                  | N/A      | N/A     |
|         └─HashAgg_574                                                             | 2511029.84   | 16011     | cop[tiflash] |                       | tiflash_task:{time:2m3.1s, loops:1, threads:1}                                                                                                                     | group by:Column#325, Column#326, Column#327, Column#328, Column#329, funcs:sum(Column#427)->Column#332, funcs:sum(Column#428)->Column#333, funcs:firstrow(Column#325)->Column#325, funcs:firstrow(Column#326)->Column#326, funcs:firstrow(Column#327)->Column#327, funcs:firstrow(Column#328)->Column#328, funcs:firstrow(Column#329)->Column#329                                   | N/A      | N/A     |
|           └─ExchangeReceiver_576                                                  | 2511029.84   | 16011     | cop[tiflash] |                       | tiflash_task:{time:2m3.1s, loops:40, threads:64}                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|             └─ExchangeSender_575                                                  | 2511029.84   | 16011     | cop[tiflash] |                       | tiflash_task:{time:2m3.1s, loops:256, threads:64}                                                                                                                  | ExchangeType: HashPartition, Hash Cols: [name: Column#325, collate: N/A], [name: Column#326, collate: N/A], [name: Column#327, collate: N/A], [name: Column#328, collate: N/A], [name: Column#329, collate: N/A]                                                                                                                                                                    | N/A      | N/A     |
|               └─HashAgg_64                                                        | 2511029.84   | 16011     | cop[tiflash] |                       | tiflash_task:{time:2m3.1s, loops:256, threads:1}                                                                                                                   | group by:Column#443, Column#444, Column#445, Column#446, Column#447, funcs:sum(Column#441)->Column#427, funcs:sum(Column#442)->Column#428                                                                                                                                                                                                                                           | N/A      | N/A     |
|                 └─Projection_582                                                  | 2511029.84   | 23751364  | cop[tiflash] |                       | tiflash_task:{time:2m2.9s, loops:256, threads:64}                                                                                                                  | cast(Column#330, decimal(41,0) BINARY)->Column#441, Column#331, Column#325, Column#326, Column#327, Column#328, Column#329                                                                                                                                                                                                                                                          | N/A      | N/A     |
|                   └─Projection_569                                                | 2511029.84   | 23751364  | cop[tiflash] |                       | tiflash_task:{time:2m2.7s, loops:256, threads:64}                                                                                                                  | Column#325, Column#326, Column#327, Column#328, Column#329, Column#330, Column#331                                                                                                                                                                                                                                                                                                  | N/A      | N/A     |
|                     └─HashAgg_570                                                 | 2511029.84   | 23751364  | cop[tiflash] |                       | tiflash_task:{time:2m3s, loops:256, threads:1}                                                                                                                     | group by:Column#325, Column#326, Column#327, Column#328, Column#329, Column#330, Column#331, funcs:firstrow(Column#325)->Column#325, funcs:firstrow(Column#326)->Column#326, funcs:firstrow(Column#327)->Column#327, funcs:firstrow(Column#328)->Column#328, funcs:firstrow(Column#329)->Column#329, funcs:firstrow(Column#330)->Column#330, funcs:firstrow(Column#331)->Column#331 | N/A      | N/A     |
|                       └─ExchangeReceiver_572                                      | 2511029.84   | 23808716  | cop[tiflash] |                       | tiflash_task:{time:1m59.1s, loops:768, threads:64}                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                         └─ExchangeSender_571                                      | 2511029.84   | 23808716  | cop[tiflash] |                       | tiflash_task:{time:1m57.2s, loops:768, threads:192}                                                                                                                | ExchangeType: HashPartition, Hash Cols: [name: Column#325, collate: N/A], [name: Column#326, collate: N/A], [name: Column#327, collate: N/A], [name: Column#328, collate: N/A], [name: Column#329, collate: N/A], [name: Column#330, collate: N/A], [name: Column#331, collate: N/A]                                                                                                | N/A      | N/A     |
|                           └─HashAgg_567                                           | 2511029.84   | 23808716  | cop[tiflash] |                       | tiflash_task:{time:1m57.2s, loops:768, threads:3}                                                                                                                  | group by:Column#325, Column#326, Column#327, Column#328, Column#329, Column#330, Column#331,                                                                                                                                                                                                                                                                                        | N/A      | N/A     |
|                             └─Union_531                                           | 24896473.89  | 0         | cop[tiflash] |                       | tiflash_task:{time:0s, loops:0, threads:0}                                                                                                                         |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               ├─Projection_517                                    | 7097077.92   | 7032994   | cop[tiflash] |                       | tiflash_task:{time:1m39.7s, loops:269, threads:64}                                                                                                                 | tpcds50.date_dim.d_year, tpcds50.item.i_brand_id, tpcds50.item.i_class_id, tpcds50.item.i_category_id, tpcds50.item.i_manufact_id, minus(tpcds50.catalog_sales.cs_quantity, coalesce(tpcds50.catalog_returns.cr_return_quantity, 0))->Column#330, minus(tpcds50.catalog_sales.cs_ext_sales_price, coalesce(tpcds50.catalog_returns.cr_return_amount, 0.0))->Column#331              | N/A      | N/A     |
|                               │ └─HashJoin_518                                    | 7097077.92   | 7032994   | cop[tiflash] |                       | tiflash_task:{time:1m39.4s, loops:269, threads:128}                                                                                                                | left outer join, equal:[eq(tpcds50.catalog_sales.cs_order_number, tpcds50.catalog_returns.cr_order_number) eq(tpcds50.catalog_sales.cs_item_sk, tpcds50.catalog_returns.cr_item_sk)]                                                                                                                                                                                                | N/A      | N/A     |
|                               │   ├─ExchangeReceiver_104(Build)                   | 7097077.92   | 7032994   | cop[tiflash] |                       | tiflash_task:{time:1m37.6s, loops:556, threads:64}                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │   │ └─ExchangeSender_103                          | 7097077.92   | 7032994   | cop[tiflash] |                       | tiflash_task:{time:5.53s, loops:1187, threads:64}                                                                                                                  | ExchangeType: HashPartition, Hash Cols: [name: tpcds50.catalog_sales.cs_order_number, collate: N/A], [name: tpcds50.catalog_sales.cs_item_sk, collate: N/A]                                                                                                                                                                                                                         | N/A      | N/A     |
|                               │   │   └─Projection_91                             | 7097077.92   | 7032994   | cop[tiflash] |                       | tiflash_task:{time:5.53s, loops:1187, threads:64}                                                                                                                  | tpcds50.catalog_sales.cs_item_sk, tpcds50.catalog_sales.cs_order_number, tpcds50.catalog_sales.cs_quantity, tpcds50.catalog_sales.cs_ext_sales_price, tpcds50.item.i_brand_id, tpcds50.item.i_class_id, tpcds50.item.i_category_id, tpcds50.item.i_manufact_id, tpcds50.date_dim.d_year                                                                                             | N/A      | N/A     |
|                               │   │     └─HashJoin_92                             | 7097077.92   | 7032994   | cop[tiflash] |                       | tiflash_task:{time:5.53s, loops:1187, threads:64}                                                                                                                  | inner join, equal:[eq(tpcds50.catalog_sales.cs_sold_date_sk, tpcds50.date_dim.d_date_sk)]                                                                                                                                                                                                                                                                                           | N/A      | N/A     |
|                               │   │       ├─ExchangeReceiver_102(Build)           | 73049.00     | 73049     | cop[tiflash] |                       | tiflash_task:{time:487.2ms, loops:2, threads:64}                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │   │       │ └─ExchangeSender_101                  | 73049.00     | 73049     | cop[tiflash] |                       | tiflash_task:{time:51.9ms, loops:2, threads:1}                                                                                                                     | ExchangeType: Broadcast                                                                                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|                               │   │       │   └─TableFullScan_100                 | 73049.00     | 73049     | cop[tiflash] | table:date_dim        | tiflash_task:{time:51.9ms, loops:2, threads:1}                                                                                                                     | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   │       └─HashJoin_93(Probe)                    | 7097077.92   | 7032994   | cop[tiflash] |                       | tiflash_task:{time:3.98s, loops:1187, threads:64}                                                                                                                  | inner join, equal:[eq(tpcds50.item.i_item_sk, tpcds50.catalog_sales.cs_item_sk)]                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   │         ├─ExchangeReceiver_97(Build)          | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:597.2ms, loops:1, threads:64}                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │   │         │ └─ExchangeSender_96                 | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:83.3ms, loops:1, threads:1}                                                                                                                     | ExchangeType: Broadcast                                                                                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|                               │   │         │   └─Selection_95                    | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:83.3ms, loops:1, threads:1}                                                                                                                     | eq(tpcds50.item.i_category, "Men")                                                                                                                                                                                                                                                                                                                                                  | N/A      | N/A     |
|                               │   │         │     └─TableFullScan_94              | 62000.00     | 62000     | cop[tiflash] | table:item            | tiflash_task:{time:73.3ms, loops:1, threads:1}                                                                                                                     | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   │         └─Selection_99(Probe)                 | 71664306.40  | 71637279  | cop[tiflash] |                       | tiflash_task:{time:1.57s, loops:1187, threads:64}                                                                                                                  | not(isnull(tpcds50.catalog_sales.cs_sold_date_sk))                                                                                                                                                                                                                                                                                                                                  | N/A      | N/A     |
|                               │   │           └─TableFullScan_98                  | 71997669.00  | 71997669  | cop[tiflash] | table:catalog_sales   | tiflash_task:{time:1.11s, loops:1187, threads:64}                                                                                                                  | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   └─ExchangeReceiver_107(Probe)                   | 7197954.00   | 7197954   | cop[tiflash] |                       | tiflash_task:{time:1m36.7s, loops:141, threads:64}                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │     └─ExchangeSender_106                          | 7197954.00   | 7197954   | cop[tiflash] |                       | tiflash_task:{time:565ms, loops:149, threads:20}                                                                                                                   | ExchangeType: HashPartition, Hash Cols: [name: tpcds50.catalog_returns.cr_order_number, collate: N/A], [name: tpcds50.catalog_returns.cr_item_sk, collate: N/A]                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │       └─TableFullScan_105                         | 7197954.00   | 7197954   | cop[tiflash] | table:catalog_returns | tiflash_task:{time:565ms, loops:149, threads:20}                                                                                                                   | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               ├─Projection_519                                    | 14195084.99  | 13507050  | cop[tiflash] |                       | tiflash_task:{time:1m54.7s, loops:464, threads:64}                                                                                                                 | tpcds50.date_dim.d_year, tpcds50.item.i_brand_id, tpcds50.item.i_class_id, tpcds50.item.i_category_id, tpcds50.item.i_manufact_id, minus(tpcds50.store_sales.ss_quantity, coalesce(tpcds50.store_returns.sr_return_quantity, 0))->Column#330, minus(tpcds50.store_sales.ss_ext_sales_price, coalesce(tpcds50.store_returns.sr_return_amt, 0.0))->Column#331                         | N/A      | N/A     |
|                               │ └─HashJoin_520                                    | 14195084.99  | 13507050  | cop[tiflash] |                       | tiflash_task:{time:1m52.8s, loops:464, threads:128}                                                                                                                | left outer join, equal:[eq(tpcds50.store_sales.ss_ticket_number, tpcds50.store_returns.sr_ticket_number) eq(tpcds50.store_sales.ss_item_sk, tpcds50.store_returns.sr_item_sk)]                                                                                                                                                                                                      | N/A      | N/A     |
|                               │   ├─ExchangeReceiver_252(Build)                   | 14195084.99  | 13507050  | cop[tiflash] |                       | tiflash_task:{time:1m49.2s, loops:1116, threads:64}                                                                                                                |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │   │ └─ExchangeSender_251                          | 14195084.99  | 13507050  | cop[tiflash] |                       | tiflash_task:{time:6.75s, loops:2347, threads:64}                                                                                                                  | ExchangeType: HashPartition, Hash Cols: [name: tpcds50.store_sales.ss_ticket_number, collate: N/A], [name: tpcds50.store_sales.ss_item_sk, collate: N/A]                                                                                                                                                                                                                            | N/A      | N/A     |
|                               │   │   └─Projection_239                            | 14195084.99  | 13507050  | cop[tiflash] |                       | tiflash_task:{time:6.75s, loops:2347, threads:64}                                                                                                                  | tpcds50.store_sales.ss_item_sk, tpcds50.store_sales.ss_ticket_number, tpcds50.store_sales.ss_quantity, tpcds50.store_sales.ss_ext_sales_price, tpcds50.item.i_brand_id, tpcds50.item.i_class_id, tpcds50.item.i_category_id, tpcds50.item.i_manufact_id, tpcds50.date_dim.d_year                                                                                                    | N/A      | N/A     |
|                               │   │     └─HashJoin_240                            | 14195084.99  | 13507050  | cop[tiflash] |                       | tiflash_task:{time:6.75s, loops:2347, threads:64}                                                                                                                  | inner join, equal:[eq(tpcds50.store_sales.ss_sold_date_sk, tpcds50.date_dim.d_date_sk)]                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|                               │   │       ├─ExchangeReceiver_250(Build)           | 73049.00     | 73049     | cop[tiflash] |                       | tiflash_task:{time:368.3ms, loops:2, threads:64}                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │   │       │ └─ExchangeSender_249                  | 73049.00     | 73049     | cop[tiflash] |                       | tiflash_task:{time:43.7ms, loops:2, threads:1}                                                                                                                     | ExchangeType: Broadcast                                                                                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|                               │   │       │   └─TableFullScan_248                 | 73049.00     | 73049     | cop[tiflash] | table:date_dim        | tiflash_task:{time:43.7ms, loops:2, threads:1}                                                                                                                     | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   │       └─HashJoin_241(Probe)                   | 14195084.99  | 13507050  | cop[tiflash] |                       | tiflash_task:{time:5.2s, loops:2347, threads:64}                                                                                                                   | inner join, equal:[eq(tpcds50.item.i_item_sk, tpcds50.store_sales.ss_item_sk)]                                                                                                                                                                                                                                                                                                      | N/A      | N/A     |
|                               │   │         ├─ExchangeReceiver_245(Build)         | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:318.3ms, loops:1, threads:64}                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │   │         │ └─ExchangeSender_244                | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:46.5ms, loops:1, threads:1}                                                                                                                     | ExchangeType: Broadcast                                                                                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|                               │   │         │   └─Selection_243                   | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:46.5ms, loops:1, threads:1}                                                                                                                     | eq(tpcds50.item.i_category, "Men")                                                                                                                                                                                                                                                                                                                                                  | N/A      | N/A     |
|                               │   │         │     └─TableFullScan_242             | 62000.00     | 62000     | cop[tiflash] | table:item            | tiflash_task:{time:46.5ms, loops:1, threads:1}                                                                                                                     | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   │         └─Selection_247(Probe)                | 137581328.73 | 137528107 | cop[tiflash] |                       | tiflash_task:{time:2.53s, loops:2347, threads:64}                                                                                                                  | not(isnull(tpcds50.store_sales.ss_sold_date_sk))                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   │           └─TableFullScan_246                 | 144004764.00 | 144004764 | cop[tiflash] | table:store_sales     | tiflash_task:{time:2s, loops:2347, threads:64}                                                                                                                     | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               │   └─ExchangeReceiver_255(Probe)                   | 14392680.00  | 14392680  | cop[tiflash] |                       | tiflash_task:{time:1m41.8s, loops:272, threads:64}                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                               │     └─ExchangeSender_254                          | 14392680.00  | 14392680  | cop[tiflash] |                       | tiflash_task:{time:530ms, loops:289, threads:34}                                                                                                                   | ExchangeType: HashPartition, Hash Cols: [name: tpcds50.store_returns.sr_ticket_number, collate: N/A], [name: tpcds50.store_returns.sr_item_sk, collate: N/A]                                                                                                                                                                                                                        | N/A      | N/A     |
|                               │       └─TableFullScan_253                         | 14392680.00  | 14392680  | cop[tiflash] | table:store_returns   | tiflash_task:{time:530ms, loops:289, threads:34}                                                                                                                   | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                               └─Projection_521                                    | 3604310.99   | 3533794   | cop[tiflash] |                       | tiflash_task:{time:1m43.2s, loops:134, threads:64}                                                                                                                 | tpcds50.date_dim.d_year, tpcds50.item.i_brand_id, tpcds50.item.i_class_id, tpcds50.item.i_category_id, tpcds50.item.i_manufact_id, minus(tpcds50.web_sales.ws_quantity, coalesce(tpcds50.web_returns.wr_return_quantity, 0))->Column#330, minus(tpcds50.web_sales.ws_ext_sales_price, coalesce(tpcds50.web_returns.wr_return_amt, 0.0))->Column#331                                 | N/A      | N/A     |
|                                 └─HashJoin_522                                    | 3604310.99   | 3533794   | cop[tiflash] |                       | tiflash_task:{time:1m42.4s, loops:134, threads:128}                                                                                                                | left outer join, equal:[eq(tpcds50.web_sales.ws_order_number, tpcds50.web_returns.wr_order_number) eq(tpcds50.web_sales.ws_item_sk, tpcds50.web_returns.wr_item_sk)]                                                                                                                                                                                                                | N/A      | N/A     |
|                                   ├─ExchangeReceiver_400(Build)                   | 3548343.34   | 3533794   | cop[tiflash] |                       | tiflash_task:{time:1m41.1s, loops:286, threads:64}                                                                                                                 |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                                   │ └─ExchangeSender_399                          | 3548343.34   | 3533794   | cop[tiflash] |                       | tiflash_task:{time:5.92s, loops:668, threads:64}                                                                                                                   | ExchangeType: HashPartition, Hash Cols: [name: tpcds50.web_sales.ws_order_number, collate: N/A], [name: tpcds50.web_sales.ws_item_sk, collate: N/A]                                                                                                                                                                                                                                 | N/A      | N/A     |
|                                   │   └─Projection_387                            | 3548343.34   | 3533794   | cop[tiflash] |                       | tiflash_task:{time:5.92s, loops:668, threads:64}                                                                                                                   | tpcds50.web_sales.ws_item_sk, tpcds50.web_sales.ws_order_number, tpcds50.web_sales.ws_quantity, tpcds50.web_sales.ws_ext_sales_price, tpcds50.item.i_brand_id, tpcds50.item.i_class_id, tpcds50.item.i_category_id, tpcds50.item.i_manufact_id, tpcds50.date_dim.d_year                                                                                                             | N/A      | N/A     |
|                                   │     └─HashJoin_388                            | 3548343.34   | 3533794   | cop[tiflash] |                       | tiflash_task:{time:5.92s, loops:668, threads:64}                                                                                                                   | inner join, equal:[eq(tpcds50.web_sales.ws_sold_date_sk, tpcds50.date_dim.d_date_sk)]                                                                                                                                                                                                                                                                                               | N/A      | N/A     |
|                                   │       ├─ExchangeReceiver_398(Build)           | 73049.00     | 73049     | cop[tiflash] |                       | tiflash_task:{time:310.6ms, loops:2, threads:64}                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                                   │       │ └─ExchangeSender_397                  | 73049.00     | 73049     | cop[tiflash] |                       | tiflash_task:{time:22ms, loops:2, threads:1}                                                                                                                       | ExchangeType: Broadcast                                                                                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|                                   │       │   └─TableFullScan_396                 | 73049.00     | 73049     | cop[tiflash] | table:date_dim        | tiflash_task:{time:22ms, loops:2, threads:1}                                                                                                                       | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                                   │       └─HashJoin_389(Probe)                   | 3548343.34   | 3533794   | cop[tiflash] |                       | tiflash_task:{time:4.32s, loops:668, threads:64}                                                                                                                   | inner join, equal:[eq(tpcds50.item.i_item_sk, tpcds50.web_sales.ws_item_sk)]                                                                                                                                                                                                                                                                                                        | N/A      | N/A     |
|                                   │         ├─ExchangeReceiver_393(Build)         | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:260.6ms, loops:1, threads:64}                                                                                                                   |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                                   │         │ └─ExchangeSender_392                | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:33.2ms, loops:1, threads:1}                                                                                                                     | ExchangeType: Broadcast                                                                                                                                                                                                                                                                                                                                                             | N/A      | N/A     |
|                                   │         │   └─Selection_391                   | 6151.00      | 6151      | cop[tiflash] |                       | tiflash_task:{time:33.2ms, loops:1, threads:1}                                                                                                                     | eq(tpcds50.item.i_category, "Men")                                                                                                                                                                                                                                                                                                                                                  | N/A      | N/A     |
|                                   │         │     └─TableFullScan_390             | 62000.00     | 62000     | cop[tiflash] | table:item            | tiflash_task:{time:33.2ms, loops:1, threads:1}                                                                                                                     | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                                   │         └─Selection_395(Probe)                | 35996850.00  | 35987985  | cop[tiflash] |                       | tiflash_task:{time:2.02s, loops:668, threads:64}                                                                                                                   | not(isnull(tpcds50.web_sales.ws_sold_date_sk))                                                                                                                                                                                                                                                                                                                                      | N/A      | N/A     |
|                                   │           └─TableFullScan_394                 | 35996850.00  | 35996850  | cop[tiflash] | table:web_sales       | tiflash_task:{time:1.49s, loops:668, threads:64}                                                                                                                   | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
|                                   └─ExchangeReceiver_403(Probe)                   | 3598924.00   | 3598924   | cop[tiflash] |                       | tiflash_task:{time:1m39.6s, loops:70, threads:64}                                                                                                                  |                                                                                                                                                                                                                                                                                                                                                                                     | N/A      | N/A     |
|                                     └─ExchangeSender_402                          | 3598924.00   | 3598924   | cop[tiflash] |                       | tiflash_task:{time:311.1ms, loops:72, threads:9}                                                                                                                   | ExchangeType: HashPartition, Hash Cols: [name: tpcds50.web_returns.wr_order_number, collate: N/A], [name: tpcds50.web_returns.wr_item_sk, collate: N/A]                                                                                                                                                                                                                             | N/A      | N/A     |
|                                       └─TableFullScan_401                         | 3598924.00   | 3598924   | cop[tiflash] | table:web_returns     | tiflash_task:{time:311.1ms, loops:72, threads:9}                                                                                                                   | keep order:false                                                                                                                                                                                                                                                                                                                                                                    | N/A      | N/A     |
+-----------------------------------------------------------------------------------+--------------+-----------+--------------+-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+
82 rows in set (2 min 3.27 sec)

windtalker added type/bug The issue is confirmed as a bug. type/enhancement The issue or PR belongs to an enhancement. and removed type/bug The issue is confirmed as a bug. labels Mar 30, 2022

windtalker mentioned this issue Mar 30, 2022

accelerate dumpTree by cache tree_id inside BlockInputStream #4497

Closed

12 tasks

SeaRise mentioned this issue Mar 30, 2022

accelerate dumpTree by cache tree_id inside BlockInputStream and replace stringstream by FmtBuffer #4506

Merged

12 tasks

ti-chi-bot closed this as completed in #4506 Mar 31, 2022

ti-chi-bot pushed a commit that referenced this issue Mar 31, 2022

accelerate dumpTree by cache tree_id inside BlockInputStream and repl…

4afdc08

…ace stringstream by FmtBuffer (#4506) close #4494

JaySon-Huang pushed a commit to JaySon-Huang/tiflash that referenced this issue Jun 28, 2022

accelerate dumpTree by cache tree_id inside BlockInputStream and repl…

62c93c1

…ace stringstream by FmtBuffer (pingcap#4506) close pingcap#4494

VelocityLight added the affects-5.4 label Feb 16, 2023

windtalker mentioned this issue Jul 17, 2023

Functions that traverse the IProfilingBlockInputStream tree may have performance issues when the tree contains SharedQueryBlockInputStream #7810

Closed

windtalker mentioned this issue Aug 5, 2024

auto pass through hashagg #9167

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`BlockInputStream->dumpTree()` is very slow if the input stream tree contains multiple `SharedQueryBlockInputStream` #4494

`BlockInputStream->dumpTree()` is very slow if the input stream tree contains multiple `SharedQueryBlockInputStream` #4494

windtalker commented Mar 30, 2022

JaySon-Huang commented Jun 29, 2022 •

edited

Loading

BlockInputStream->dumpTree() is very slow if the input stream tree contains multiple SharedQueryBlockInputStream #4494

BlockInputStream->dumpTree() is very slow if the input stream tree contains multiple SharedQueryBlockInputStream #4494

Comments

windtalker commented Mar 30, 2022

Bug Report

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

JaySon-Huang commented Jun 29, 2022 • edited Loading

`BlockInputStream->dumpTree()` is very slow if the input stream tree contains multiple `SharedQueryBlockInputStream` #4494

`BlockInputStream->dumpTree()` is very slow if the input stream tree contains multiple `SharedQueryBlockInputStream` #4494

JaySon-Huang commented Jun 29, 2022 •

edited

Loading