Performance comparison of multisearch and date range bucket aggregation

[kaituo]

Performance comparison of multisearch and date range bucket aggregation
Currently, we use multisearch in SearchFeatureDao

[kaituo]

running on two benchmarks with multisearch and time range query. date_range query is 2~10 times faster than multisearch. Every time I ran a query, I restarted ec2 nodes to invalidate OS and shard cache.

date_range query:

POST nab_art_daily_jumpsup/_search
 {
   "size": 0,
   "aggs": {
     "range": {
       "date_range": { 
         "field": "timestamp",
         "format": "epoch_millis",
         "ranges": [
           {"from" : 1583391275000, "to": 1583391575000},
           {"from" : 1583392475000, "to": 1583392775000},
           ...
        },
       "aggs": {
         "avg_value": {
           "sum": {
             "field": "value"
           }
         }
       }
     }
   }
 } 

multisearch query:

GET /nab_art_daily_jumpsup/_msearch?pretty
{"index" : "nab_art_daily_jumpsup"}
{"query":{"bool":{"must":[{"range":{"timestamp":{"from":1583391275000,"to":1583391575000,"include_lower":true,"include_upper":true,"boost":1}}},{"match_all":{"boost":1}}],"adjust_pure_negative":true,"boost":1}},"aggregations":{"7isajm0Bs-gslJuCOan7":{"sum":{"field":"value"}}}}
{"index" : "nab_art_daily_jumpsup"}
...

Environment:

• single node cluster
• cluster is not loaded (no other workload besides my queries)

Two benchmarks:

1. nab_art_daily_jumpsup, sparse data, every 5 minutes one matching document. So each multisearch and date_time query match one doc. I did 3 runs. The following is the recorded time (millisecond)

multisearch	date_time
844	75
973	70
1040	59

2. server metrics, dense data, every 5 minutes multiple matching documents (e.g., 105). So each multisearch and date_time query match multiple docs. I did 3 runs. The following is the recorded time (millisecond)

multisearch	date_time
663	205
716	181
439	224