Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

benchmark: sysbench 2.0 vs 1.0 #699

Merged
merged 4 commits into from
Apr 27, 2018
Merged

benchmark: sysbench 2.0 vs 1.0 #699

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e9cf47f
benchmark: sysbench 2.0 vs 1.0
coocood Apr 26, 2018
3fa6c69
Update sysbench-v2.md
CaitinChen Apr 26, 2018
9096440
*: remove 2048 threads
coocood Apr 26, 2018
8a37beb
*: refinement
coocood Apr 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 133 additions & 0 deletions benchmark/sysbench-v2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
title: TiDB Sysbench 性能对比测试报告 - v2.0.0 对比 v1.0.0
category: benchmark
---

# TiDB Sysbench 性能对比测试报告 - v2.0.0 对比 v1.0.0

## 测试目的
对比 TiDB 2.0 版本和 1.0 版本在 OLTP 场景下的性能。

## 测试版本、时间、地点

TiDB 版本：v1.0.8 Vs v2.0.0-rc6
时间：2018 年 4 月
地点：北京

## 测试环境

IDC机器

| 类别 | 名称 |
| :--------: | :---------: |
| OS | linux (CentOS 7.3.1611) |
| CPU | 40 vCPUs, Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz |
| RAM | 128GB |
| DISK | Optane 500GB SSD * 1 |

Sysbench 测试脚本：
https://github.com/pingcap/tidb-bench/tree/master/sysbench


## 测试方案

### TiDB 版本信息

### v1.0.8

| 组件 | GitHash |
| :--------: | :---------: |
| TiDB | 571f0bbd28a0b8155a5ee831992c986b90d21ab7 |
| TiKV | 4ef5889947019e3cb55cc744f487aa63b42540e7 |
| PD | 776bcd940b71d295a2c7ed762582bc3aff7d3c0e |

### v2.0.0-rc6

| 组件 | GitHash |
| :--------: | :---------: |
| TiDB | 82d35f1b7f9047c478f4e1e82aa0002abc8107e7 |
| TiKV | 7ed4f6a91f92cad5cd5323aaebe7d9f04b77cc79 |
| PD | 2c8e7d7e33b38e457169ce5dfb2f461fced82d65 |

### TiKV 参数配置


* v1.0.8
```
sync-log = false
grpc-concurrency = 8
grpc-raft-conn-num = 24
```

* v2.0.0-rc6
```
sync-log = false
grpc-concurrency = 8
grpc-raft-conn-num = 24
use-delete-range: false
```

### 集群拓扑

| 机器 IP | 部署实例 |
|--------------|------------|
| 172.16.21.1 | 1*tidb 1*pd 1*sysbench |
| 172.16.21.2 | 1*tidb 1*pd 1*sysbench |
| 172.16.21.3 | 1*tidb 1*pd 1*sysbench |
| 172.16.11.4 | 1*tikv |
| 172.16.11.5 | 1*tikv |
| 172.16.11.6 | 1*tikv |
| 172.16.11.7 | 1*tikv |
| 172.16.11.8 | 1*tikv |
| 172.16.11.9 | 1*tikv |

## 测试结果

### 标准 Select 测试

| 版本 | table count | table size | sysbench threads |qps | latency(avg / .95) |
| :---: | :---: | :---: | :---: | :---: | :---: |
| v2.0.0-rc6 | 32 | 1000 万 | 128 * 3 | 201936 | 1.9033 ms / 5.67667 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 256 * 3 | 208130 | 3.69333 ms / 8.90333 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 512 * 3 | 211788 | 7.23333 ms / 15.59 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 1024 * 3 | 212868 | 14.5933 ms / 43.2133 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 2048 * 3 | 212785 | 28.9267 ms / 67.7333 ms |
| v1.0.8 | 32 | 1000 万 | 128 * 3 | 188686 | 2.03667 ms / 5.99 ms |
| v1.0.8 | 32 | 1000 万 | 256 * 3 | 195090 |3.94 ms / 9.12 ms |
| v1.0.8 | 32 | 1000 万 | 512 * 3 | 203012 | 7.57333 ms / 15.3733 ms |
| v1.0.8 | 32 | 1000 万 | 1024 * 3 | 205932 | 14.9267 ms / 40.7633 ms |

GA2.0比GA1.0 在 select 查询性能上，最高提升了 10% 左右。


### 标准 OLTP 测试

| 版本 | table count | table size | sysbench threads | tps | qps | latency(avg / .95) |
| :---: | :---: | :---: | :---: | :---: | :---: | :---:|
| v2.0.0-rc6 | 32 | 1000 万 | 128 * 3 | 5404.22 | 108084.4 | 87.2033 ms / 110 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 256 * 3 | 5578.165 | 111563.3 | 167.673 ms / 275.623 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 512 * 3 | 5874.045 | 117480.9 | 315.083 ms / 674.017 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 1024 * 3 | 6290.7 | 125814 | 529.183 ms / 857.007 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 2048 * 3 | 6212.55 | 124251 | 981.143 ms / 1328.05 ms |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这一个删掉吧，并发有点大

| v1.0.8 | 32 | 1000 万 | 128 * 3 | 5523.91 | 110478 | 69.53 ms / 88.6333 ms |
| v1.0.8 | 32 | 1000 万 | 256 * 3 | 5969.43 | 119389 |128.63 ms / 162.58 ms |
| v1.0.8 | 32 | 1000 万 | 512 * 3 | 6308.93 | 126179 | 243.543 ms / 310.913 ms |
| v1.0.8 | 32 | 1000 万 | 1024 * 3 | 6444.25 | 128885 | 476.787ms / 635.143 ms |

GA2.0比GA1.0 在 oltp 性能上，性能基本一致。

### 标准 Insert 测试

| 版本 | table count | table size | sysbench threads |qps | latency(avg / .95) |
| :---: | :---: | :---: | :---: | :---: | :---: |
| v2.0.0-rc6 | 32 | 1000 万 | 128 * 3 | 31707.5 | 12.11 ms / 21.1167 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 256 * 3 | 38741.2 | 19.8233 ms / 39.65 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 512 * 3 | 45136.8 | 34.0267 ms / 66.84 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 1024 * 3 | 48667 | 63.1167 ms / 121.08 ms |
| v2.0.0-rc6 | 32 | 1000 万 | 2048 * 3 | 49228 | 124.777 ms / 182.12 ms |
| v1.0.8 | 32 | 1000 万 | 128 * 3 | 31125.7 | 12.3367 ms / 19.89 ms |
| v1.0.8 | 32 | 1000 万 | 256 * 3 | 36800 | 20.8667 ms / 35.3767 ms |
| v1.0.8 | 32 | 1000 万 | 512 * 3 | 44123 | 34.8067 ms / 63.32 ms |
| v1.0.8 | 32 | 1000 万 | 1024 * 3 | 48496 | 63.3333 ms / 118.92 ms |

GA2.0比GA1.0 在 insert 性能上，最高提升了5%。