Skip to content

Commit

Permalink
Merge #32704
Browse files Browse the repository at this point in the history
32704: workload/ycsb: add flag to use column families r=nvanbenschoten a=nvanbenschoten

This change adds a new `--families` flag to the ycsb workload. Now
that #18168 is addressed, this significantly reduces the contention
present in the workload by avoiding conflicts on updates to different
columns in the same table.

I just confirmed that this still provides a huge speedup. On a 24 cpu machine:
```
workload run ycsb --init --workload='A' --concurrency=128 --duration=1m


--families=false gives:

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
   60.0s        0         103089         1718.0     13.9      0.6      1.8    604.0   1476.4  read
   60.0s        0         102947         1715.6     59.5      5.2     11.5   2281.7   8321.5  update

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__result
   60.0s        0         206036         3433.6     36.7      3.3      8.9   1342.2   8321.5


--families=true gives:

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__total
   60.0s        0         333477         5557.8      9.2      0.6      6.0    302.0   1275.1  read
   60.0s        0         332366         5539.3     13.7      6.8     17.8     54.5   4831.8  update

_elapsed___errors_____ops(total)___ops/sec(cum)__avg(ms)__p50(ms)__p95(ms)__p99(ms)_pMax(ms)__result
   60.0s        0         665843        11097.1     11.5      3.9     16.3    268.4   4831.8
```

cc. @robert-s-lee @drewdeally 

Release note: None

Co-authored-by: Nathan VanBenschoten <nvanbenschoten@gmail.com>
  • Loading branch information
craig[bot] and nvanbenschoten committed Nov 29, 2018
2 parents 0b95ad3 + 16ffee3 commit d033b2a
Showing 1 changed file with 31 additions and 1 deletion.
32 changes: 31 additions & 1 deletion pkg/workload/ycsb/ycsb.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,30 @@ const (
FIELD8 TEXT,
FIELD9 TEXT
)`
usertableSchemaRelationalWithFamilies = `(
ycsb_key VARCHAR(255) PRIMARY KEY NOT NULL,
FIELD0 TEXT,
FIELD1 TEXT,
FIELD2 TEXT,
FIELD3 TEXT,
FIELD4 TEXT,
FIELD5 TEXT,
FIELD6 TEXT,
FIELD7 TEXT,
FIELD8 TEXT,
FIELD9 TEXT,
FAMILY (ycsb_key),
FAMILY (FIELD0),
FAMILY (FIELD1),
FAMILY (FIELD2),
FAMILY (FIELD3),
FAMILY (FIELD4),
FAMILY (FIELD5),
FAMILY (FIELD6),
FAMILY (FIELD7),
FAMILY (FIELD8),
FAMILY (FIELD9)
)`
usertableSchemaJSON = `(
ycsb_key VARCHAR(255) PRIMARY KEY NOT NULL,
FIELD JSONB
Expand All @@ -64,6 +88,7 @@ type ycsb struct {
seed int64
initialRows int
json bool
families bool
splits int

workload string
Expand All @@ -89,6 +114,7 @@ var ycsbMeta = workload.Meta{
g.flags.IntVar(&g.initialRows, `initial-rows`, 10000,
`Initial number of rows to sequentially insert before beginning Zipfian workload`)
g.flags.BoolVar(&g.json, `json`, false, `Use JSONB rather than relational data`)
g.flags.BoolVar(&g.families, `families`, true, `Place each column in its own column family`)
g.flags.IntVar(&g.splits, `splits`, 0, `Number of splits to perform before starting normal operations`)
g.flags.StringVar(&g.workload, `workload`, `B`, `Workload type. Choose from A-F.`)
g.flags.StringVar(&g.distribution, `request-distribution`, `zipfian`, `Distribution for random number generator [zipfian, uniform].`)
Expand Down Expand Up @@ -169,7 +195,11 @@ func (g *ycsb) Tables() []workload.Table {
if g.json {
usertable.Schema = usertableSchemaJSON
} else {
usertable.Schema = usertableSchemaRelational
if g.families {
usertable.Schema = usertableSchemaRelationalWithFamilies
} else {
usertable.Schema = usertableSchemaRelational
}
}
return []workload.Table{usertable}
}
Expand Down

0 comments on commit d033b2a

Please sign in to comment.