vegeta plot command #317

tsenart · 2018-07-13T16:57:02Z

This PR introduces a new vegeta plot command that is an evolution of the previous vegeta report -reporter=plot. Additionally, all plot related code has moved to its own package.

Features

Support for plotting very large result sets with bounded memory usage.
Downsampling via LTTB algorithm, used via a flag in the command.
Plotting multiple attacks overlaid in the same line chart.

Fixes #137

TODO

Golden test for plot output
Update Dygraphs library to latest version
Fork https://github.com/dgryski/go-tsz to use uint64 timestamps.
Research and implement improvements to Gorilla as per https://github.com/burmanm/gorilla-tsc.

Future work

Real-time plotting with a web sockets / SSE / HTTP2 Push + JS integration.

This commit introduces timeseries downsampling in the plot reporter with the aim to be performant even with very large result sets. Fixes #137

golangci · 2018-07-13T16:57:39Z

lib/reporters.go

+				points := make([]lttb.Point, 0, len(results[i]))
+				for _, r := range results[i] {
+					points = append(points, lttb.Point{
+						X: float64(r.Timestamp.Sub(results[i][0].Timestamp).Seconds()),


unnecessary conversion

golangci · 2018-07-13T16:57:39Z

lib/reporters.go

+				for _, r := range results[i] {
+					points = append(points, lttb.Point{
+						X: float64(r.Timestamp.Sub(results[i][0].Timestamp).Seconds()),
+						Y: float64(r.Latency.Seconds() * 1000),


unnecessary conversion

tsenart · 2018-07-13T17:08:07Z

Here's a visual representation of the effect of this change set. As for the performance effects, you can test by downloading the attached HTML files and opening each of them.

Plot Reports

Before

After

golangci · 2018-07-14T13:39:14Z

lib/plot_test.go

 	// Start benchmark
 	b.ReportAllocs()
 	b.StartTimer()
 	for i := 0; i < b.N; i++ {
-		rep.Report(ioutil.Discard)
+		NewHTMLPlot("Vegeta Plot", 5000, rs).WriteTo(ioutil.Discard)


Error return value of (github.com/tsenart/vegeta/lib.HTMLPlot).WriteTo is not checked

golangci · 2018-07-14T15:03:04Z

plot.go

+		fmt.Fprintln(os.Stderr, plotUsage)
+	}
+
+	return command{fs, func(args []string) error {


github.com/tsenart/vegeta.command composite literal uses unkeyed fields

golangci · 2018-07-14T15:03:04Z

plot.go

+	}
+
+	return command{fs, func(args []string) error {
+		fs.Parse(args)


Error return value of fs.Parse is not checked

This commit introduces a new plot command that was extracted from vegeta report so that it supports specific flags. Along with it, `NewPlotReporter` has been refactored into an `HTMLPlot` that knows how to `WriteTo` an `io.Writer`.

golangci · 2018-07-18T13:30:13Z

lib/plot_test.go

+	b.ReportAllocs()
+	b.StartTimer()
+	for i := 0; i < b.N; i++ {
+		NewHTMLPlot("Vegeta Plot", 5000, rs).WriteTo(ioutil.Discard)


Error return value of (github.com/tsenart/vegeta/lib.HTMLPlot).WriteTo is not checked

golangci · 2018-07-25T21:40:24Z

lib/plot.go

+	cw := countingWriter{w: w}
+	err = plotTemplate.Execute(&cw, &plotData{
+		Title:         p.title,
+		HTML2CanvasJS: template.JS(asset(html2canvas)),


G203: this method will not auto-escape HTML. Verify data is well formed.

golangci · 2018-07-25T21:40:24Z

lib/plot.go

+	err = plotTemplate.Execute(&cw, &plotData{
+		Title:         p.title,
+		HTML2CanvasJS: template.JS(asset(html2canvas)),
+		DygraphsJS:    template.JS(asset(dygraphs)),


G203: this method will not auto-escape HTML. Verify data is well formed.

golangci · 2018-07-25T21:40:25Z

lib/plot.go

+		Title:         p.title,
+		HTML2CanvasJS: template.JS(asset(html2canvas)),
+		DygraphsJS:    template.JS(asset(dygraphs)),
+		Data:          template.JS(data),


G203: this method will not auto-escape HTML. Verify data is well formed.

golangci · 2018-07-29T18:03:21Z

lib/lttb_test.go

+// From https://raw.githubusercontent.com/sveinn-steinarsson/flot-downsample/master/demo_data.js
+var points = [][]point{
+	{
+		{0, 29.357995947822218},


github.com/tsenart/vegeta/lib.point composite literal uses unkeyed fields

golangci · 2018-07-29T18:03:21Z

lib/lttb_test.go

+var points = [][]point{
+	{
+		{0, 29.357995947822218},
+		{1, 29.40932479606209},


github.com/tsenart/vegeta/lib.point composite literal uses unkeyed fields

golangci · 2018-07-29T18:03:21Z

lib/lttb_test.go

+	{
+		{0, 29.357995947822218},
+		{1, 29.40932479606209},
+		{2, 29.28168582006162},


github.com/tsenart/vegeta/lib.point composite literal uses unkeyed fields

golangci · 2018-07-29T18:03:22Z

lib/lttb_test.go

+
+	b.Run("tsenart", func(b *testing.B) {
+		for i := 0; i < b.N; i++ {
+			lttb(len(data), 1000, newIterator(points[0]))


Error return value of lttb is not checked

This commit makes timestamps and sequence numbers maintain the exact same ordering.

golangci · 2018-08-11T23:43:28Z

lib/lttb/lttb_test.go

+
+	b.Run("tsenart", func(b *testing.B) {
+		for i := 0; i < b.N; i++ {
+			Downsample(len(data), 1000, newIterator(points[0]))


Error return value of Downsample is not checked

report: Downsample time series in plot reporter

c8e7de8

This commit introduces timeseries downsampling in the plot reporter with the aim to be performant even with very large result sets. Fixes #137

tsenart requested a review from xla as a code owner July 13, 2018 16:57

golangci reviewed Jul 13, 2018

View reviewed changes

tsenart added 2 commits July 13, 2018 18:58

vendor: Add github.com/dgryski/go-lttb

40f7924

reporters: Elide unecessary type conversions

cf9f116

reporters: Fix incorrect allocation optimisation

435185e

tsenart added the WIP label Jul 14, 2018

golangci reviewed Jul 14, 2018

View reviewed changes

tsenart force-pushed the plot-reporter-downsampling branch from ab4f5e2 to 0aa7bbf Compare July 14, 2018 15:02

golangci reviewed Jul 14, 2018

View reviewed changes

tsenart force-pushed the plot-reporter-downsampling branch from 0aa7bbf to d1de3bb Compare July 14, 2018 15:03

plot: Introduce vegeta plot command

0d2e7f7

This commit introduces a new plot command that was extracted from vegeta report so that it supports specific flags. Along with it, `NewPlotReporter` has been refactored into an `HTMLPlot` that knows how to `WriteTo` an `io.Writer`.

tsenart force-pushed the plot-reporter-downsampling branch from d1de3bb to 0d2e7f7 Compare July 14, 2018 15:46

golangci reviewed Jul 18, 2018

View reviewed changes

plot: Highcharts and streaming refactoring WIP

e1a4f38

tsenart force-pushed the plot-reporter-downsampling branch from 4a4e7ba to e1a4f38 Compare July 18, 2018 20:59

tsenart added 3 commits July 19, 2018 16:27

git: Add *.gob and *.lz to .gitignore

d21bc64

plot: Revert back to Dygraphs

f9995df

lttb: Implement on-line version of the LTTB algorithm

15bec40

golangci reviewed Jul 25, 2018

View reviewed changes

golangci reviewed Jul 29, 2018

View reviewed changes

tsenart force-pushed the plot-reporter-downsampling branch 2 times, most recently from 28b2dc3 to b313306 Compare July 30, 2018 08:41

lttb: Fix LTTB bucketing and add better tests

05d47cf

tsenart force-pushed the plot-reporter-downsampling branch from b313306 to 05d47cf Compare August 11, 2018 16:21

tsenart added 4 commits August 11, 2018 22:29

lttb: Move to independent package

4be48a3

attack: Same order of timestamps and seq numbers

b274d1f

This commit makes timestamps and sequence numbers maintain the exact same ordering.

plot: Buffering logic

bcddd8a

lttb: Checkin

6b1a62d

golangci reviewed Aug 11, 2018

View reviewed changes

tsenart added 7 commits August 12, 2018 15:19

lttb: Improve Iter comment

cbb1581

plot: Rename attackSeries to labeledSeries

f3fb801

plot: Ensure timestamp monotonicity

eacb7a8

plot: HTMLPlotOpts

f63b43a

plot: Move to independent package

c9b9ae4

plot: Rename BenchmarkHTMLPlot

48ba0db

plot: TestLabeledSeries

df544bc

tsenart changed the title ~~report: Downsample time series in plot reporter~~ vegeta plot Aug 12, 2018

tsenart changed the title ~~vegeta plot~~ vegeta plot command Aug 12, 2018

tsenart added 12 commits August 13, 2018 15:58

plot: Golden test for plot output

24703ba

plot: Update Dygraphs and HTML2Canvas

ca99fe5

plot: Use github.com/tsenart/go-tsz

e19fc14

Update .gitignore

0992b8e

targets: Simplify generation code

449e551

plot: Update README and add examples

8a24986

plot: Improve TestLabeledSeries

52b6595

README: Update Plot image url

0754549

plot: Update golden file

b294ddb

plot: Deterministic WriteTo output

dd2ea0d

Merge branch 'master' into plot-reporter-downsampling

65989a8

plot: Add missing assets

055ddc6

tsenart added enhancement and removed WIP labels Aug 14, 2018

Update deps

9388665

tsenart removed the request for review from xla August 14, 2018 23:31

tsenart merged commit f477673 into master Aug 14, 2018

tsenart deleted the plot-reporter-downsampling branch August 14, 2018 23:38

tsenart mentioned this pull request Aug 18, 2018

Use go generate for packaging dygraphs #83

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vegeta plot command #317

vegeta plot command #317

tsenart commented Jul 13, 2018 •

edited

Loading

golangci Jul 13, 2018

golangci Jul 13, 2018

tsenart commented Jul 13, 2018

golangci Jul 14, 2018

golangci Jul 14, 2018

golangci Jul 14, 2018

golangci Jul 18, 2018

golangci Jul 25, 2018

golangci Jul 25, 2018

golangci Jul 25, 2018

golangci Jul 29, 2018

golangci Jul 29, 2018

golangci Jul 29, 2018

golangci Jul 29, 2018

golangci Aug 11, 2018

vegeta plot command #317

vegeta plot command #317

Conversation

tsenart commented Jul 13, 2018 • edited Loading

Features

TODO

Future work

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsenart commented Jul 13, 2018

Before

After

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tsenart commented Jul 13, 2018 •

edited

Loading