Skip to content

Commit

Permalink
Created standard bench interface
Browse files Browse the repository at this point in the history
  • Loading branch information
Lucas Wojciechowski committed Mar 8, 2016
1 parent d654ee3 commit dac9f84
Show file tree
Hide file tree
Showing 11 changed files with 375 additions and 251 deletions.
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@
/_site
/dist/mapbox-gl-dev.js
/dist/mapbox-gl.js
/dist/mapbox-gl.js.map
*.js.map
/node_modules
npm-debug.log
*.sublime-*
coverage
.DS_Store
/bench/buffer/buffer_benchmark.js.map
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,10 @@ There are two test suites associated with Mapbox GL JS
- `npm test` runs quick unit tests
- `npm run test-suite` runs slower rendering tests from the [mapbox-gl-test-suite](https://github.com/mapbox/mapbox-gl-test-suite) repository

### Running Benchmarks

See [`bench/README.md`](https://github.com/mapbox/mapbox-gl-js/blob/master/bench/README.md).

### Running the FPS Benchmark

The FPS benchmarking page compares the performance of your local copy of GL JS against previously released versions. Benchmarking configuration is within `bench/fps/site.js`.
Expand All @@ -103,7 +107,7 @@ $ open "http://localhost:9966/bench/fps/?access_token="`echo $MapboxAccessToken`

### Writing Documentation

See [docs/README.md](https://github.com/mapbox/mapbox-gl-js/blob/master/docs/README.md).
See [`docs/README.md`](https://github.com/mapbox/mapbox-gl-js/blob/master/docs/README.md).

### Recommended Reading

Expand Down
86 changes: 86 additions & 0 deletions bench/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Benchmarks

Benchmarks help us catch performance regressions and improve performance.

## Running a Benchmark

Start the benchmark server

```bash
npm run start-bench
```

Open the benchmark runner

- [buffer benchmark](http://localhost:6699/bench/?benchmark=buffer)

## Writing a Benchmark

Good benchmarks

- are precise (i.e. running the test multiple times returns roughly the same result)
- operate in a way that mimics real-world usage
- use large, diverse, real-world data
- are conceptually simple

Benchmarks are implemented as a function that returns an instance of `Evented`.

```js
runBenchmark(options: { accessToken: string; }): Evented
```

The instance of `Evented` may fire any number of `log` and `error` events. The
instance of `Evented` must fire exactly one `end` event.

### `log`

Fire the `log` event to report benchmark progress to the user.

```js
{
message: string;
color: string = 'blue'; // name of a Mapbox base color https://mapbox.com/base/styling/color
}
```

If your benchmark has a notion of running multiple "samples", you might emit
one `log` event per sample.

```js
benchmark.fire('log', {
message: 'Finished sample ' + i + ' in ' + formatNumber(time) + ' ms'
});
```

These events have no machine-semantic meaning.

### `end`

Fire the `end` event to indicate the benchmark has finished running and report
its results.

These events have both human-readable results (`message`) and machine-readable results (`score`). Smaller `score`s are "better."

```js
{
message: string;
score: number;
}
```

```js
benchmark.fire('end', {
message: 'Average time is ' + formatNumber(averageTime)) + 'ms',
score: averageTime
});
```

### `error`

Fire the `error` event to indicate the benchmark has encountered a fatal error.

```js
{
error: Error;
}
```
61 changes: 0 additions & 61 deletions bench/buffer/site.js

This file was deleted.

Loading

0 comments on commit dac9f84

Please sign in to comment.