Skip to content

Commit 60b24f6

Browse files
nnethercotennethercote
committed
Add a brief description of each benchmark.
Fixes #391.
1 parent c96fe7c commit 60b24f6

File tree

3 files changed

+99
-2
lines changed

3 files changed

+99
-2
lines changed

‎README.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ The primary required setup is to provide a folder with a `retries` file and a `t
77
is gathered into https://github.com/rust-lang-nursery/rustc-timing by Rust Infrastructure; cloning
88
that is the best approach for working on the frontend.
99

10-
Additional documentation on running and setting up the frontend and backend can be found in README
11-
files in the `collector` and `site` directories.
10+
Additional documentation on running and setting up the frontend and backend can
11+
be found in the `README` files in the `collector` and `site` directories.
1212

13+
Additional documentation on the benchmark programs can be found in the `README`
14+
file in the `collector/benchmarks` directory.

‎collector/README.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
# Rust Compiler Performance Benchmarking and Profiling
22

3+
## The benchmarks
4+
5+
The individual benchmarks are described in the `README` file in the
6+
`collector/benchmarks` directory.
7+
38
## How to build
49

510
Before doing anything else, you should build `collector` (for running the

‎collector/benchmarks/README.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# The Benchmark Suite
2+
3+
This file describes the programs in the benchmark suite and explains why they
4+
were included.
5+
6+
## Real programs that are important
7+
8+
These are real programs that are important in some way, and worth tracking.
9+
10+
- **cargo**: The Rust package manager. An important program in the Rust
11+
ecosystem.
12+
- **clap-rs**: A command line argument parser. A crate used by many Rust
13+
programs.
14+
- **cranelift-codegen**: The largest crate from a code generator. Used by
15+
Firefox.
16+
- **crates.io**: Source code for `crates.io`. An important site in the
17+
Rust ecosystem.
18+
- **futures**: A futures implementation. Used by many Rust programs.
19+
- **helloworld**: A trivial program. Gives a lower bound on compile time.
20+
- **hyper**: An HTTP library. An important part of the Rust ecosystem.
21+
- **piston-image**: A modular game engine. An interesting Rust program.
22+
- **regex**: A regular expression parser. Used by many Rust programs.
23+
- **ripgrep**: A line-oriented search tool. A widely-used utility.
24+
- **script-servo**: Servo's `script` crate. A particularly large crate.
25+
- **serde**: A serialization/deserialization crate. Used by many other
26+
Rust programs.
27+
- **style-servo**: Servo's `style` crate. A large crate, and one used by
28+
Firefox.
29+
- **syn**: A library for parsing Rust code. An important part of the Rust
30+
ecosystem.
31+
- **tokio-webpush-simple**: A simple web server built with tokio. Uses futures
32+
a lot.
33+
- **webrender**: A web renderer. Used by Firefox and Servo.
34+
35+
## Real programs that stress the compiler
36+
37+
These are real programs that are known to stress the compiler in interesting
38+
ways.
39+
40+
- **encoding**: Character encoding support. Contains some large tables.
41+
- **html5ever**: An HTML parser. Stresses macro parsing code significantly.
42+
- **inflate**: An old implementation of the DEFLATE algorithm. Stresses the
43+
compiler in certain ways.
44+
- **keccak**: A cryptography algorithm. Contains a very high number of locals
45+
and basic blocks.
46+
- **packed-simd**: Portable packed SIMD vectors. Exercises SIMD features
47+
heavily, and uses a compiler plugin.
48+
- **sentry-cli**: A command line utility.
49+
[Reported](https://github.com/rust-lang-nursery/rustc-perf/issues/232#issuecomment-391439420)
50+
as being slow to compile.
51+
- **ucd**: A Unicode crate. Contains large statics that
52+
[stress](https://github.com/rust-lang/rust/issues/53643) the borrow checker's
53+
implementation of NLL.
54+
- **unicode_normalization**: Unicode character composition and decomposition
55+
utilities. Uses huge `match` statements that stress the compiler in unusual
56+
ways.
57+
- **wg-grammar**: A parser generator.
58+
[Stresses](https://github.com/rust-lang/rust/issues/58178) the borrow
59+
checker's implementation of NLL.
60+
61+
## Artificial stress tests
62+
63+
These are artificial programs that stress one particular aspect of the
64+
compiler. Some are entirely artificial, and some are extracted from real
65+
programs.
66+
67+
- **coercions**: Contains a static array with 65,536 string literals, which
68+
caused [poor performance](https://github.com/rust-lang/rust/issues/32278) in
69+
the past.
70+
- **ctfe-stress-2**: A stress test for compile-time function evaluation.
71+
- **deeply-nested**: A small program that caused [exponential
72+
behavior](https://github.com/rust-lang/rust/issues/38528) in the past.
73+
- **deep-vector**: A test containing a single large vector of zeroes, which
74+
caused [poor performance](https://github.com/rust-lang/rust/issues/20936) in
75+
the past.
76+
- **issue-46449**: A small program that caused [poor
77+
performance](https://github.com/rust-lang/rust/issues/46449) in the past.
78+
- **regression-31157**: A small program that caused a [large performance
79+
regression](https://github.com/rust-lang/rust/issues/31157) from the past.
80+
- **tuple-stress**: Contains a single array of 65,535 nested `(i32, (f64, f64,
81+
f64))` tuples. The data was extracted and reduced from a [program dealing
82+
with grid coordinates](https://github.com/urschrei/ostn15_phf) that was
83+
causing rustc to [run out of
84+
memory](https://github.com/rust-lang/rust/issues/36799).
85+
- **unify-linearly**: Contains many variables that all have equality relations
86+
between them, which caused [exponential
87+
behavior](https://github.com/rust-lang/rust/pull/32062) in the past.
88+
- **unused-warnings**: Contains many unused imports, which caused [quadratic
89+
behavior](https://github.com/rust-lang/rust/issues/43572) in the past.
90+

0 commit comments

Comments
 (0)