Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TEST_BENCH does nothing #21324

Closed
brson opened this issue Jan 17, 2015 · 4 comments · Fixed by #21399
Closed

TEST_BENCH does nothing #21324

brson opened this issue Jan 17, 2015 · 4 comments · Fixed by #21399
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue.

Comments

@brson
Copy link
Contributor

brson commented Jan 17, 2015

make tips says that TEST_BENCH=1 will run benchmarks but it is not true. Are we ever running benchmarks?
@brson brson added E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. A-build labels Jan 17, 2015
@kmcallister
Copy link
Contributor

Isn't it PLEASE_BENCH=1 now?

@estsauver
Copy link
Contributor

It looks to me like neither the TEST_BENCH nor the PLEASE_BENCH commands are actually used anywhere. If the PLEASE_BENCH flag is set then the TEST_BENCH flag is set to --bench, but I don't know if those flags are used anywhere else. My grepping results are below.

Earls-MacBook-Pro:rust earljstsauver$ ag TEST_BENCH -C 3
mk/tests.mk
38-  TESTARGS += --ignored
39-endif
40-
41:TEST_BENCH =
42-
43-# Arguments to the cfail/rfail/rpass/bench tests
44-ifdef CFG_VALGRIND
45-  CTEST_RUNTOOL = --runtool "$(CFG_VALGRIND)"
46:  TEST_BENCH =
47-endif
48-
49-ifdef PLEASE_BENCH
50:  TEST_BENCH = --bench
51-endif
52-
53-# Arguments to the perf tests
Earls-MacBook-Pro:rust earljstsauver$ ag PLEASE_BENCH -C 3
Makefile.in
69-#
70-#   * `TESTNAME=...` - Specify the name of tests to run
71-#   * `CHECK_IGNORED=1` - Run normally-ignored tests
72:#   * `PLEASE_BENCH=1` - Run crate benchmarks (enable `--bench` flag)
73-#
74-#   * `CFG_ENABLE_VALGRIND=1` - Run tests under valgrind
75-#   * `VALGRIND_COMPILE=1` - Run the compiler itself under valgrind

mk/tests.mk
46-  TEST_BENCH =
47-endif
48-
49:ifdef PLEASE_BENCH
50-  TEST_BENCH = --bench
51-endif
52-

@kmcallister kmcallister added the A-testsuite Area: The testsuite used to check the correctness of rustc label Jan 17, 2015
@nagisa
Copy link
Member

nagisa commented Jan 18, 2015

I used PLEASE_BENCH successfully not too long ago.

@lilyball
Copy link
Contributor

It looks like PLEASE_BENCH was broken in 611ef49, which removed all the ratcheting stuff.

lilyball added a commit to lilyball/rust that referenced this issue Jan 19, 2015
@lilyball
Fix make check PLEASE_BENCH=1
c166017 
611ef49 removed all the metrics stuff
from tests.mk, but this meant that `PLEASE_BENCH=1` no longer did
anything.

Fixes rust-lang#21324.
@lilyball lilyball mentioned this issue Jan 19, 2015
Merged
barosl added a commit to barosl/rust that referenced this issue Jan 20, 2015
@barosl
Rollup merge of rust-lang#21399 - kballard:fix-PLEASE_BENCH, r=Gankro
5b57aa7 
611ef49 removed all the metrics stuff
from tests.mk, but this meant that `PLEASE_BENCH=1` no longer did
anything.

Fixes rust-lang#21324.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix make check PLEASE_BENCH=1 lilyball/rust
5 participants
@lilyball @brson @kmcallister @nagisa @estsauver