More useful test error messages on should_panic(expected=...) mismatch #66503

thomasetter · 2019-11-18T00:32:17Z

Shows both the actual as well as the expected panic value when a test with should_panic(expected=...) fails.
This makes should_panic more consistent with assert_eq.

I am not sure whether printing the Any::type_id() is useful, is there something better that we could print for non-string panic values?

rust-highfive · 2019-11-18T00:32:28Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @joshtriplett (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

src/libtest/test_result.rs

joshtriplett · 2019-11-20T06:29:07Z

Looks reasonable to me!

@bors r+

bors · 2019-11-20T06:29:08Z

📋 Looks like this PR is still in progress, ignoring approval.

Hint: Remove [WIP] from this PR's title when it is ready for review.

Centril · 2019-11-20T22:23:49Z

@bors r=joshtriplett

bors · 2019-11-20T22:23:50Z

📌 Commit 16bf4f5 has been approved by joshtriplett

bors · 2019-11-20T23:06:46Z

⌛ Testing commit 16bf4f5 with merge 446a51bafa9f6bfdc812024c948acf4842b4e2e3...

rust-highfive · 2019-11-20T23:10:28Z

Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-11-20T23:10:34Z

💔 Test failed - checks-azure

thomasetter · 2019-11-21T18:09:30Z

Seems to be an independent error, could someone with sufficient privileges retry?

@bors retry

bors · 2019-11-21T18:09:31Z

@thomasetter: 🔑 Insufficient privileges: not in try users

bors · 2019-11-21T18:12:13Z

@thomasetter: 🔑 Insufficient privileges: not in try users

Centril · 2019-11-21T18:30:55Z

@bors retry spurious network

bors · 2019-11-22T03:16:07Z

⌛ Testing commit 16bf4f5 with merge 2363a5241004ed3ae5f4ef0cf3d16713f0c38392...

rust-highfive · 2019-11-22T04:06:39Z

Your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-11-22T03:17:34.0240490Z do so (now or later) by using -b with the checkout command again. Example:
2019-11-22T03:17:34.0240854Z 
2019-11-22T03:17:34.0240938Z   git checkout -b <new-branch-name>
2019-11-22T03:17:34.0241004Z 
2019-11-22T03:17:34.0241088Z HEAD is now at 2363a5241 Auto merge of #66503 - thomasetter:panic-error-msg, r=joshtriplett
2019-11-22T03:17:34.0676400Z ##[section]Starting: Decide whether to run this job
2019-11-22T03:17:34.0784223Z ==============================================================================
2019-11-22T03:17:34.0784477Z Task         : Bash
2019-11-22T03:17:34.0784568Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-11-22T03:17:35.3167405Z BUILD_SOURCEBRANCHNAME=auto
2019-11-22T03:17:35.3167502Z BUILD_SOURCESDIRECTORY=D:\a\1\s
2019-11-22T03:17:35.3167600Z BUILD_SOURCEVERSION=2363a5241004ed3ae5f4ef0cf3d16713f0c38392
2019-11-22T03:17:35.3167706Z BUILD_SOURCEVERSIONAUTHOR=bors
2019-11-22T03:17:35.3167835Z BUILD_SOURCEVERSIONMESSAGE=Auto merge of #66503 - thomasetter:panic-error-msg, r=joshtriplett
2019-11-22T03:17:35.3168096Z CI_JOB_NAME=x86_64-msvc-aux
2019-11-22T03:17:35.3168216Z COBERTURA_HOME=C:\cobertura-2.1.1
2019-11-22T03:17:35.3168323Z COMMONPROGRAMFILES=C:\Program Files\Common Files
2019-11-22T03:17:35.3168648Z COMMON_TESTRESULTSDIRECTORY=D:\a\1\TestResults
---
2019-11-22T03:17:35.3170915Z GeckoWebDriver=C:\SeleniumWebDrivers\GeckoDriver
2019-11-22T03:17:35.3171000Z HOME=/c/Users/VssAdministrator
2019-11-22T03:17:35.3171145Z HOMEDRIVE=C:
2019-11-22T03:17:35.3171219Z HOMEPATH=\Users\VssAdministrator
2019-11-22T03:17:35.3171404Z I am not sure whether printing the `Any::type_id()` is useful, is there something better that we could print for non-string panic values?
2019-11-22T03:17:35.3171698Z INPUT_ARGUMENTS=
2019-11-22T03:17:35.3171942Z ImageVersion=20191028.1
2019-11-22T03:17:35.3172095Z JAVA_HOME=C:\Program Files\Java\zulu-8-azure-jdk_8.40.0.25-8.0.222-win_x64
2019-11-22T03:17:35.3172513Z JAVA_HOME_11_X64=C:\Program Files\Java\zulu-11-azure-jdk_11.33.15-11.0.4-win_x64
---
2019-11-22T03:17:35.3174227Z MSDEPLOY_HTTP_USER_AGENT=VSTS_d439fc94-e01f-4249-b63e-d8392bc0247c_build_10_0
2019-11-22T03:17:35.3174380Z MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\
2019-11-22T03:17:35.3174463Z MSYSTEM=MINGW64
2019-11-22T03:17:35.3174749Z MonAgentClientLocation=C:\Packages\Plugins\Microsoft.Azure.Geneva.GenevaMonitoring\2.12.0.1\Monitoring\Agent
2019-11-22T03:17:35.3174890Z More useful test error messages on should_panic(expected=...) mismatch
2019-11-22T03:17:35.3175173Z NPM_CONFIG_PREFIX=C:\npm\prefix
2019-11-22T03:17:35.3175255Z NUMBER_OF_PROCESSORS=2
2019-11-22T03:17:35.3175378Z OS=Windows_NT
2019-11-22T03:17:35.3176335Z PATH=/mingw64/bin:/usr/bin:/c/Users/VssAdministrator/bin:/c/agents/2.160.1/externals/git/cmd:/c/hostedtoolcache/windows/Python/3.6.8/x64:/c/hostedtoolcache/windows/Python/3.6.8/x64/Scripts:/c/Program Files/Mercurial:/c/ProgramData/kind:/c/vcpkg:/c/cf-cli:/c/Program Files (x86)/NSIS:/c/Program Files/Mercurial:/c/Program Files/Boost/1.69.0:/c/Program Files/dotnet:/c/mysql-5.7.21-winx64/bin:/c/Program Files/Java/zulu-8-azure-jdk_8.40.0.25-8.0.222-win_x64/bin:/c/npm/prefix:/c/Rust/.cargo/bin:/c/hostedtoolcache/windows/Ruby/2.5.5/x64/bin:/c/Go1.12.7/bin:/bin:/usr/bin:/mingw64/bin:/c/hostedtoolcache/windows/Python/3.6.8/x64/Scripts:/c/hostedtoolcache/windows/Python/3.6.8/x64:/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/c/Program Files/Microsoft MPI/Bin:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0:/c/ProgramData/Chocolatey/bin:/c/Program Files/Docker:/c/Program Files/PowerShell/6:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/c/Program Files (x86)/Microsoft SQL Server/110/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/120/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/130/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/140/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/150/DTS/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Program Files/Microsoft Service Fabric/bin/Fabric/Fabric.Code:/c/Program Files/Microsoft SDKs/Service Fabric/Tools/ServiceFabricLocalClusterManager:/cmd:/mingw64/bin:/usr/bin:/c/tools/php:/c/Program Files (x86)/Subversion/bin:/c/Program Files/nodejs:/c/ProgramData/chocolatey/lib/maven/apache-maven-3.6.2/bin:/c/Program Files/CMake/bin:/c/Strawberry/c/bin:/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/c/Program Files/OpenSSL/bin:/c/Users/VssAdministrator/.dotnet/tools:/c/Program Files (x86)/Microsoft SQL Server/120
---
2019-11-22T03:17:35.3187069Z SYSTEM_TEAMPROJECTID=e71b0ddf-dd27-435a-873c-e30f86eea377
2019-11-22T03:17:35.3187226Z SYSTEM_TIMELINEID=a49d49e0-95c0-4776-94b8-be1a6502d92c
2019-11-22T03:17:35.3187317Z SYSTEM_TOTALJOBSINPHASE=16
2019-11-22T03:17:35.3187441Z SYSTEM_WORKFOLDER=D:\a
2019-11-22T03:17:35.3187558Z Shows both the actual as well as the expected panic value when a test with `should_panic(expected=...)` fails.
2019-11-22T03:17:35.3187811Z TEMP=/tmp
2019-11-22T03:17:35.3209688Z TERM=cygwin
2019-11-22T03:17:35.3209793Z TF_BUILD=True
2019-11-22T03:17:35.3209871Z TMP=/tmp
2019-11-22T03:17:35.3209871Z TMP=/tmp
2019-11-22T03:17:35.3209980Z TOOLSTATE_ISSUES_API_URL=https://api.github.com/repos/rust-lang/rust/issues
2019-11-22T03:17:35.3210078Z TOOLSTATE_PUBLISH=1
2019-11-22T03:17:35.3210192Z TOOLSTATE_REPO=https://github.com/rust-lang-nursery/rust-toolstate
2019-11-22T03:17:35.3210300Z This makes `should_panic` more consistent with `assert_eq`.
2019-11-22T03:17:35.3210500Z USERDOMAIN=fv-az379
2019-11-22T03:17:35.3210790Z USERDOMAIN_ROAMINGPROFILE=fv-az379
2019-11-22T03:17:35.3210950Z USERNAME=VssAdministrator
2019-11-22T03:17:35.3211077Z USERPROFILE=C:\Users\VssAdministrator
---
2019-11-22T03:21:19.4117618Z 
2019-11-22T03:21:19.4117756Z 
2019-11-22T03:21:19.4637527Z kill: 1005: No such process
2019-11-22T03:21:19.4648224Z kill: 1004: No such process
2019-11-22T04:05:21.1168680Z Chocolatey timed out waiting for the command to finish. The timeout 
2019-11-22T04:05:21.1169361Z  specified (or the default value) was '2700' seconds. Perhaps try a 
2019-11-22T04:05:21.1169475Z  higher `--execution-timeout`? See `choco -h` for details.
2019-11-22T04:05:22.0015163Z The install of msys2 was NOT successful.
2019-11-22T04:05:22.0016664Z Error while running 'C:\ProgramData\chocolatey\lib\msys2\tools\chocolateyinstall.ps1'.
2019-11-22T04:05:22.0017272Z  See log for details.
2019-11-22T04:05:24.0736917Z Chocolatey installed 0/1 packages. 1 packages failed.
2019-11-22T04:05:24.0737085Z  See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
2019-11-22T04:05:24.0741033Z 
2019-11-22T04:05:24.0746390Z Failures
2019-11-22T04:05:24.0746390Z Failures
2019-11-22T04:05:24.0753709Z  - msys2 (exited -1) - Error while running 'C:\ProgramData\chocolatey\lib\msys2\tools\chocolateyinstall.ps1'.
2019-11-22T04:05:24.0753882Z  See log for details.
2019-11-22T04:05:24.0769604Z Did you know the proceeds of Pro (and some proceeds from other
2019-11-22T04:05:24.0769745Z  licensed editions) go into bettering the community infrastructure?
2019-11-22T04:05:24.0769894Z  Your support ensures an active community, keeps Chocolatey tip top,
2019-11-22T04:05:24.0769983Z  plus it nets you some awesome features!
2019-11-22T04:05:24.0769983Z  plus it nets you some awesome features!
2019-11-22T04:05:24.0770113Z  https://chocolatey.org/compare
2019-11-22T04:05:34.5314479Z The STDIO streams did not close within 10 seconds of the exit event from process 'C:\Program Files\Git\bin\bash.exe'. This may indicate a child process inherited the STDIO streams and has not yet exited.
2019-11-22T04:05:34.5387119Z ##[error]Bash exited with code '127'.
2019-11-22T04:05:39.5553869Z ##[section]Starting: Checkout
2019-11-22T04:05:39.5681100Z ==============================================================================
2019-11-22T04:05:39.5681259Z Task         : Get sources
2019-11-22T04:05:39.5681374Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-11-22T04:06:47Z

💔 Test failed - checks-azure

hdhoang · 2019-11-28T06:17:26Z

@bors retry
chocolatey install timeout

bors · 2019-11-28T06:17:27Z

@hdhoang: 🔑 Insufficient privileges: not in try users

Centril · 2019-11-30T07:58:04Z

@bors retry spurious

@gilescope

…triplett More useful test error messages on should_panic(expected=...) mismatch Fixes rust-lang#66304 r? @gilescope Shows both the actual as well as the expected panic value when a test with `should_panic(expected=...)` fails. This makes `should_panic` more consistent with `assert_eq`. I am not sure whether printing the `Any::type_id()` is useful, is there something better that we could print for non-string panic values?

@ghost

Rollup of 9 pull requests Successful merges: - #66503 (More useful test error messages on should_panic(expected=...) mismatch) - #66662 (Miri: run panic-catching tests in liballoc) - #66679 (Improve lifetime errors with implicit trait object lifetimes) - #66726 (Use recursion_limit for const eval stack limit) - #66790 (Do `min_const_fn` checks for `SetDiscriminant`s target) - #66832 (const_prop: detect and avoid catching Miri errors that require allocation) - #66880 (Add long error code explanation message for E0203) - #66890 (Format liballoc with rustfmt) - #66896 (pass Queries to compiler callbacks) Failed merges: r? @ghost

bors · 2019-12-01T09:04:29Z

☔ The latest upstream changes (presumably #66917) made this pull request unmergeable. Please resolve the merge conflicts.

Thomas Etter added 2 commits November 18, 2019 00:51

print a more useful error message on should_panic mismatch

be993be

replace some asserts with assert_eq for better error readability

48a86e0

rust-highfive assigned joshtriplett Nov 18, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 18, 2019

Centril reviewed Nov 18, 2019

View reviewed changes

src/libtest/test_result.rs Outdated Show resolved Hide resolved

Simplify if else as suggested in PR feedback

16bf4f5

thomasetter force-pushed the panic-error-msg branch from eb63306 to 16bf4f5 Compare November 19, 2019 20:45

thomasetter changed the title ~~[WIP] More useful test error messages on should_panic(expected=...) mismatch~~ More useful test error messages on should_panic(expected=...) mismatch Nov 20, 2019

thomasetter requested a review from Centril November 20, 2019 21:23

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 20, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 20, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 21, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 22, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 30, 2019

Centril mentioned this pull request Dec 1, 2019

Rollup of 9 pull requests #66917

Merged

bors merged commit 16bf4f5 into rust-lang:master Dec 1, 2019

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 1, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More useful test error messages on should_panic(expected=...) mismatch #66503

More useful test error messages on should_panic(expected=...) mismatch #66503

thomasetter commented Nov 18, 2019 •

edited

Loading

rust-highfive commented Nov 18, 2019

joshtriplett commented Nov 20, 2019

bors commented Nov 20, 2019

Centril commented Nov 20, 2019

bors commented Nov 20, 2019

bors commented Nov 20, 2019

rust-highfive commented Nov 20, 2019

bors commented Nov 20, 2019

thomasetter commented Nov 21, 2019 •

edited

Loading

bors commented Nov 21, 2019

bors commented Nov 21, 2019

Centril commented Nov 21, 2019

bors commented Nov 22, 2019

rust-highfive commented Nov 22, 2019

bors commented Nov 22, 2019

hdhoang commented Nov 28, 2019

bors commented Nov 28, 2019

Centril commented Nov 30, 2019

bors commented Dec 1, 2019

More useful test error messages on should_panic(expected=...) mismatch #66503

More useful test error messages on should_panic(expected=...) mismatch #66503

Conversation

thomasetter commented Nov 18, 2019 • edited Loading

rust-highfive commented Nov 18, 2019

joshtriplett commented Nov 20, 2019

bors commented Nov 20, 2019

Centril commented Nov 20, 2019

bors commented Nov 20, 2019

bors commented Nov 20, 2019

rust-highfive commented Nov 20, 2019

bors commented Nov 20, 2019

thomasetter commented Nov 21, 2019 • edited Loading

bors commented Nov 21, 2019

bors commented Nov 21, 2019

Centril commented Nov 21, 2019

bors commented Nov 22, 2019

rust-highfive commented Nov 22, 2019

bors commented Nov 22, 2019

hdhoang commented Nov 28, 2019

bors commented Nov 28, 2019

Centril commented Nov 30, 2019

bors commented Dec 1, 2019

thomasetter commented Nov 18, 2019 •

edited

Loading

thomasetter commented Nov 21, 2019 •

edited

Loading