Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Summarize differences with compiletest-rs? #13

Closed
dekellum opened this issue Jun 3, 2019 · 3 comments
Closed

Summarize differences with compiletest-rs? #13

dekellum opened this issue Jun 3, 2019 · 3 comments

Comments

@dekellum
Copy link

dekellum commented Jun 3, 2019

I currently have an unmerged branch in one project (tao-log) using compiletest-rs for the same use case. Its unmerged because compiletest-rs appears, in my very limited experience, to be nightly-only and quite sensitive to ongoing changes in the rustc nightly builds.

Would it be reasonable to briefly summarize differences in design, approach or goals with trybuild, either here, or for example in a Prior Art or Alternatives section in the README?

@dtolnay
Copy link
Owner

dtolnay commented Jun 30, 2019

I got pretty unhappy with the design of compiletest and with the way the library is maintained. This library does the small set of things that I want from an error message testing library without all the downsides of the compiletest crate; hopefully it is useful to others too.

What's wrong with compiletest:

Differences in trybuild:

  • We invoke cargo instead of rustc. No handwritten rustc flags, no "multiple matching crates" errors, all dev-dependencies immediately accessible, no unstable libtest, no rustc_private, no nightly features.

@dekellum
Copy link
Author

dekellum commented Jul 1, 2019

So I wasn't alone in my compiletest-rs stability concerns, but you've given a lot more detail than I was aware of, thanks. I found converting my compile tests to trybuild straight forward:

https://github.com/dekellum/tao-log/compare/2efb701a..70889823

...and as a bonus I can now enable my tests on stable rust 1.35.0 and nightly. (Before 1.35.0, the rustc error messages under test are different.) Other minor differences I'll note from the fresh conversion experience:

  • trybuild uses separate *.stderr files for comparing compiler output, instead of source inline //~ ERROR style comments as used in rust/src/tools/compiletest and compiletest-rs. The workflow feature of writing the actual compiler output to compatible files in a wip directory makes this much easier than guessing at the supported comment syntax of the former.

  • trybuild compile tests expect an fn main (It presumably compiles using --bin or equivalent. Without this, an extra "error[E0601]: main function not found in crate $CRATE" will be recorded.)

So perhaps rust/src/tools/compiletest should also be mentioned in a Prior Art type summary?
Compiling using cargo seems to be a big stability and ease of use advantage for trybuild. Thanks for sharing this work!

@aldanor
Copy link

aldanor commented Aug 18, 2019

@dtolnay That's a pretty good description; would it make sense to list some of that directly in the readme? I believe most people who've stumbled upon this crate would be coming directly from compiletest-rs.

bors bot added a commit to taiki-e/auto_enums that referenced this issue Sep 26, 2019
61: Use trybuild instead of compiletest r=taiki-e a=taiki-e

Refs: dtolnay/trybuild#13

Co-authored-by: Taiki Endo <te316e89@gmail.com>
bors bot added a commit to taiki-e/futures-async-stream that referenced this issue Sep 26, 2019
19: Use trybuild instead of compiletest r=taiki-e a=taiki-e

Refs: dtolnay/trybuild#13

Co-authored-by: Taiki Endo <te316e89@gmail.com>
Repository owner locked and limited conversation to collaborators Apr 26, 2020
@dtolnay dtolnay closed this as completed Jun 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants