Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Guide answers "what" but not "why" and "how" for codegen tests #1965

Open
workingjubilee opened this issue Apr 23, 2024 · 0 comments
Open

Guide answers "what" but not "why" and "how" for codegen tests #1965

workingjubilee opened this issue Apr 23, 2024 · 0 comments
Labels
A-codegen Area: code generation A-FileCheck Area: LLVM FileCheck tool A-test-suite Area: our test suite `tests/` C-enhancement Category: enhancement E-hard Difficulty: might require advanced knowledge T-compiler Relevant to compiler team

Comments

@workingjubilee
Copy link
Member

The section on codegen tests is bereft of an explanation of how to use FileCheck, and I do not think the documentation for FileCheck is the best example for it. The documentation for FileCheck focuses on its usage as a CLI tool first, but we are not passing random args to FileCheck!

In fact, it's probably exactly the reverse order for what we want. What is probably most important is how and when to use the regex capturing, introduced at the end, since most tests shouldn't break if a variable changes its name but uses a few different ones: we don't want to focus on LLVM's exact behavior like that. Then we usually want people to use CHECK-LABELs, again, introduced just before regex usage. Then we want to discuss when to use CHECK-NEXT, CHECK-NOT, and CHECK-DAG, and when not to use them.

In our actual usage, CHECK, the directive they introduce first and focus on using, is mostly a directive for corraling CHECK-NEXT and CHECK-DAGs when CHECK-LABEL does not suffice for that, as CHECK is itself the hardest-to-contain directive.

A step-by-step explanation for

  • how to write an okay FileCheck test
  • how to write a really good one

would be better.

And neither of these answer why we use them, which is particular to the Rust codebase. We can't simply upstream all our tests unless LLVM adds rustc to the test suite, as we want e.g. our MIR opts to be taken into account when running our codegen tests.

@jieyouxu jieyouxu added T-compiler Relevant to compiler team C-enhancement Category: enhancement A-codegen Area: code generation A-test-suite Area: our test suite `tests/` A-FileCheck Area: LLVM FileCheck tool E-hard Difficulty: might require advanced knowledge labels Nov 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-codegen Area: code generation A-FileCheck Area: LLVM FileCheck tool A-test-suite Area: our test suite `tests/` C-enhancement Category: enhancement E-hard Difficulty: might require advanced knowledge T-compiler Relevant to compiler team
Projects
None yet
Development

No branches or pull requests

2 participants