-
Notifications
You must be signed in to change notification settings - Fork 13.6k
Open
Labels
A-compiletestArea: The compiletest test runnerArea: The compiletest test runnerA-testsuiteArea: The testsuite used to check the correctness of rustcArea: The testsuite used to check the correctness of rustcC-bugCategory: This is a bug.Category: This is a bug.E-mediumCall for participation: Medium difficulty. Experience needed to fix: Intermediate.Call for participation: Medium difficulty. Experience needed to fix: Intermediate.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Description
These headers allow individual tests to define custom output-normalization rules, so that output snapshots are less sensitive to things like platform differences, line numbers, and timestamps.
Unfortunately, they have a few limitations and pitfalls:
- The value parser for these headers is very lax, and when it does fail to parse, the header is silently ignored.
- The parser doesn't require a colon
:
after the header name, so the test suite contains an inconsistent mix of with-colon and without-colon rules. - The rules cannot contain literal double-quote
"
characters, which is very awkward for some kinds of output (e.g. JSON). - The parser uses
parse_cfg_name_directive
to help read the header name, which is sometimes useful (e.g. for defining separate rules for 32-bit and 64-bit), but most of the time it means that tests have to awkwardly writenormalize-stderr-test
instead of the more intuitivenormalize-stderr
. - The rules are applied to both compile output and run output, which is usually not a problem in practice, but can be unexpected.
- Like many other headers, they are only meaningful in
ui
-mode tests, and are silently ignored if they appear in other kinds of test.
@rustbot label +A-compiletest
jieyouxu
Metadata
Metadata
Assignees
Labels
A-compiletestArea: The compiletest test runnerArea: The compiletest test runnerA-testsuiteArea: The testsuite used to check the correctness of rustcArea: The testsuite used to check the correctness of rustcC-bugCategory: This is a bug.Category: This is a bug.E-mediumCall for participation: Medium difficulty. Experience needed to fix: Intermediate.Call for participation: Medium difficulty. Experience needed to fix: Intermediate.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)