Compile ignored tests rather than skipping

[klutzy]

Ignored tests:

Lack of support in codegen for closures/generics are always inline:

• src/test/codegen-units/item-collection/cross-crate-closures.rs
• src/test/codegen-units/item-collection/non-generic-closures.rs
• src/test/codegen-units/partitioning/methods-are-with-self-type.rs

This compile successfully, but it should fail:

• src/test/ui/lint/dead-code/closure-bang.rs

[Sawyer47]

My idea is to add a way to specify why a test is ignored (causes ice, fails to compile, fails to run). Test runner could compile such tests and check whether they still compile / cause ice / fail to compile / fail to run and warn if the something changed.

[steveklabnik]

Triage: I'm not aware of any changes in this area.

[Mark-Simulacrum]

Attempted to triage ignored tests. Summary below, also filed #41629 to unignore or update the ignore statements on a few tests to indicate that they are auxiliary files.

Nightly-only:

• src/test/rustdoc/issue-27362.rs

Currently fail, undiagnosed reasons:

• src/test/codegen-units/item-collection/cross-crate-closures.rs
• src/test/codegen-units/item-collection/non-generic-closures.rs
• src/test/codegen-units/partitioning/methods-are-with-self-type.rs
• src/test/run-pass/issue-31702.rs

I don't even know what this tried to test, but I expect it's gone by now (delete?):

• src/test/parse-fail/regions-out-of-scope-slice.rs

This compile successfully, but it should fail:

• src/test/compile-fail/dead-code-closure-bang.rs

Test claims that it has different output on 32/64 bit (#23926):

• src/test/compile-fail/huge-array-simple.rs

Unsized enums aren't supported:

• src/test/compile-fail/issue-17025.rs

This does not appear to be a compile-fail test:

• src/test/compile-fail/macro-expanded-include/foo/mod.rs

This should be a UI-test, probably -- compile-fail can't test this I think:

• src/test/compile-fail/fully-qualified-type-name3.rs

No associated stderr file for undiagnosed reasons:

• src/test/ui/codemap_tests/two_files_data.rs

[pnkfelix]

I don't even know what this tried to test, but I expect it's gone by now (delete?):

* [ ]  `src/test/parse-fail/regions-out-of-scope-slice.rs`

I did some research into where this came from; wow was that a rabbit hole.

Here's the commit where that test came from: 2db4259#diff-d7743dac25dc59daac6583aab0dc1bb8

It has a lot of interesting stuff (beyond just the look back into very old Rust syntax), like a hard-coded blk region for the current block (!).

From its inception, it was tagged an "xfail-test" (as in, ignore-test). The reason for it being ignored, according to f8bd955#diff-884e9fc8b00b17708aa78ee7e6645c6b , is that "blk" wasn't supported in the front-end. (Though I think the type checker was still doing something special for it, as shown here :

rust/src/librustc/middle/typeck/check.rs

Line 580 in f8bd955

let blk_br = ty::br_named(special_idents::blk);

)

So, yeah, we should just kill that test.

[Mark-Simulacrum]

I've filed #71711 which further updates some things here and put an updated description of what's left as ignored tests in the issue description. I'm personally leaning towards closing this at this point as I think the remaining tests are sufficiently uninteresting to not be worth having a meta tracking issue like this for.

[TornaxO7]

May I ask what the state of this issue is? What has to be done?
(I'm a newbie in the rust-eco) Can I help here?

[jyn514]

Oh goodness, this issue is ancient. It looks like the current header command for this is ignore-test, which is used in a bunch of different tests:

; rg ignore-test tests | wc -l
65

@TornaxO7 if you want to help, I think the steps would look something like this:

1. Categorize why the tests are ignored.
   • In particular, a lot have "not a test" comments, so they can be moved to auxiliary folders to avoid compiletest trying to build them.
2. Decide how to fix them. A lot have more specific issues associated with them and those you can probably just ignore. The rest will be on a case-by-case basis; you can ask on Zulip if you get stuck. I think all ignored tests should either have an associated issue or be removed all together.

[jieyouxu]

(I know this issue is ancient, but)

After categorizing and possibly making decisions about how to fix them, we should have compiletest enforce that ignore-test must be given a reason and not just a //@ ignore-test last modified 10 years ago without any context. In general, we should (eventually) enforce that any ignore-* directives must come with a reason justifying the ignore.

[jieyouxu]

compilestest triage: current ignore-test (it is possible that some conditional test running directives produce a combination that leads to tests that never run in CI)

tests\debuginfo\drop-locations.rs
2://@ ignore-test: #128971

tests\rustdoc\macro-document-private-duplicate.rs
1://@ ignore-test (fails spuriously, see issue #89228)

tests\rustdoc\inline_cross\assoc-const-equality.rs
3://@ ignore-test (FIXME: #125092)

tests\ui\explicit-tail-calls\drop-order.rs
2://@ ignore-test: tail calls are not implemented in rustc_codegen_ssa yet, so this causes 🧊

tests\run-make\branch-protection-check-IBT\_rmake.rs
8://@ ignore-test

tests\run-make\branch-protection-check-IBT\Makefile
10:# ignore-test

tests\ui\generic-const-items\recursive.rs
3://@ ignore-test

tests\ui\issues\issue-59756.rs
2://@ ignore-test (rustfix needs multiple suggestions)

tests\ui\issues\issue-49298.rs
9://@ ignore-test (#54987)

tests\ui\lint\dead-code\self-assign.rs
4://@ ignore-test FIXME(81658, 83171)

tests\ui\match\issue-27021.rs
7://@ ignore-test (#54987)

tests\ui\match\issue-26996.rs
7://@ ignore-test (#54987)

tests\ui\lint\unknown-lints\other.rs
1://@ ignore-test

tests\ui\panics\panic-short-backtrace-windows-x86_64.rs
3://@ ignore-test (#92000)

tests\ui\traits\next-solver\object-soundness-requires-generalization.rs
2://@ ignore-test

tests\ui\precondition-checks\write_bytes.rs
5://@ ignore-test

tests\ui\precondition-checks\write.rs
5://@ ignore-test

tests\ui\precondition-checks\read.rs
5://@ ignore-test