Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking Issue for porting run-make tests to use Rust #121876

Open
jieyouxu opened this issue Mar 1, 2024 · 33 comments
Open

Tracking Issue for porting run-make tests to use Rust #121876

jieyouxu opened this issue Mar 1, 2024 · 33 comments
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. E-help-wanted Call for participation: Help is requested to fix this issue. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@jieyouxu
Copy link
Member

jieyouxu commented Mar 1, 2024

We want to stop run-make tests from relying on make, and improve the run-make tests so that
they are more accessible to rustc contributors by allowing the tests to be written in Rust (see
#40713 for context). PR #113026 was merged to address this, and now we are able to write run-make
tests in Rust recipes. We would like your help to port over existing run-make tests still using
Makefiles to use Rust recipes instead.

If you would like to work on porting one of the run-make tests, please link to this issue in your
PR and leave a comment to claim the test (or multiple tests). If you are stuck, please don't hesitate to open a thread
on Rust's Zulip.

run-make is the most flexible fallback test kind, but some run-make tests could be migrated to become e.g. ui tests instead. Please check if you can convert the run-make test into other test kinds before porting!

When you try to port a test, also consider:

  • Add some comments on what the test is trying to test.
  • Add some comments on how the test tries to accomplish its goals if the test is non-trivial. This is especially true if you discovered that a test has platform/compiler/architecture/tooling-specific behavior that is a pain to debug.
  • Any related issues? Any relevant links?
  • Is the test still applicable? Has it become outdated? Is it duplicated?
  • Can it be written in other test suites?

Context: Rust recipes?

PR #113026 adds basic infrastructure support to write run-make tests using small Rust programs,
called recipes.

We aim to eliminate the dependency on make and Makefiles for building run-make-style tests.
Makefiles are replaced by recipes (rmake.rs). The PR implements running run-make recipes in
3 steps:

  1. We build the support library run_make_support which the rmake.rs recipes depend on as a tool
    lib.
  2. We build the recipe rmake.rs and link in the support library.
  3. We run the recipe to build and run the tests.

rmake.rs is basically a replacement for Makefile, and allows running arbitrary Rust code. The
support library is built using cargo, and so can depend on external crates if desired.

The infrastructure implemented by the PR is very barebones, and is the minimally required
infrastructure needed to build, run and pass the two example run-make tests ported over to the new
infrastructure:

You likely will find that you would need to improve the API of the support library, and extend
the functionality of the support library.

Common traps and pitfalls, and tips and tricks

  • $PATH uses ; on Windows and : for *nixes. Use std::env::{join,split}_paths to properly
    handle $PATH.

  • Be careful of path separator platform differences. Always prefer PathBuf operations not string
    paths if possible.

  • tests/ are not (currently) formatted by rustfmt.

  • Consult https://github.com/rust-lang/rust/blob/master/tests/run-make/tools.mk for which flags
    and envs are passed to various executables or libraries. May have to triple check on those.

  • You can always request your reviewer to run try jobs to test out your PR on environments you don't locally have access to. Good candidate CI jobs include:

    • aarch64-apple (apple, 64 bits, aarch64/arm64)
    • armhf-gnu (cross-compile)
    • test-various (cross-compile, wasm)
    • x86_64-mingw (x86_64, 64 bits, windows, mingw)
    • x86_64-msvc (x86_64, 64 bits, windows, msvc)
    • x86_64-gnu-llvm-18 (x86_64, 64 bits, windows, gnu, llvm 18)
    • i686-msvc (x86, 32 bits, windows, msvc)

Tests that need porting

@jieyouxu jieyouxu added A-testsuite Area: The testsuite used to check the correctness of rustc T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. E-help-wanted Call for participation: Help is requested to fix this issue. C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. labels Mar 1, 2024
@5225225

This comment was marked as resolved.

@abhay-51

This comment was marked as resolved.

@high-cloud

This comment was marked as resolved.

@0xhiro

This comment was marked as resolved.

@JoverZhang

This comment was marked as resolved.

@workingjubilee workingjubilee added the E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. label Mar 22, 2024
@Oneirical

This comment was marked as resolved.

@Oneirical

This comment was marked as resolved.

workingjubilee added a commit to workingjubilee/rustc that referenced this issue Mar 31, 2024
Rewrite `core-no-fp-fmt-parse` test in Rust

Claiming the simple "core-no-fp-fmt-parse" test from rust-lang#121876. `run_make_support` was altered with `arg_path` written in rust-lang#121918 by `@abhay-51,` with additional doc comment.

Preliminary GSoC contribution for the project proposal mentored by `@jieyouxu.`
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Mar 31, 2024
Rollup merge of rust-lang#123180 - Oneirical:master, r=Mark-Simulacrum

Rewrite `core-no-fp-fmt-parse` test in Rust

Claiming the simple "core-no-fp-fmt-parse" test from rust-lang#121876. `run_make_support` was altered with `arg_path` written in rust-lang#121918 by `@abhay-51,` with additional doc comment.

Preliminary GSoC contribution for the project proposal mentored by `@jieyouxu.`
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Apr 4, 2024
Port hir-tree run-make test to ui test

As part of rust-lang#121876

cc `@jieyouxu`
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 4, 2024
Rollup merge of rust-lang#122448 - high-cloud:move-hir-tree, r=oli-obk

Port hir-tree run-make test to ui test

As part of rust-lang#121876

cc `@jieyouxu`
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Apr 5, 2024
…e-enum, r=Mark-Simulacrum

Port argument-non-c-like-enum to Rust

Part of rust-lang#121876.
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Apr 5, 2024
…imulacrum

Port `run-make/issue-7349` to a codegen test

The test does not need to be a run-make test, it can use the codegen test infrastructure.

Also took the opportunity to rename the test to `no-redundant-item-monomorphization` so it's not just some opaque issue number.

Part of rust-lang#121876.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 5, 2024
Rollup merge of rust-lang#123474 - jieyouxu:issue-7349-port, r=Mark-Simulacrum

Port `run-make/issue-7349` to a codegen test

The test does not need to be a run-make test, it can use the codegen test infrastructure.

Also took the opportunity to rename the test to `no-redundant-item-monomorphization` so it's not just some opaque issue number.

Part of rust-lang#121876.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 5, 2024
Rollup merge of rust-lang#123149 - jieyouxu:rmake-arguments-non-c-like-enum, r=Mark-Simulacrum

Port argument-non-c-like-enum to Rust

Part of rust-lang#121876.
tgross35 added a commit to tgross35/rust that referenced this issue Aug 23, 2024
…jieyouxu

Migrate `libtest-thread-limit` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

Please try, but **only if normal CI is green**:

try-job: aarch64-apple
try-job: aarch64-gnu
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 23, 2024
…eyouxu

Migrate `libtest-thread-limit` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

Please try, but **only if normal CI is green**:

try-job: aarch64-apple
try-job: aarch64-gnu
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Aug 23, 2024
…r=jieyouxu

Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

The final Makefile! Every Makefile test is now claimed.

This is difficult to test due to the uncommon architecture it is specific to. I don't think it is in the CI (I didn't find it in `jobs.yml`, but if there is a way to test it, please do.

Locally, on Linux, it compiles and panics at the `llvm_filecheck` part (if I replace the `x86_64-fortanix-unknown-sgx` with `x86_64-unknown-linux-gnu`, of course), which is expected.

For this reason, the Makefile and associated script have been kept, but with a leading underscore.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Aug 23, 2024
Rollup merge of rust-lang#129055 - Oneirical:fortanix-fortification, r=jieyouxu

Migrate `x86_64-fortanix-unknown-sgx-lvi` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

The final Makefile! Every Makefile test is now claimed.

This is difficult to test due to the uncommon architecture it is specific to. I don't think it is in the CI (I didn't find it in `jobs.yml`, but if there is a way to test it, please do.

Locally, on Linux, it compiles and panics at the `llvm_filecheck` part (if I replace the `x86_64-fortanix-unknown-sgx` with `x86_64-unknown-linux-gnu`, of course), which is expected.

For this reason, the Makefile and associated script have been kept, but with a leading underscore.
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 23, 2024
…eyouxu

Migrate `libtest-thread-limit` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

Please try, but **only if normal CI is green**:

// try-job: armhf-gnu // <- failed on this
try-job: aarch64-gnu
jieyouxu added a commit to jieyouxu/rust that referenced this issue Aug 23, 2024
…jieyouxu

Migrate `libtest-thread-limit` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

Please try, but **only if normal CI is green**:

// try-job: armhf-gnu // <- failed on this
try-job: aarch64-gnu
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 23, 2024
…eyouxu

Migrate `libtest-thread-limit` `run-make` test to rmake

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

Please try, but **only if normal CI is green**:

// try-job: armhf-gnu // <- failed on this
try-job: aarch64-gnu
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Aug 29, 2024
…ouxu

Add `needs-unwind` compiletest directive to `libtest-thread-limit` and replace some `Path` with `path` in `run-make`

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

This PR does two things:

1. Add this to `libtest-thread-limit` ([Why?](rust-lang#128507 (comment)))
```
//@ needs-unwind
// Reason: this should be ignored in cg_clif (Cranelift) CI and anywhere
// else that uses panic=abort.
```

2. Use `path` instead of `Path` to simplify multiple run-make tests.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Aug 29, 2024
Rollup merge of rust-lang#129690 - Oneirical:run-make-tidbits, r=jieyouxu

Add `needs-unwind` compiletest directive to `libtest-thread-limit` and replace some `Path` with `path` in `run-make`

Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).

This PR does two things:

1. Add this to `libtest-thread-limit` ([Why?](rust-lang#128507 (comment)))
```
//@ needs-unwind
// Reason: this should be ignored in cg_clif (Cranelift) CI and anywhere
// else that uses panic=abort.
```

2. Use `path` instead of `Path` to simplify multiple run-make tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc C-tracking-issue Category: A tracking issue for an RFC or an unstable feature. E-help-wanted Call for participation: Help is requested to fix this issue. E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
Status: In progress
Development

No branches or pull requests