Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rust building and testing fails while bootstrapping it in poky #115642

Closed
Yashinde145 opened this issue Sep 7, 2023 · 56 comments · Fixed by #119619
Closed

Rust building and testing fails while bootstrapping it in poky #115642

Yashinde145 opened this issue Sep 7, 2023 · 56 comments · Fixed by #119619
Labels
T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@Yashinde145
Copy link

Yashinde145 commented Sep 7, 2023

Getting error while building & testing rust 1.72 via the src/bootstrap/bootstrap.py file in poky (cmd- bitbake rust and then python3 src/bootstrap/bootstrap.py test test_suites_names --target x86_64-poky-linux-gnu)in poky environment. Until rust 1.70 I was able to build & test the rust sources successfully.
But after updating to rust 1.72, there's an assertion failure causing the problem. The rust build fails as follows-

Note that not all variants of mir-opt tests are going to be blessed, as no mapping between a 
32bit and a 64bit target was found for x86_64-poky-linux-gnu.
You can add that mapping by changing MIR_OPT_BLESS_TARGET_MAPPING in src/bootstrap/test.rs
thread 'main' panicked at 'assertion failed: `(left == right)`
  left: `x86_64-unknown-linux-gnu`,
 right: `x86_64-poky-linux-gnu`: Cannot obtain compiler for non-native build triple at stage 0', compile.rs:1474:13

The code at compile.rs:1474

1470     fn run(self, builder: &Builder<'_>) -> Compiler {
1471         let target_compiler = self.target_compiler;
1472 
1473         if target_compiler.stage == 0 {
1474             assert_eq!(
1475                 builder.config.build, target_compiler.host,
1476                 "Cannot obtain compiler for non-native build triple at stage 0"
1477             );
1478             // The stage 0 compiler for the build triple is always pre-built.
1479             return target_compiler;
1480         }

Build terminates as follows-

test [mir-opt] tests/mir-opt/reference_prop.rs ... ok
test [mir-opt] tests/mir-opt/inline/polymorphic_recursion.rs ... ok
test result: ok. 249 passed; 0 failed; 13 ignored; 0 measured; 0 filtered out; finished in 1.39s

error: the option `Z` is only accepted on the nightly compiler
note: selecting a toolchain with `+toolchain` arguments require a rustup proxy; see <https://rust-lang.github.io/rustup/concepts/index.html>
help: consider switching to a nightly toolchain: `rustup default nightly`
note: for more information about Rust's stability policy, see <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html#unstable-features>
thread 'main' panicked at 'failed to gather the target spec for i686-unknown-linux-gnu', synthetic_targets.rs:66:9
Build completed unsuccessfully in 0:05:27

Version it worked on

It most recently worked on Rust 1.70 and earlier versions

Version with regression

Rust 1.72

Backtrace

Backtrace

stack backtrace:                                                                                                                                                                                                
   0: rust_begin_unwind                                                                                                                                                                                         
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5                                                                                                              
   1: core::panicking::panic_fmt                                                                                                                                                                                
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14                                                                                                             
   2: core::panicking::assert_failed_inner                                                                                                                                                                      
   3: core::panicking::assert_failed                                                                                                                                                                            
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:228:5                                                                                                             
   4: <bootstrap::compile::Assemble as bootstrap::builder::Step>::run                                                                                                                                           
             at ./src/bootstrap/compile.rs:1474:13                                                                                                                                                              
   5: bootstrap::builder::Builder::ensure                                                                                                                                                                       
             at ./src/bootstrap/builder.rs:2055:23                                                                                                                                                              
   6: bootstrap::builder::Builder::compiler                                                                                                                                                                     
             at ./src/bootstrap/builder.rs:943:9                                                                                                                                                                
   7: <bootstrap::test::Compiletest as bootstrap::builder::Step>::run                                                                                                                                           
             at ./src/bootstrap/test.rs:1462:24                                                                                                                                                                 
   8: bootstrap::builder::Builder::ensure                                                                                                                                                                       
             at ./src/bootstrap/builder.rs:2055:23                                                                                                                                                              
   9: <bootstrap::test::UiFullDeps as bootstrap::builder::Step>::run                                                                                                                                            
             at ./src/bootstrap/test.rs:1260:17                                                                                                                                                                 
  10: bootstrap::builder::Builder::ensure                                                                                                                                                                       
             at ./src/bootstrap/builder.rs:2055:23                                                                                                                                                              
  11: <bootstrap::test::UiFullDeps as bootstrap::builder::Step>::make_run                                                                                                                                       
             at ./src/bootstrap/test.rs:1256:17                                                                                                                                                                 
  12: bootstrap::builder::StepDescription::maybe_run                                                                                                                                                            
             at ./src/bootstrap/builder.rs:278:13                                                                                                                                                               
  13: bootstrap::builder::StepDescription::run                                                                                                                                                                  
             at ./src/bootstrap/builder.rs:315:21                                                                                                                                                               
  14: bootstrap::builder::Builder::run_step_descriptions                                                                                                                                                        
             at ./src/bootstrap/builder.rs:935:9                                                                                                                                                                
  15: bootstrap::builder::Builder::execute_cli                                                                                                                                                                  
             at ./src/bootstrap/builder.rs:916:9                                                                                                                                                                
  16: bootstrap::Build::build                                                                                                                                                                                   
             at ./src/bootstrap/lib.rs:697:17                                                                                                                                                                   
  17: bootstrap::main                                                                                                                                                                                           
             at ./src/bootstrap/bin/main.rs:74:5                                                                                                                                                                
  18: core::ops::function::FnOnce::call_once                                                                                                                                                                    
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/ops/function.rs:250:5                                                                                                          
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.                                                                                                                         
Build completed unsuccessfully in 0:00:20


<!--
@rustbot modify labels: +regression-from-stable-to-{stable} -regression-untriaged 

@Yashinde145 Yashinde145 added C-bug Category: This is a bug. regression-untriaged Untriaged performance or correctness regression. labels Sep 7, 2023
@rustbot rustbot added I-prioritize Issue: Indicates that prioritization has been requested for this issue. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Sep 7, 2023
@Noratrieb
Copy link
Member

I believe MIR_OPT_BLESS_TARGET_MAPPING is unrelated to the issue you're seeing, but I'm not sure.

@Yashinde145
Copy link
Author

Yeah, I guess the assertion failure is main cause.
Why the two strings are different though? Until rust 1.70, I didn't faced this assertion failure(diff in builder.config.build i.e left: x86_64-unknown-linux-gnu, and target_compiler.host i.e right: x86_64-poky-linux-gnu). When did git blame for these lines, found that this code was present (from 5 years i.e 2018)even in the earlier rust versions.

@apiraino
Copy link
Contributor

Adding useful labels for triaging

@rustbot label -I-prioritize +T-bootstrap -needs-triage

@rustbot rustbot added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) and removed I-prioritize Issue: Indicates that prioritization has been requested for this issue. needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. labels Sep 12, 2023
@onur-ozkan
Copy link
Member

onur-ozkan commented Sep 13, 2023

I couldn't reproduce this problem from the e39976f HEAD.

The assertation passes as expected(on `./x test bootstrap --target=..), I also printed them out.
image

The diff:
image

Sorry for sharing the the outputs in the image format; the attempt was done inside of QEMU(Debian 11).

@Yashinde145
Copy link
Author

Yashinde145 commented Sep 14, 2023

Thanks for checking @onur-ozkan,
But, my build shows following-
Note- The targets are printed 3 times and the assertion fails in the 3rd call

Compiling clap v4.2.4
   Compiling clap_complete v4.2.2
    Finished dev [unoptimized] target(s) in 20.26s

builder.config.build = x86_64-unknown-linux-gnu
target_compiler.host = x86_64-unknown-linux-gnu
builder.config.build = x86_64-unknown-linux-gnu
target_compiler.host = x86_64-unknown-linux-gnu
builder.config.build = x86_64-unknown-linux-gnu
target_compiler.host = x86_64-poky-linux-gnu

Moreover, if I bypass the assertion check and then build, why do the compiler asks to switch to nightly channel instead of continuing to stable? I am wondering how does it gets the Z option during the build since I did not not provide any such options.

test [mir-opt] tests/mir-opt/reference_prop.rs ... ok
test [mir-opt] tests/mir-opt/inline/polymorphic_recursion.rs ... ok
test result: ok. 249 passed; 0 failed; 13 ignored; 0 measured; 0 filtered out; finished in 1.39s

error: the option `Z` is only accepted on the nightly compiler
note: selecting a toolchain with `+toolchain` arguments require a rustup proxy; see <https://rust-lang.github.io/rustup/concepts/index.html>
help: consider switching to a nightly toolchain: `rustup default nightly`
note: for more information about Rust's stability policy, see <https://doc.rust-lang.org/book/appendix-07-nightly-rust.html#unstable-features>
thread 'main' panicked at 'failed to gather the target spec for i686-unknown-linux-gnu', synthetic_targets.rs:66:9
Build completed unsuccessfully in 0:05:27

Let me know if I am missing something

@onur-ozkan
Copy link
Member

Do you have a custom config.toml?

@Yashinde145
Copy link
Author

The config.toml generated during the rust build is as follows-

build_dir is the directory where I am building rust (using cmd bitbake rust in yocto poky sources)

[target.x86_64-poky-linux-gnu]                                                                                                                                                                                  
llvm-config = "build_dir/recipe-sysroot/usr/lib/llvm-rust/bin/llvm-config"                                                                         
cxx = "build_dir/wrapper/target-rust-cxx"                                                                                                          
cc = "build_dir/wrapper/target-rust-cc"                                                                                                            
linker = "build_dir/wrapper/target-rust-ccld"                                                                                                      
                                                                                                                                                                                                                
[target.x86_64-unknown-linux-gnu]                                                                                                                                                                               
llvm-config = "build_dir/recipe-sysroot-native/usr/lib/llvm-rust/bin/llvm-config"                                                                  
cxx = "build_dir/wrapper/build-rust-cxx"                                                                                                           
cc = "build_dir/wrapper/build-rust-cc"                                                                                                             
linker = "build_dir/wrapper/build-rust-ccld"                                                                                                       
                                                                                                                                                                                                                
[llvm]                                                                                                                                                                                                          
static-libstdcpp = false                                                                                                                                                                                        
                                                                                                                                                                                                                
[rust]                                                                                                                                                                                                          
rpath = true                                                                                                                                                                                                    
channel = "stable"                                                                                                                                                                                              
optimize = true                                                                                                                                                                                                 
verbose-tests = true                                                                                                                                                                                            
                                                                                                                                                                                                                
[build]                                                                                                                                                                                                         
submodules = false                                                                                                                                                                                              
docs = false                                                                                                                                                                                                    
rustc = "build_dir/rust-snapshot/bin/rustc"                                                                                                        
profiler = true                                                                                                                                                                                                 
cargo = "build_dir/rust-snapshot/bin/cargo"                                                                                                        
vendor = true                                                                                                                                                                                                   
target = ["x86_64-poky-linux-gnu"]                                                                                                                                                                              
host = ["x86_64-poky-linux-gnu"]                                                                                                                                                                                
build = "x86_64-unknown-linux-gnu"                                                                                                                                                                              
                                                                                                                                                                                                                
[install]                                                                                                                                                                                                       
prefix = "build_dir/image/usr"                                                                                                                     
bindir = "build_dir/image/usr/bin"                                                                                                                 
libdir = "build_dir/image/usr/lib"                                                                                                                 
datadir = "build_dir/image/usr/share"                                                                                                              
mandir = "build_dir/image/usr/share/man"

@onur-ozkan
Copy link
Member

iirc bitbake rust will give you the stable 1.68 compiler. If so, please remove build.cargo and build.rust from your configuration, and use the beta compiler which will be downloaded from CI automatically during the bootstrapping.

@Yashinde145
Copy link
Author

Actually, bitbake rust will give stable 1.70 compiler(on master branch). I am manually upgrading the version to 1.72 and then getting the error in bootstrapping build.

I didn't understand removing build.cargo and build.rust and using beta compiler from CI during bootstrapping. Can you enlighten me on it?

@onur-ozkan
Copy link
Member

onur-ozkan commented Sep 14, 2023

Actually, bitbake rust will give stable 1.70 compiler(on master branch).

Still, this is wrong.

I didn't understand removing build.cargo and build.rust and using beta compiler from CI during bootstrapping. Can you enlighten me on it?

Remove the following lines:

rustc = "build_dir/rust-snapshot/bin/rustc"                                                                                                        
cargo = "build_dir/rust-snapshot/bin/cargo"                                                                                                        

from your configuration

@Yashinde145
Copy link
Author

Ok, will check and get back.

@Yashinde145
Copy link
Author

Yashinde145 commented Sep 21, 2023

@onur-ozkan Thanks for the suggestion, but I can't modify config.toml. The same config.toml worked with previous rust versions. Actually, the build compiler fetches some -Z (nightly build flag option) while building on the stable channel and it's recommended not to shift to nightly build. Is there any way I can find which flag causes this build failure and suppress that flag to proceed the build?

@Yashinde145
Copy link
Author

@onur-ozkan, any updates?

@onur-ozkan
Copy link
Member

onur-ozkan commented Oct 20, 2023

Sorry, I somehow missed the notification.

Bootstrap requires the immediate previous version to compile the next version, allowing it to handle unstable features. So there's an issue with your configuration (rustc and cargo), and you should find a way to update your config file. I'm surprised you didn't encounter an error message like: Unexpected rustc version: 1.70.0; we should use **** to build the source with ****. It might be more helpful to receive this error message at the beginning of the bootstrap process, I suppose.

@Yashinde145
Copy link
Author

@onur-ozkan

Actually, I see that -Zdoctest-in-workspace a nightly flag is passed by default in Rust 1.72 release notes here

More info- docs rust-lang/cargo#12221 rust-lang/cargo#12288

I suspect that -Zdoctest-in-workspace is causing buid failure as seen in above comment here, #115642 (comment).

@onur-ozkan
Copy link
Member

As I explained above, you cannot build upstream rustc with 1.70 anymore, even for non-poky environments. You should either not set rustc manually in the build configuration or set it to the proper version.

kraj pushed a commit to YoeDistro/poky that referenced this issue Dec 8, 2023
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

(From OE-Core rev: 769e3f7e3871a375b1037dfb391e67876f987450)

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
kraj pushed a commit to YoeDistro/poky that referenced this issue Dec 8, 2023
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

(From OE-Core rev: 769e3f7e3871a375b1037dfb391e67876f987450)

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
kraj pushed a commit to YoeDistro/poky that referenced this issue Dec 8, 2023
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

(From OE-Core rev: 769e3f7e3871a375b1037dfb391e67876f987450)

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
kraj pushed a commit to YoeDistro/poky that referenced this issue Dec 8, 2023
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

(From OE-Core rev: 769e3f7e3871a375b1037dfb391e67876f987450)

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
kraj pushed a commit to YoeDistro/poky that referenced this issue Dec 9, 2023
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

(From OE-Core rev: 3faebfc49cd8aa00a63fbe1cf4d21ff4e6f6a667)

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
halstead pushed a commit to yoctoproject/poky that referenced this issue Dec 9, 2023
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

(From OE-Core rev: e5f007349ae92be839d13e489963b258b53d0c59)

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Dec 9, 2023
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
@onur-ozkan
Copy link
Member

onur-ozkan commented Jan 6, 2024

#119619 PR will close this issue upon merging, as we have a separate issue (#97669) open for the other problem.

@Yashinde145
Copy link
Author

@onur-ozkan , Thanks for the help to fix this issue.

Here we have an issue for the remote-test tool problem #97669
It seems to occur randomly in QEMU environments from time to time. Retrying seems to fix the issue for each occurrence. I will try to identify the root cause in the coming week.

Yeah, I faced this panic failures randomly after the test suite testing is completed starting from rust 1.71 version

@Yashinde145
Copy link
Author

Just remembered now,

How do I handle the assertion mentioned in #115642 (comment)?
Until now I was by-passing the assertion check with the following changes:
Should I continue with it?

diff --git a/src/bootstrap/src/core/build_steps/compile.rs b/src/bootstrap/src/core/build_steps/compile.rs                          
--- a/src/bootstrap/src/core/build_steps/compile.rs                                                                                 
+++ b/src/bootstrap/src/core/build_steps/compile.rs                                                                                 
@@ -1533,13 +1533,15 @@                                                                                                              
     /// must have been previously produced by the `stage - 1` builder.build                                                        
     /// compiler.                                                                                                                  
     fn run(self, builder: &Builder<'_>) -> Compiler {                                                                              
-        let target_compiler = self.target_compiler;                                                                                
+        let mut target_compiler = self.target_compiler;                                                                            
+        target_compiler.host = builder.config.build;                                                                                                                                              
                                                                                                                                    
         if target_compiler.stage == 0 {                                                                                            
             assert_eq!( 
                       builder.config.build, target_compiler.host,
                       "Cannot obtain compiler for non-native build triple at stage 0"
                       );
             // The stage 0 compiler for the build triple is always pre-built.
             return target_compiler;
         }

@onur-ozkan
Copy link
Member

onur-ozkan commented Jan 6, 2024

I will add another commit to handle that.

update: e7fc441

@Yashinde145
Copy link
Author

Do we still need the patch mentioned in #115642 (comment) along with the changes in #119619 PR?

I think the three file changes in the #119619 PR should be sufficient, right?

@onur-ozkan
Copy link
Member

Do we still need the patch mentioned in #115642 (comment) along with the changes in #119619 PR?

I think the three file changes in the #119619 PR should be sufficient, right?

#119619 PR is enough, you don't need any other patches.

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Jan 8, 2024
…oli-obk

mir-opt and custom target fixes

From rust-lang#115642 (comment)

> > Could you please test the last two commits from https://github.com/onur-ozkan/rust/commits/panic-abort-mir-opt when you have the time? The first commit should resolve the error of using the nightly flag with a stable compiler, and the second one should resolve the custom target issue.

> I tested with the two commits and the errors of using nightly flag and custom target specs were not seen.
Testing was completed for the test suites like ui, run-pass-valgrind, coverage, mir-opt, codegen, assembly, incremental.

Fixes rust-lang#115642
@bors bors closed this as completed in 985b2ce Jan 9, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Jan 9, 2024
Rollup merge of rust-lang#119619 - onur-ozkan:panic-abort-mir-opt, r=oli-obk

mir-opt and custom target fixes

From rust-lang#115642 (comment)

> > Could you please test the last two commits from https://github.com/onur-ozkan/rust/commits/panic-abort-mir-opt when you have the time? The first commit should resolve the error of using the nightly flag with a stable compiler, and the second one should resolve the custom target issue.

> I tested with the two commits and the errors of using nightly flag and custom target specs were not seen.
Testing was completed for the test suites like ui, run-pass-valgrind, coverage, mir-opt, codegen, assembly, incremental.

Fixes rust-lang#115642
github-actions bot pushed a commit to rust-lang/miri that referenced this issue Jan 10, 2024
mir-opt and custom target fixes

From rust-lang/rust#115642 (comment)

> > Could you please test the last two commits from https://github.com/onur-ozkan/rust/commits/panic-abort-mir-opt when you have the time? The first commit should resolve the error of using the nightly flag with a stable compiler, and the second one should resolve the custom target issue.

> I tested with the two commits and the errors of using nightly flag and custom target specs were not seen.
Testing was completed for the test suites like ui, run-pass-valgrind, coverage, mir-opt, codegen, assembly, incremental.

Fixes #115642
@Yashinde145
Copy link
Author

Here we have an issue for the remote-test tool problem #97669

It seems to occur randomly in QEMU environments from time to time. Retrying seems to fix the issue for each occurrence. I will try to identify the root cause in the coming week.

Hi @onur-ozkan,
Did you get a chance to look at the remote test-tool problem?.

@onur-ozkan
Copy link
Member

Did you get a chance to look at the remote test-tool problem?

Not yet

@onur-ozkan
Copy link
Member

Are you able to run any tests with remote-test or does it give the error instantly at the beginning?

@Yashinde145
Copy link
Author

Yes, I am able to run the tests. The error is seen at
Building stage0 tool remote-test-server (x86_64-unknown-linux-gnu -> x86_64-poky-linux-gnu)

@onur-ozkan
Copy link
Member

I meant sending tests and running in the qemu environment

I tested with the two commits and the errors of using nightly flag and custom target specs were not seen. Testing was completed for the test suites like ui, run-pass-valgrind, coverage, mir-opt, codegen, assembly, incremental.

But, after that while coping libs to qemu emulator(we run the test suite on qemu), following io::copy panic was observed:

 REMOTE copy libs to emulator (x86_64-poky-linux-gnu)                                                                          
 Building stage0 tool remote-test-server (x86_64-unknown-linux-gnu -> x86_64-poky-linux-gnu)
 
                        
 done pushing "/home/build-st/tmp/work/core2-64-poky-linux/rust/1.75.0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-poky-linux-gnu/lib/libcstr-d49e684941d617ce.so"                                   
 done pushing "/home/build-st/tmp/work/core2-64-poky-linux/rust/1.75.0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-poky-linux-gnu/lib/libzerovec_derive-52a1b6c914833752.so"                         
 done pushing "/home/build-st/tmp/work/core2-64-poky-linux/rust/1.75.0/rustc-1.75.0-src/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-poky-linux-gnu/lib/librustc_fluent_macro-4b4331439c3bd942.so"
 
thread 'main' panicked at src/tools/remote-test-client/src/main.rs:353:5: 
io::copy(&mut file, dst) failed with Connection reset by peer (os error 104)                                                  
 stack backtrace:                                                                                                              
    0: rust_begin_unwind                                                                                                       
              at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597:5                            
    1: core::panicking::panic_fmt                                                                                              
              at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72:14                           
    2: remote_test_client::send                                                                                                
    3: remote_test_client::main                                                                                                
 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.                                       
 Build completed unsuccessfully in 0:09:27

Is the error always the same, like this?

@Yashinde145
Copy link
Author

Yashinde145 commented Jan 15, 2024

Yes, the error is always same.

@onur-ozkan
Copy link
Member

Can you try this patch onur-ozkan@43183f7 and tell me if it fixes the problem?

@Yashinde145
Copy link
Author

The onur-ozkan@43183f7 patch does not fix the problem. Still, the same error is seen.

@onur-ozkan
Copy link
Member

I was able to repreduce the io::copy(&mut file, dst) failed with Connection reset by peer (os error 104) error, but only by sending large files (and the patch I linked fixes it).

Unfortunately I don't know your use case, and I can't invest time on #115642 (comment) for debugging or trying to reproduce the error. If you can't debug and fix the problem you get, I suggest you to give more context about your use case (configurations, commands, etc.) under the #97669 issue.

@Yashinde145
Copy link
Author

I got some work around for this,
by excluding ui-fulldeps test suite and src/tools/replace-version-placeholder, src/tools/jsondoclint tests and it works for x86_64-poky-linux-gnu.

But, when I check for other cross-compiled targets(x86 32 bit, arm 32 & 64 bits and mips64), the following lib failure is seen

error: linking with `cc` failed: exit status: 1

build-st/tmp/hosttools/ld:  build-st/tmp/work/core2-32-poky-linux/rust/1.74.1/rustc-1.74.1-src/build/x86_64-unknown-linux-gnu/stage1-std/i686-poky-linux-gnu-synthetic-miropt-abort/release/deps/std-55f86614dec482ec.3j457j23godxdmvr.rcgu.o: file class ELFCLASS32 incompatible with ELFCLASS64
build-st/tmp/hosttools/ld: final link failed: file in wrong format

collect2: error: ld returned 1 exit status
error: could not compile `std` (lib) due to previous error

I am investigating the problem and will let you know if there are any updates.

@onur-ozkan
Copy link
Member

That seems to be related with MirOptPanicAbortSyntheticTarget tests

let panic_abort_target = builder.ensure(MirOptPanicAbortSyntheticTarget {
compiler: self.compiler,
base: target,
});
run(panic_abort_target);

@Yashinde145
Copy link
Author

Yes, the log says the same and the error was seen after MirOptPanicAbortSyntheticTarget test cases while building stage1 library artifacts.

Meanwhile, I came across the following and have a doubt,
As mentioned in #40049 (comment), do we have to pass the -C linker= flag to RUSTFLAGS explicitly for the linker error in #115642 (comment)?

Earlier it was working without passing the flag explicitly.

@onur-ozkan
Copy link
Member

Earlier we didn't have those tests, so they might require passing -C linker=. You can try and see if that solves the problem.

@Yashinde145
Copy link
Author

That seems to be related with MirOptPanicAbortSyntheticTarget tests

let panic_abort_target = builder.ensure(MirOptPanicAbortSyntheticTarget {
compiler: self.compiler,
base: target,
});
run(panic_abort_target);

Is there any way to handle this panic failure and make the tests run successfully?

@onur-ozkan
Copy link
Member

You can add --skip mir-opt to your command to exclude mir-opt tests, then run x test mir-opt to run mir-opt tests without MirOptPanicAbortSyntheticTarget.

lnicola pushed a commit to lnicola/rust-analyzer that referenced this issue Apr 7, 2024
mir-opt and custom target fixes

From rust-lang/rust#115642 (comment)

> > Could you please test the last two commits from https://github.com/onur-ozkan/rust/commits/panic-abort-mir-opt when you have the time? The first commit should resolve the error of using the nightly flag with a stable compiler, and the second one should resolve the custom target issue.

> I tested with the two commits and the errors of using nightly flag and custom target specs were not seen.
Testing was completed for the test suites like ui, run-pass-valgrind, coverage, mir-opt, codegen, assembly, incremental.

Fixes #115642
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this issue Apr 27, 2024
mir-opt and custom target fixes

From rust-lang/rust#115642 (comment)

> > Could you please test the last two commits from https://github.com/onur-ozkan/rust/commits/panic-abort-mir-opt when you have the time? The first commit should resolve the error of using the nightly flag with a stable compiler, and the second one should resolve the custom target issue.

> I tested with the two commits and the errors of using nightly flag and custom target specs were not seen.
Testing was completed for the test suites like ui, run-pass-valgrind, coverage, mir-opt, codegen, assembly, incremental.

Fixes #115642
daregit pushed a commit to daregit/yocto-combined that referenced this issue May 22, 2024
After rust is upgraded to 1.71 and later versions, the rust oe-selftest gives
build errors due to unstable nightly options(see the error mentioned below). Thus, disable the test suite
until the issue is fixed

error: the option `Z` is only accepted on the nightly compiler
thread 'main' panicked at 'failed to gather the target spec for x86_64-poky-linux-gnu', synthetic_targets.rs:66:9

Following issues are created in Yocto bugzilla and rust upstream to track this issue-

https://bugzilla.yoctoproject.org/show_bug.cgi?id=15275
rust-lang/rust#115642
https://users.rust-lang.org/t/does-rust-test-suite-supports-nightly-options-during-bootstrapping-in-rust/103108
https://users.rust-lang.org/t/unable-to-read-target-specs-when-rust-1-73-is-bootstrapped-in-yocto-poky/102959

(From OE-Core rev: fa2d3cda8f7f932c0ed5c534c4bc820af2761cd5)

Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants