Comments for "https://os.phil-opp.com/minimal-rust-kernel/"

[utterances-bot]

This is a general purpose comment thread for the “A Minimal Rust Kernel” post.

[Redrield]

I'm not really a fan of using this magic utility to turn our blob of Rust into a bootable image. The parts in raw assembly in the first edition were a bit tedious but I liked how they gave us a deeper understanding about how the process worked. I hope that this utility won't be sticking around for the post on UEFI booting...

[pixelherodev]

"They are designed to make the bootloader simple instead of the kernel. For example, the kernel needs to be linked with an adjusted default page size, because GRUB can't find the Multiboot header otherwise."

WRONG!

That is true of Multiboot2, a later standard which is less used and NOT multiboot-compatible.

However, the actual Multiboot standard does not have that problem at all.

[phil-opp]

@Redrield Fair point. On the other hand we already used grub-mkrescue as such a magic utility in the first edition, which did exactly the same thing (transforming a Rust binary into a bootable image). The only difference was that GRUB only put us into protected mode, so we had to do the last steps ourselves.

Apart from a less tedious start, the new bootloader gives us an opportunity for an even deeper understanding of the boot process than before: We're planning to write several posts on how the tool and the underlying bootloader works, in a similar fashion as the other posts (i.e. every line of code is in the post and you can follow along). Thus, we can hide the complexity at the beginning of the main post series and directly start with the more interesting part, but also provide resources to those who want to understand it all the way down.

[phil-opp]

@pixelherodev Huh? I was under the impression that Multiboot2 was the successor of Multiboot? Either way, Multiboot 1 is not a better solution for us since it does have other problems (if I remember correctly it does not support 64-bit ELF files at all).

[pixelherodev]

Fair enough. Multiboot 2 is essentially the spiritual successor to multiboot IIRC. I also remember multiboot being easier to get started with, but that's neither here nor there :p Honestly, while I don't think ignoring multiboot was a good idea, I can understand why you did so.

[phil-opp]

Honestly, while I don't think ignoring multiboot was a good idea, I can understand why you did so.

Yeah, there are clear tradeoffs in this case. I really like the idea behind multiboot, but the standard and the implementation not so much. I plan to add grub compatibility to the bootimage tool, so that you can boot it in a multi-OS setting too.

[ghost]

First of all, great tutorial, thanks a lot for the effort!
If you're on Ubuntu, like me, and building openssl-sys when trying to cargo install bootimage fails, make sure you have libssl-dev installed!

[phil-opp]

@FuzzyHerbivore Thanks a lot! I added a hint to the post in 5f195a8.

[kylegalloway]

Disclaimer: I'm on Windows.

There is a problem with running qemu-system-x86_64 -drive format=raw,file=bootimage.bin on windows. QEMU opens and flashes but doesn't display the "Hello World!" text.

Also, converting to a VDI for VirtualBox doesn't seem to work. I get:

$ VBoxManage convertdd .\bootimage.bin bootimage.vdi --format VDI
Converting from raw image file=".\bootimage.bin" to file="bootimage.vdi"...
Creating dynamic image with size 86016 bytes (1MB)...
VBoxManage.exe: error: Cannot create the disk image "bootimage.vdi": VERR_VD_INVALID_SIZE

[AaronCoad]

@kyle I'm also using Windows with cargo 1.26.0 nightly and xargo 0.3.11. Are these the same versions for you? I'm stuck with building using xargo:

xargo build --target hyenaos results in error: multiple input filenames provided
set RUST_TARGET_PATH=cd xargo build --target hyenaos results in nothing (including no build folder) as does RUST_TARGET_PATH=cd xargo build --target hyenaos.

Using cargo 1.26.0 nightly and xargo 0.3.11.

[AaronCoad]

@kylegalloway sorry

[AaronCoad]

Feeling a little silly, still stuck on the above, but realised I was setting the path incorrectly. The following steps still fail:

set RUST_TARGET_PATH=C:\HyenaOS\ or set RUST_TARGET_PATH=C:\HyenaOS
xargo build --target hyenaos or xargo build --target C:\HyenaOS\hyenaos

These are always with the multiple input filenames provided error.

Apologies for the multiple posts.

[phil-opp]

@kylegalloway

There is a problem with running qemu-system-x86_64 -drive format=raw,file=bootimage.bin on windows. QEMU opens and flashes but doesn't display the "Hello World!" text.

I pushed a new version of the bootloader yesterday, seems like this is the cause of the error. I reverted the change for now, so it should work again. Thanks for reporting!

Also, converting to a VDI for VirtualBox doesn't seem to work. I get:

Hmm, I have no idea about this error. I only found https://forum.lede-project.org/t/error-convert-img-to-vdi/152, which indicates that the image needs to be padded somehow before converting.

[phil-opp]

@AaronCoad Strange, I've never seen this error. Could you try it with --verbose?

[SomeAnotherDude]

Hi, I have a little problem with bootimage.
When I run it in debug mode and pass a created "bootimage.bin" file to qemu it fails with this:

    qemu-system-x86_64: Trying to execute code outside RAM or ROM at 0x000000018d47f075
    This usually means one of the following happened:

    (1) You told QEMU to execute a kernel for the wrong machine type, and it crashed on startup (eg trying to run a raspberry pi kernel on a versatilepb QEMU machine)
    (2) You didn't give QEMU a kernel or BIOS filename at all, and QEMU executed a ROM full of no-op instructions until it fell off the end
    (3) Your guest kernel has a bug and crashed by jumping off into nowhere

    This is almost always one of the first two, so check your command line and that you are using the right type of kernel for this machine.
    If you think option (3) is likely then you can try debugging your guest with the -d debug options; in particular -d guest_errors will cause the log to include a dump of the guest register state at this point.

    Execution cannot continue; stopping here.

But when I run it with --release flag everything works properly. Why could it happen?

.S my host OS is Kubuntu 17.10

[phil-opp]

@SomeAnotherDude Sounds like a problem in the bootloader. Do you have your project online somewhere so that I can try to reproduce?

[AaronCoad]

@phil-opp I gave it another whirl today and identified the issue and alerted xargo's developers to it. On my main machine my windows user account name has a space in it, so the folder path to .xargo also contains a space. Because the folder path isn't passed through to --systools with surrounding double-quotes it was picking up the folder path as being 2 separate paths. Building on secondary machine with no spaces in the username was successful.

Qemu reportedly has a bug for Windows as well in their latest version that was identified on March 22 and they're pushing a fix into rc1. http://lists.gnu.org/archive/html/qemu-devel/2018-03/msg05484.html

@kylegalloway now that I'm at that point, with VirtualBox 5.2.8 you get a more verbose error of "The given disk size 85992 is not aligned on a sector boundary (512 bytes)".

[SomeAnotherDude]

@phil-opp Uploaded to https://github.com/SomeAnotherDude/blog_os

[SomeAnotherDude]

@phil-opp It's a miracle! when I clone this repo from github and try to reproduce... It just works...

[SomeAnotherDude]

There was something wrong with ${progect_root}/target directory. Just after I deleted it and rebuilt the project the problem is gone.

[phil-opp]

@SomeAnotherDude Ah, it's probably because the bootloader is cached in the target directory. So it continued to use the old (broken) bootloader for debug and the new fixed version for release. Sorry again for the breakage.

@AaronCoad Good job in pinpointing the xargo bug! For reference, the error is tracked in japaric/xargo#206. Unfortunately, xargo is in maintanance mode, so the author probably won't invest time in fixing this. However, they said they are still willing to merge PRs, so if you don't have time I could try to give it a shot.

@AaronCoad @kylegalloway

with VirtualBox 5.2.8 you get a more verbose error of "The given disk size 85992 is not aligned on a sector boundary (512 bytes)".

Oh, this is something we can fix in bootimage.

[phil-opp]

@AaronCoad @kylegalloway I updated bootimage in rust-osdev/bootimage@ba6f206 to produce aligned disk images. It is on crates.io as version 0.2.0-alpha-006. You can reinstall the latest version via cargo install bootimage --force. Could you try if this fixes the VirtualBox boot?

[phil-opp]

@AaronCoad I opened japaric/xargo#207 for the xargo bug.

[AaronCoad]

@phil-opp thankyou. I've tested the bootimage change and it is now just the invalid size error. Based on some other reported instances, it looks like it may be that VirtualBox looks for a minimum size. I reckon once it gets over that size it should convert no worries.

In the meantime, I've just setup an Ubuntu VM so that I can use qemu and look at translating and merging some of the older sections into what I've got.

[kylegalloway]

@phil-opp The new bootimage worked for me on the qemu command. Thanks for the help!

[phil-opp]

@AaronCoad @kylegalloway Cool, thanks for testing!

[AbleTheAbove]

I'm getting an error when I run cargo build --target target.json (I named the target file target)
The error is this

  = note: /usr/bin/ld: cannot find Scrt1.o: No such file or directory
          /usr/bin/ld: cannot find crti.o: No such file or directory
          collect2: error: ld returned 1 exit status
error: aborting due to previous error
error: could not compile `compiler_builtins`

[phil-opp]

@AbleTheAbove Are you sure that you're using the #![no_std] attribute for your crate? If that doesn't fix your issue, it would be helpful if you could upload your code somewhere so that we can take a look at it.

[delta1]

"target may not be installed"

when running cargo build --target x86_64-blog_os.json initially you may get the above error first/instead of "can't find crate for core"

to resolve: continue with build-std config and run rustup component add rust-src

[bishtpawan]

I'm getting this error

error[E0463]: can't find crate for `std`
  |
  = note: the `x86_64-os-in-rust-12704004753539460791` target may not be installed

Can anyone please help me out to resolve this error?
Thanks

[delta1]

@bishtpawan ^^ comment right above yours.

continue with the build-std steps that follow in the blog, and also run rustup component add rust-src

[bishtpawan]

Thanks, @delta1 for your response, I updated the rustup and the problem is resolved now.

[OscarTHZhang]

I am getting this error when running qemu-system-x86_64 on Mac OS:

dyld: Library not loaded: /usr/local/opt/nettle/lib/libnettle.6.dylib
  Referenced from: /usr/local/bin/qemu-system-x86_64
  Reason: image not found
[1]    29819 abort      qemu-system-x86_64 -drive

[bishtpawan]

Hi @OscarTHZhang, well I'm not sure about MAC but can you share the command that you are using to run your bootable image in Qemu?

[bjorn3]

It lookslike the libnettle dependency of qemu is not installed on your system.

[OscarTHZhang]

Hi @OscarTHZhang, well I'm not sure about MAC but can you share the command that you are using to run your bootable image in Qemu?

This is the command I ran:

qemu-system-x86_64 -drive format=raw,file=target/x86_64-blog_os/debug/bootimage-blog_os.bin

[OscarTHZhang]

It lookslike the libnettle dependency of qemu is not installed on your system.

I checked my system and I do have qemu-system-x86_64 in my /usr/local/bin directory. I am not sure what is the "libnettle dependency". I also searched on Google and there are similar issue like this one: https://stackoverflow.com/questions/17703510/dyld-library-not-loaded-reason-image-not-found , but I was still not able to solve this.

[delta1]

@OscarTHZhang try brew install nettle

[OscarTHZhang]

@OscarTHZhang try brew install nettle

Nice. I reinstall nettle and qemu and then it works! Thank you so much!

[wooster0]

If you are interested in building your own bootloader: Stay tuned, a set of posts on this topic is already planned!

Hello, thank you for this great series. Is this still planned? Or is this perhaps depended on rust-osdev/bootloader#24?

[atultw]

For windows people like myself:

The process for creating bootimage worked as written in the guide. I didn't have qemu so I chose to test with virtualbox:

• Add your virtualbox installation directory to path so we can use the VBoxManage utility. (for me it was C:\Program Files\Oracle\VirtualBox)
• Virtualbox utility may fail to convert because of the image size. I went into WSL and used qemu-img resize YOURFILENAME.bin 1M which fixed the issue. (the command may also work with qemu for windows but I don't know)
• From cmd or powershell, navigate to the directory with your .bin image and run VBoxManage convertfromraw YOURFILENAME.bin disk.vdi --format VDI. This converts the raw bin into a disk image usable by vbox.
• I set the OS type as redhat in virtualbox and it worked. This was necessary since selecting unknown raised an error about long mode at boot.

Hope this helps someone!

[google0101-ryan]

I followed the instructions, installed rust-src, and ran the build after cloning the repository, and I still get an error that the crate for std cannot be found. It only happens when I try to install bootimage.

[phil-opp]

@r00ster91 I already started writing about creating an UEFI bootloader in the upcoming third edition of the blog. I'm not sure when there will be post about BIOS booting, but it probably makes sense to wait for rust-osdev/bootloader#24 as it will simplify some things.

[phil-opp]

@hello01-debug

I followed the instructions, installed rust-src, and ran the build after cloning the repository, and I still get an error that the crate for std cannot be found. It only happens when I try to install bootimage.

Could you post a link to your code so that we can look into your issue?

[google0101-ryan]

Sure, as soon as I can.

…

________________________________ From: Philipp Oppermann ***@***.***> Sent: Tuesday, April 27, 2021 5:17 AM To: phil-opp/blog_os ***@***.***> Cc: Ilari, Ryan Nathaniel ***@***.***>; Comment ***@***.***> Subject: Re: [phil-opp/blog_os] Comments for "https://os.phil-opp.com/minimal-rust-kernel/" (#403) [EXTERNAL EMAIL - Please use proper judgment and caution when opening attachments, clicking links, or responding to this email.] @hello01-debug<https://github.com/hello01-debug> I followed the instructions, installed rust-src, and ran the build after cloning the repository, and I still get an error that the crate for std cannot be found. It only happens when I try to install bootimage. Could you post a link to your code so that we can look into your issue? — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#403 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANPNOP73YB7PDJTA2YA6YPTTKZ6LXANCNFSM4EUUEX3A>.

[kennystrawnmusic]

Starting from scratch and noticing this problem with bootloader 0.10.2 ― host OS is Fedora Rawhide and Rust version is 1.54.0-nightly (4de757209 2021-05-01):

Error: An error occured while trying to build the bootloader: The `bootloader` dependency has not the right format: No `package.metadata.bootloader.target` key found in Cargo.toml of bootloader

Particularly bizarre since I'm following this to the T and it worked fine last year. Oh, and obviously 0.10.2 > 0.5.1.

[kennystrawnmusic]

This blog post needs to be updated for bootloader 0.10.x. Causing way too many problems. Case in point, after following the updated docs.rs/bootloader instructions:

--- stderr
  thread 'main' panicked at 'The UEFI bootloader must be compiled for the `x86_64-unknown-uefi` target.'

This is from attempting to add uefi_bin as a feature requirement to the bootloader crate dependency. Meanwhile if I don’t do that, builder fails with this message:

error: none of the selected packages contains these features: uefi_bin

So either there’s a bug in the code that’s looking for the wrong target or this post needs to be fundamentally overhauled.

[bjorn3]

@phil-opp is currently working on a third edition of blog os that uses bootloader 0.10.x. This is taking a while as there are major changed between 0.9.x and 0.10.x to support not just BIOS booting, but also UEFI. In addition UEFI requires APIC (advanced programmable interrupt controller) instead of the legacy PIC that is currently used by blog os. For now you may want to stay on bootloader 0.9.x.

[kennystrawnmusic]

Alright, guess I’ll just have to patiently wait until that new edition comes out, thanks.

[InterestingBrainPoops]

Could you post a link to your code so that we can look into your issue?
Ive run into the same issue, here is the link to the gh repo: https://github.com/InterestingBrainPoops/learning/tree/master/alcolu/alcolu

[InterestingBrainPoops]

wait shoot, the quotes went down too far. But yea, I encountered the same problem, and I put the src on github at the link above
cc: @phil-opp

[phil-opp]

@InterestingBrainPoops I just tried your code and it compiled fine using the following build command:

cargo build --target x86_64-alcolu.json -Z build-std=core -Z build-std-features=compiler-builtins-mem

Instead of these command line parameters, you can also create a .cargo/config.json file as described in the post.

[Sevendaye]

Starting from scratch and noticing this problem with bootloader 0.10.2 ― host OS is Fedora Rawhide and Rust version is 1.54.0-nightly (4de757209 2021-05-01):

Error: An error occured while trying to build the bootloader: The `bootloader` dependency has not the right format: No `package.metadata.bootloader.target` key found in Cargo.toml of bootloader

Particularly bizarre since I'm following this to the T and it worked fine last year. Oh, and obviously 0.10.2 > 0.5.1.

Hi,

I had the same problem, I was able to solve it by:

• bootloader = "0.9.18" // In Cargo.toml

$cargo clean
$cargo bootimage --target .\x86_64-blog_os.json
$cargo run

It seems that there is a problem after version 0.9.18 of the bootlader crate

[ty-n-42]

Really loving reading this blog. For anyone else who is a bit nervous about using brew/macports on MacOS (to get qemu), I managed to get qemu working in an Ubuntu guest with VMware Fusion. Setting up VS Code for Rust in Ubuntu is also simple; so it ends up being a nice developer experience.