Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Boot on real RISC-V hardware - the MangoPi MQ-Pro #99

Merged
merged 23 commits into from
Nov 12, 2023
Merged

Boot on real RISC-V hardware - the MangoPi MQ-Pro #99

merged 23 commits into from
Nov 12, 2023

Conversation

IsaacWoods
Copy link
Owner

@IsaacWoods IsaacWoods commented Oct 26, 2023

Closes #98

This PR builds out support for the desired development workflow for the MangoPi MQ-Pro - using xfel to load code from a host computer, which will also handle DRAM initialization etc.

Booting from persistent media (SD-card or flash) will require more support, namely an initial boot shim that can initialize the DRAM and load further boot stages from the media (this PR will likely contain the start of this package (d1_boot0) - this was because I initially thought we'd need this shim on xfel-booted systems too, but looks like we won't).

  • Boot the board from xfel, make sure serial output works etc.
  • Bring OpenSBI in-tree and support building it from xtask
  • Separate out platforms for RISC-V on QEMU and the MQ-Pro
  • Fix CI for the new platforms
  • xtask support for compiling device trees
  • Find/amalgamate a working device tree for the MQ-Pro
  • Get upstream OpenSBI booting against our device tree (XXX: currently requiring hacks)
  • Use correct linker script for platform (maybe using Cargo features to select platform)
  • Seed: configure the UART driver based on FDT description
  • Seed: abstract driver support for Virtio vs ramdisk (and future SD-card etc.) via platform features

Before merging:

  • Rebase fix CI commits into original breakages
  • Make cargo xtask opensbi respect config file platform
  • Clean-up and commit initial work on d1_boot0

@IsaacWoods IsaacWoods marked this pull request as draft October 26, 2023 22:10
…ld it

We had this in-tree a while ago and got rid of it because QEMU ships with
its own version. However, now we're building for hardware, we need a way
to supply it, and it seems sensible to have a nice `xtask` command to build
it. In the future this can be extended to know how to build OpenSBI for
each platform.
This is just intended to prove that the D1 + FEL etc actually does work,
because naively uploading OpenSBI and running it does not produce output
(probably because it doesn't know where the UART is without a device tree
lmao). It's named `d1_boot0` because it will become the boot-shim (known as
the SPL in U-Boot terminology, or `boot0` in the D1 bootloader) for both
FEL booting and booting from a persistent medium.
We were actually getting problems with squirting out too many bytes and
messing up the UART, so listen to its line control register to send bytes
at a rate it can handle. This differs from QEMU's UART in that each register
is 4-bytes instead of 1 - we need some way of dealing with all these platform
differences in the future, but for `boot0` we can just shove a new serial
definition in here.

Also includes some formatting changes bc we got nvim's auto-formatter
working again.
@IsaacWoods
Copy link
Owner Author

IsaacWoods commented Nov 10, 2023

Getting upstream OpenSBI working

The remaining issues with getting upstream OpenSBI to boot seem to be around strange changes to the address of things from 0x800xxxxx to 0x400xxxxx midway through execution. This mainly causes hangs because OpenSBI uses linked-list structures to emulate iterators, which then get stuck in infinite loops because the terminating condition is checking for what is now the wrong address.

Confusingly, building OpenSBI with FW_TEXT_START=0x8000_0000 resolves this issue (but makes everything appear at 0x400xxxxx throughout), despite being the wrong address. I suspect the D1 maps addresses at 0x8000_0000 down by ignoring the high bit, so that they mirror DRAM, so even if it does end up accessing these addresses it produces correct results. This allows us to move forward for now, but it's not particularly satisfying - will need more investigation.

Working theory is something strange is going on with the early relocation logic - will need to work out what it's doing.

Edit:
This is amazingly dumb. For some reason, make clean PLATFORM=generic doesn't seem to clean up the generated linker scripts correctly, and so we had a linker script generated with FW_TEXT_START=0x8000_0000 lying around, and that was actually being used despite specifying the new address. Manually deleting all of bundled/opensbi/build and rebuilding seems to be the way around this. We can now boot with upstream OpenSBI 🚀

@IsaacWoods
Copy link
Owner Author

IsaacWoods commented Nov 11, 2023

Getting the D1 booting our code through OpenSBI has been more time-consuming than I'd expected (e.g. finding a valid device tree, getting OpenSBI booted, etc.) so I think I'll reduce the scope of this PR to basically getting Seed booting (which is now largely succeeding!) and will work on further functionality in other PRs or directly into main. This is mainly because the next step (loading the kernel and user tasks through a ramdisk) can be developed more easily on the QEMU target and then brought over to the MQ-Pro.

@IsaacWoods IsaacWoods mentioned this pull request Nov 11, 2023
13 tasks
These are both RISC-V platforms, obviously, but actually need relatively
different build infra. For now, we're just completely severing the relation
between them, but I imagine in the future we can reconnect similar platforms
that'll have shared components.
The flags thing was acting strangely for some reason, which was fixed by
upgrading it. Also makes some of the code a little nicer!
The device tree we got out of U-Boot for the MQ-Pro is full of extra stuff
and also doesn't actually work... This was a real pain but we've extracted
a working version from Linux (manually sticking bits of their device-tree
system together lmao) - this provides an easy way to (mostly) correctly
pre-process and then compile a device tree in this format.
The D1's UART controller has 4-byte registers instead of one. This information
should be gathered from the device tree, but for now this just hardcodes it.
We need to load at different addresses on each platform - the easiest way
to do this seems to be duplicating the linker scripts, but this doesn't
seem super optimal. Fine for now.
Finding this was way more difficult than I anticipated...

This is clearly not entirely correct, because it won't boot on an unpatched
OpenSBI lmao. However, it's basically directly copied from the Linux DT
for the same platform, so not quite sure what's going wrong yet. Will need
more investigation.
This avoids having to open something like `minicom` separately, which is
a nice ergonomics win. Not sure if we'll need something more complex in
the future, but this is a nice start for now.
I'm not sure why the original device tree doesn't include this - a mystery
for the future because Seed needs one to boot, and we can easily just add
it here?

There are actually multiple models of the MQ-Pro that have different amounts
of RAM, so we could support that in the future.
This is obviously not required as part of the D1 boot, but does:
- Stop it from crashing the D1 when a Virtio device is not present
- Provide an abstraction we can then use for the ramdisk for FEL booting
- Provide an abstraction we can use for an SDHC driver in the future
- Hopefully move kernel loading forward for both Risc-V platforms (by
getting rid of the horrendous QEMU-memory-poking hack we're currently using)
The UART spuriously produces invalid UTF-8, presumably because of corrupted
bytes (likely due to the slightly dodgy situation with the wiring). We'll
resolder it when we have access to an iron, but for now we (and probably
for the long-term) it's probably a good idea to just accept dodgy input,
rather than hard-erroring.
For some reason we were saving and then outputting the output from building
OpenSBI - this is not the behaviour we want.
This is actually left from when we were going in a very different direction,
but reasonably might be useful at some point so we'll keep it around. We
do need to gracefully handle a platform not having a PCIe host node though.
See comment in crate. This should be ignored and is useful for now, but
will be needed when we work on booting the D1 from persistent media, so we
might as well leave it lying around.
@IsaacWoods IsaacWoods marked this pull request as ready for review November 12, 2023 16:49
@IsaacWoods IsaacWoods merged commit 57f9c9b into main Nov 12, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Boot on real RISC-V hardware!
1 participant