Keystone Enclave support on Starfive Visionfive 2

[dzourn]

Are there plans in order to support Keystone on the Starfive VisionFive 2 SBC?

Starfive Opensbi fork issue expressing interest on Keystone.

I have also created a thread on RVSpace in order to initiate discussions and describe some small steps I have managed to make.

[jarkkojs]

Attestation flow could not be supported because lack of hardware root of trust but I guess you could make fairly efficient simulation, if it has pmpcfg* and pmpaddr* CSRs. So the question would could the attestation be simulated with software.

[jarkkojs]

AFAIK, this would also need custom OpenSBI but I have no idea if you can use such in the SBC in question. Interested tho.

[dayeol]

@msanft should be working on this.

[msanft]

As Dayeol already mentioned, I'm currently working on deploying a Keystone system onto the board. I will update you regularly here

[jarkkojs]

@msanft just for heads up, is it possible to use your custom SBI (i.e. custom SM and BLL) in the board? I'm testing further modified SM from Keystone. Thus, the interest. And I happen to own this board.

[msanft]

Hello @jarkkojs

First, as a disclaimer, I only did very shallow research on this so far and cannot guarantee anything of this works (especially out of the box).

I think we can replace the whole boot chain, except for their Firmware / Boot ROM. There is a bootloader recovery binary from StarFive which should allow to replace the U-Boot SPL and the U-Boot / OpenSBI, which can be built as described here. This should allow you to also boot a custom SM.

I flash the board with these commands, assuming picocom being installed and a serial connection of the board on /dev/ttyUSB0:

picocom -b 115200 -s "sz -X" /dev/ttyUSB0

# Wait for board output (CCCCC...) to show up
# Ctrl-A + Ctrl-S
# Send path to recovery binary
/path/to/recovery-binary
# Wait for "Transfer complete" to show up

# Wait for board output (CCCCC...) to show up
# Ctrl-A + Ctrl-S
# Send path to U-Boot SPL
/path/to/u-boot-spl
# Wait for "Transfer complete" to show up

# Wait for recovery binary menu to show up
# Select Option 2 (by sending "2")
2
# Wait for board output (CCCCC...) to show up

# Ctrl-A + Ctrl-S
# Send path to U-Boot/OpenSBI binary (flashing it to the board)
/path/to/u-boot-binary
# Wait for "Transfer complete" to show up

[jarkkojs]

Hello @jarkkojs

First, as a disclaimer, I only did very shallow research on this so far and cannot guarantee anything of this works (especially out of the box).

I think we can replace the whole boot chain, except for their Firmware / Boot ROM. There is a bootloader recovery binary from StarFive which should allow to replace the U-Boot SPL and the U-Boot / OpenSBI, which can be built as described here. This should allow you to also boot a custom SM.

I flash the board with these commands, assuming picocom being installed and a serial connection of the board on /dev/ttyUSB0:

picocom -b 115200 -s "sz -X" /dev/ttyUSB0

# Wait for board output (CCCCC...) to show up
# Ctrl-A + Ctrl-S
# Send path to recovery binary
/path/to/recovery-binary
# Wait for "Transfer complete" to show up

# Wait for board output (CCCCC...) to show up
# Ctrl-A + Ctrl-S
# Send path to U-Boot SPL
/path/to/u-boot-spl
# Wait for "Transfer complete" to show up

# Wait for recovery binary menu to show up
# Select Option 2 (by sending "2")
2
# Wait for board output (CCCCC...) to show up

# Ctrl-A + Ctrl-S
# Send path to U-Boot/OpenSBI binary (flashing it to the board)
/path/to/u-boot-binary
# Wait for "Transfer complete" to show up

As a disclaimer I just put that if I blow up my house by running this, the blame is on my side :-) So no worries, I get that it is 100% stable. Thanks a bunch!

[dzourn]

Hello from my side!

I managed to run u-boot spl -> opensbi/sm -> u-boot proper (I don't remember if I managed to boot a kernel/rootfs - but I am sure I did not run any example).

What I did was:

• got this repo
• remove extern variables from sm.c
• compiled keystone/sm with FW_TEXT_START=0x40000000
• compiled VisionFive u-boot with OPENSBI=/path/to/fw_dynamic.bin
• followed instructions on how to build and flash the SD card from VF2 repo

Note1: that I have not tested it for over 2 months so I am not entirely sure on whether this still will work.

Here is the result:

Note2: Of course we do not have the keys setup from bootrom(and linker script) so the sm/sanctum keys are zero.
Note3: If I remember correctly when I tried to launch the kernel - it panicked, but I could not resolve the issue.
Note4: this is more or less the boot/built process that I followed.

Please feel free to ask me any question!

[msanft]

Hey @dzourn,

Thank you very much, this already sounds very helpful. I will try to get this set up this weekend and try to debug the kernel panic if it should happen.

[its-luca]

Hey @ALL

I stumbled over this issue when trying to setup Keystone on the Visionfive2. I had some trouble following the manual from @dzourn . Especially with the step

• compiled VisionFive u-boot with OPENSBI=/path/to/fw_dynamic.bin

For me, this always succeeds, even if the file passed to OPENSBI does not exist at all.
This does not seem correct.

However, I found that you can simply replace the opensbi folder in the VisionFive2 SDK Repo with the sm folder from Keystone. After slightly patching the Makefile, you can then use the standard build from the VisionFive2 SDK repo.

diff --git a/Makefile b/Makefile
index 161a79a..577770b 100644
--- a/Makefile
+++ b/Makefile
@@ -278,8 +278,18 @@ linux-menuconfig: $(linux_wrkdir)/.config
 $(sbi_bin): $(uboot) $(vmlinux)
 	rm -rf $(sbi_wrkdir)
 	mkdir -p $(sbi_wrkdir)
-	cd $(sbi_wrkdir) && O=$(sbi_wrkdir) CFLAGS="-mabi=$(ABI) -march=$(ISA)" ${MAKE} -C $(sbi_srcdir) CROSS_COMPILE=$(CROSS_COMPILE) \
-		PLATFORM=generic FW_PAYLOAD_PATH=$(uboot) FW_FDT_PATH=$(uboot_dtb_file) FW_TEXT_START=0x40000000
+	$(sbi_srcdir)/opensbi/scripts/carray.sh -i $(sbi_srcdir)/plat/generic/platform_override_modules.carray > ${sbi_srcdir}/plat/generic/platform_override_modules.c
+	cd $(sbi_wrkdir) && O=$(sbi_wrkdir)  ${MAKE} -C $(sbi_srcdir)/opensbi CROSS_COMPILE=$(CROSS_COMPILE) \
+		PLATFORM_DIR=${sbi_srcdir}/plat \
+		PLATFORM=generic \
+		FW_PAYLOAD_PATH=$(uboot) \
+		FW_FDT_PATH=$(uboot_dtb_file) \
+		FW_TEXT_START=0x40000000 \
+		FW_PAYLOAD=y \
+		PLATFORM_RISCV_XLEN=64 \
+		PLATFORM_RISCV_ISA=rv64imafdc_zifencei \
+		PLATFORM_RISCV_ABI=lp64d
+

 $(fit): $(sbi_bin) $(vmlinux_bin) $(uboot) $(its_file) ${initramfs}
 	$(uboot_wrkdir)/tools/mkimage -f $(its_file) -A riscv -O linux -T flat_dt $@
--

Strangely this worked, without incorporating the VisionFive2 specific opensbi code

To me it looks like this might be the easier route to provide Keystone support for the Visionfive2, as this could be implemented by simply pointing the opensbi submodule to the keystone repo

[jarkkojs]

Hello @jarkkojs
First, as a disclaimer, I only did very shallow research on this so far and cannot guarantee anything of this works (especially out of the box).
I think we can replace the whole boot chain, except for their Firmware / Boot ROM. There is a bootloader recovery binary from StarFive which should allow to replace the U-Boot SPL and the U-Boot / OpenSBI, which can be built as described here. This should allow you to also boot a custom SM.
I flash the board with these commands, assuming picocom being installed and a serial connection of the board on /dev/ttyUSB0:

picocom -b 115200 -s "sz -X" /dev/ttyUSB0

# Wait for board output (CCCCC...) to show up
# Ctrl-A + Ctrl-S
# Send path to recovery binary
/path/to/recovery-binary
# Wait for "Transfer complete" to show up

# Wait for board output (CCCCC...) to show up
# Ctrl-A + Ctrl-S
# Send path to U-Boot SPL
/path/to/u-boot-spl
# Wait for "Transfer complete" to show up

# Wait for recovery binary menu to show up
# Select Option 2 (by sending "2")
2
# Wait for board output (CCCCC...) to show up

# Ctrl-A + Ctrl-S
# Send path to U-Boot/OpenSBI binary (flashing it to the board)
/path/to/u-boot-binary
# Wait for "Transfer complete" to show up

As a disclaimer I just put that if I blow up my house by running this, the blame is on my side :-) So no worries, I get that it is 100% stable. Thanks a bunch!

So I just took my VisionFive 2 "out from the attic" and started by flashing the latest U-boot and OpenSBI (just the proper ones, not Keystone ones). For that I used tftp method but for a live system there is also flashcp method. See for both: https://jamesachambers.com/starfive-visionfive-2-firmware-update-guide/.

So while the procedure describe above is correct, I'd figure that for a non-bricked system you can achieve the same with either tftp or flashcp method, and not have to switch boot mode jumpers to the UART, right? I still like to use UART for serial console so updating like this would be way too much effort.

Not criticizing, I'm just sanity checking that my assumptions are correct :-)

Also since OpenSBI is included in the payload binary, by using two other routes, you can I guess do only the 2nd flash most of the time, unless you need to update U-boot? Or does U-boot have compiled in signature for OpenSBI? Just thinking about this because it would further simplify the process, and wear off less the internal flash.

One thing I wonder is that fw_payload.bin built by Keystone project has also kernel with initramfs rootfs with it but AFAIK the one built by this project does not. Kernel image is a separate file, right? Just spotting a difference, that's all.

[jarkkojs]

All of the methods are sub-optimal in the sense that they require re-flash u-boot and/or opensbi. Optimal (for development) would be a method, where for a single power-cycle (boot), one could put the system live with custom images without flashing. Of course, they would be gone after power down, but that is kind of the point.

Maybe this would be a good issue for Starfiv VisionFive 2 repository as it is a generic problem spanning for any low-level experiementation with the SBC in the discussion.

[jarkkojs]

I updated my issue over there. That is a bit more interesting issue across the board (not just Keystone), right?

And worth of reporting to the level that even if this SBC cannot handle it, perhaps a future SBC could have a jumper configuration or something to put the board to a mode where the whole bootchain is loaded for every single boot.

It sort of enhances the prototyping capabilities in many levels. E.g. let's say you have RISC-V extension and opcode's are implemented only on FPGA. If you can easily edit-compile-run your own OpenSBI, then you have capabilities to easily emulate those opcodes in the trap handler, and use an off-the-shelf ASIC board for that purpose.