How do you test heads with QEMU? #516

itay-grudev · 2019-02-08T11:15:45Z

Kyle Ranking told me that when playing with heads one option to test is to simulate it with QEMU and some of you are using it. That will be much more convenient than re-flashing every time, but when I tried setting it up I couldn't.

Do you have a script or can you provide the qemu command with all the args and options you use?

ThatLurker · 2019-02-08T12:08:03Z

Is this what your looking for https://github.com/osresearch/heads-wiki/blob/master/Emulating-Heads.md

tlaurion · 2019-02-08T12:39:45Z

@itay-grudev : I think make run does it, but wasn't able to simply test it.

Emulating Heads needs to be updated. Heads doesn't include Xen anymore since kexec got fixed.

itay-grudev · 2019-02-08T12:41:19Z

I'm compiling the qemu-coreboot image right now. Will give feedback later today.

tlaurion · 2019-02-08T19:34:15Z

if anyone knows how to play around qemu in docker, that would be aweome to share.
I innocently tried here, without success.

tlaurion · 2019-02-08T22:38:23Z

Another Heads issue directs to bypass coreboot booting and instruct to boot kernel and initrd, which unfortunately misses the point of measuring coreboot.

tlaurion · 2019-02-15T06:33:20Z

@itay-grudev how were your tests? :)

itay-grudev · 2019-02-15T10:52:18Z

make run fails with:

❯ make run
qemu-system-x86_64 \
	--machine q35 \
	--serial /dev/tty \
	--bios /home/ito/Projects/heads/build/qemu-coreboot/coreboot.rom \
; stty sane
qemu-system-x86_64: Initialization of device e1000e failed: failed to find romfile "efi-e1000e.rom"

paulmenzel · 2019-02-15T10:54:49Z

On 02/15/19 11:52, Itay Grudev wrote: `make run` fails with: ``` ❯ make run qemu-system-x86_64 \ --machine q35 \ --serial /dev/tty \ --bios /home/ito/Projects/heads/build/qemu-coreboot/coreboot.rom \ ; stty sane qemu-system-x86_64: Initialization of device e1000e failed: failed to find romfile "efi-e1000e.rom" ```

As a workaround, please add `-nic none` to that call, and check, if it fixes the issue. Otherwise, you need to find the option ROM somewhere.

itay-grudev · 2019-02-15T10:59:53Z

Thanks. I was able to get it working with:

diff --git a/boards/qemu-coreboot/qemu-coreboot.config b/boards/qemu-coreboot/qemu-coreboot.config
index 9427a2a..a0520fe 100644
--- a/boards/qemu-coreboot/qemu-coreboot.config
+++ b/boards/qemu-coreboot/qemu-coreboot.config
@@ -42,4 +42,5 @@ run:
                --machine q35 \
                --serial /dev/tty \
                --bios $(build)/$(BOARD)/coreboot.rom \
+               -nic none \
        ; stty sane

itay-grudev · 2019-02-15T11:40:34Z

Do you know how can I get the GUI working? I am currently getting:

Error: Can't find usable screen

It's probably some QEMU option, I just can't find it.

tlaurion · 2019-02-16T19:09:56Z

FYI: debian-9 can install qemu tools in a TemplateVM, fedora-29 can't for the moment. It will be fixed in QubesOS 4.1.

I gave up a long time ago trying to build Heads in debian templates, and CI based on ubuntu are currently failing @osresearch.

tlaurion · 2019-02-16T19:11:08Z

Do you know how can I get the GUI working? I am currently getting:
Error: Can't find usable screen
It's probably some QEMU option, I just can't find it.

@itay-grudev : https://github.com/flammit/heads/tree/qemu-gui-init
Ping me back if that works. That should probably merged in master too.

itay-grudev · 2019-02-16T23:14:34Z

Yep. That works. I realised that some of the settings were missing, but didn't replicate them exactly. And that PR did it. Thanks a lot. I will close this and vote on your #527 PR.

@tlaurion Should we also create a PR with -nic none (diff above)?

add "-nic none" to fix #516 (comment)

tlaurion · 2019-05-12T21:35:31Z

@itay-grudev @paulmenzel Are you able to access a block device under qemu-coreboot build?
qemu-img create -f qcow2 -o size=10G ubuntu.img

Documentation source

Drive option

qemu-system-x86_64 \
	--machine q35 \
	--serial /dev/tty \
	--bios /home/user/heads/build/qemu-coreboot/coreboot.rom \
        -drive file=/media/user/Insurgo/ubuntu.img

hda standard support:

qemu-system-x86_64 \
	--machine q35 \
	--serial /dev/tty \
	--bios /home/user/heads/build/qemu-coreboot/coreboot.rom \
        -hda /media/user/Insurgo/ubuntu.img

ide-drive:
qemu-system-x86_64 --machine q35 --serial /dev/tty --bios /home/user/heads/build/qemu-coreboot/coreboot.rom -device ide-drive,bus=ide.0,drive=testhdd -drive id=testhdd,if=none,file=/media/user/Insurgo/ubuntu.img

AHCI
qemu-system-x86_64 --machine q35 -m 1G --serial stdio --bios build/qemu-coreboot/coreboot.rom -drive id=disk,file=/media/user/Insurgo/ubuntu.img,if=none,format=qcow2 -device ahci,id=ahci -device ide-drive,drive=disk,bus=ahci.0 -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0

There is no /dev/sda under Heads (debian-9 QubesOS template with qemu deployed, so no kvm). Only /sys/block/loop* and /sys/block/ram* devices.

Was wondering if I was doing something wrong or if there is a concept i'm missing?

tlaurion · 2019-05-12T22:13:48Z

@flammit used:
qemu-system-x86_64 --enable-kvm --machine q35 -m 1G --serial stdio --vga cirrus --bios build/qemu-coreboot/coreboot.rom -hda ../test.hd1.img --drive id=stick,if=none,format=raw,file=/home/flam/Downloads/Fedora-Workstation-Live-x86_64-29-1.2.iso -usb -device usb-ehci,id=ehci -device usb-tablet,bus=usb-bus.0 -device usb-storage,bus=ehci.0,drive=stick

tlaurion · 2019-05-15T15:07:48Z

Trying to find my way with qemu without KVM since QubesOS doesn't support nested virtualization.

Works on QubesOS debian-9 template based qube, on which qemu was installed on parent template.

Didn't found a way to pass devices yet, which is problematic, but I can at least test script changes without having to flash real hardware internally for everything.

gui-init requires an existing /boot. My testing requires a functional /media where oem-provisioning is found.

Create disks and required content to be tested:

cd ~/heads
dd if=/dev/zero of=initrd/boot.img bs=1M count=1
sudo mkfs.ext4 initrd/boot.img
sudo mount initrd/boot.img /mnt/
sudo touch /mnt/oem
sudo umount /mnt
dd if=/dev/zero of=initrd/media.img bs=1M count=1
sudo mkfs.ext4 initrd/media.img
sudo mount  initrd/media.img /mnt
sudo cp oem-reownership/oem-provisioning.example /mnt/oem-provisioning

Note: unfortunately, minimal size for LUKS container is 1049600 bytes, and adding a 500Mb + disk image into rom is not desirable. Wish I could find a way to test this by providing qemu-image disks to qemu without hacking my way like this, but it really seems kvm is required for this, even when "if=none" is passed, which from my understanding doesn't depend on virtio internals. Ho well.

Make those disks mounted by /initrd/init which feeds /etc/fstab correctly:

diff --git a/boards/qemu-coreboot/qemu-coreboot.config b/boards/qemu-coreboot/qemu-coreboot.config
index d670dc2..142c7f4 100644
--- a/boards/qemu-coreboot/qemu-coreboot.config
+++ b/boards/qemu-coreboot/qemu-coreboot.config
@@ -24,12 +24,12 @@ CONFIG_DROPBEAR=y
 
 #Uncomment only one of the following block
 #Required for graphical gui-init (FBWhiptail)
-#CONFIG_CAIRO=y
-#CONFIG_FBWHIPTAIL=y
+CONFIG_CAIRO=y
+CONFIG_FBWHIPTAIL=y
 #
 #text-based init (generic-init and gui-init)
-CONFIG_NEWT=y
-CONFIG_SLANG=y
+#CONFIG_NEWT=y
+#CONFIG_SLANG=y
 
 endif
 
@@ -40,15 +40,15 @@ CONFIG_LINUX_E1000=y
 
 #Uncomment only one BOOTSCRIPT:
 #Whiptail-based init (text-based or FBWhiptail)
-#export CONFIG_BOOTSCRIPT=/bin/gui-init
+export CONFIG_BOOTSCRIPT=/bin/gui-init
 #
 #text-based original init:
-export CONFIG_BOOTSCRIPT=/bin/generic-init
+#export CONFIG_BOOTSCRIPT=/bin/generic-init
 
 export CONFIG_TPM=n
 
-export CONFIG_BOOT_DEV="/dev/sda1"
-export CONFIG_USB_BOOT_DEV="/dev/sdb1"
+export CONFIG_BOOT_DEV="/boot.img"
+export CONFIG_USB_BOOT_DEV="/media.img"
 
 #run: coreboot.intermediate
 run:

build and test run:
make BOARD=qemu-coreboot && ./build/make-4.2.1/make BOARD=qemu-coreboot run

tlaurion · 2019-05-15T15:31:44Z

@osresearch :

It also seems that from Gentoo documentation that it would be possible to pass host random stream by adding: -object rng-random,id=rng0,filename=/dev/urandom -device virtio-rng-pci,rng=rng0

tlaurion · 2019-10-06T18:55:25Z

To be tested:
https://github.com/tpm2-software/tpm2-tcti-uefi/blob/master/lib/efi-test-setup.sh

tlaurion · 2020-04-13T02:30:55Z

As pointed in #700 (comment) by @orangecms :
-netdev user,id=u1 -device e1000,netdev=u1 maps the e1000 to host interface and works.

Should be fixed in qemu-coreboot
@flammit : have a problem with that fix?
Edit: implemented in #707

tlaurion · 2020-04-23T05:53:13Z

So qemu-coreboot TPM support, which would rely on swtpm support which depends on libtpms support seems possible as dependencies of the qemu-coreboot board, but would be linked to a totally different build path depending if on fedora build system or debian (debian not providing libtpms packages for some reason, requiring to build everything from source)

@orangecms: any success that path?

tlaurion · 2020-05-05T18:40:30Z

A lot of documentation lies in this ticket for PR propositions. Reopening.

pietrushnic · 2020-05-14T23:02:55Z

So qemu-coreboot TPM support, which would rely on swtpm support which depends on libtpms support seems possible as dependencies of the qemu-coreboot board, but would be linked to a totally different build path depending if on fedora build system or debian (debian not providing libtpms packages for some reason, requiring to build everything from source)

@orangecms: any success that path?

@tlaurion I'm not sure what qemu-coreboot is, but no matter what emulated TPM using swtpm requires initialization by firmware, which for example is implemented in SeaBIOS. Or maybe Linux kernel TIS/CRB driver is able to bring up TPM from scratch without firmware support - this should be pretty easy to check. Am I missing something?

tlaurion · 2020-06-18T21:36:02Z

@pietrushnic : qemu-coreboot in Heads term is just the board config

So one can build Heads for qemu calling
make BOARD=qemu-coreboot or make BOARD=qemu-coreboot-fbwhiptail
and then call
make BOARD=qemu-coreboot run or make BOARD=qemu-coreboot-fbwhiptail run
and be able to test Heads, with really limited functionnalities, as exposed through #688 and #701 #354 .

Any advice welcome, since building those qemu board configs without them being linked to a software TPM (swtpm), not having functional USB passthrough (to be able to sign /boot components and remotely attest through HOTP) nor assigned prepared block devices/images attached (installed OS or at least a /dev/sda1 to be recognized by Heads as boot partition which is a requirement, sign /boot config with passed USB device) is pretty useless but to show that heads produces a valid ROM only.

qemu-coreboot boards have TPM deactivated, and as reported by issue #688, since no valid block devices are passed to qemu, the board is a diskless machine without proper changes on board configs themselves doing some more magic for the end user calling make BOARD=qemu-coreboot-fbwhiptail run to have measured boot, install an OS and have verified /boot as a Heads user would expect to be able to test out of the box.

@orangecms helped here. Maybe he could share some more light on the subject?

ReLIFE9527 · 2020-06-23T02:36:26Z

@pietrushnic : qemu-coreboot in Heads term is just the board config

So one can build Heads for qemu calling
make BOARD=qemu-coreboot or make BOARD=qemu-coreboot-fbwhiptail
and then call
make BOARD=qemu-coreboot run or make BOARD=qemu-coreboot-fbwhiptail run
and be able to test Heads, with really limited functionnalities, as exposed through #688 and #701 #354 .

Any advice welcome, since building those qemu board configs without them being linked to a software TPM (swtpm), not having functional USB passthrough (to be able to sign /boot components and remotely attest through HOTP) nor assigned prepared block devices/images attached (installed OS or at least a /dev/sda1 to be recognized by Heads as boot partition which is a requirement, sign /boot config with passed USB device) is pretty useless but to show that heads produces a valid ROM only.

qemu-coreboot boards have TPM deactivated, and as reported by issue #688, since no valid block devices are passed to qemu, the board is a diskless machine without proper changes on board configs themselves doing some more magic for the end user calling make BOARD=qemu-coreboot-fbwhiptail run to have measured boot, install an OS and have verified /boot as a Heads user would expect to be able to test out of the box.

@orangecms helped here. Maybe he could share some more light on the subject?

So the coreboot.rom or linuxboot.rom built by heads when I calling 'make BOARD=qemu-linuxboot run' is not able to boot the real kernel and root filesystem, since there is no disk can be mounted. Is it possible to build the necessary components separately and then combine them into the linuxboot.rom, and subsequently run it on QEMU with the real kernel and root filesystem? Because I found QEMU can directly run the real kernel and root filesystem without linuxboot.rom, but when I added it, it would be stuck in the recovery shell, just as same as ‘make BOARD=qemu-linuxboot run’. Or, do you have some other good way to solve this problem? Sincerely thanks a lot.

tlaurion · 2020-07-18T21:30:00Z

@ReLIFE9527 : I was successful applying some hacks here while some help would be required to pass disks (or img) from the host to qemu so that the original mappings would be valid without hacking Heads board configurations but by having a make run statement that works out of the box successfully: mapping usb devices (HOTP tests) with a swtpm being setuped correctly in a precedent step, with its required dependencies.

I am on unknown territories here.
@pietrushnic any advices or name tagging here that would be helpful, or any link more specific then the previously documented path that someone with more experience could shed some light on?

Edit:

a qemu deployment depending on a host swtpm working instance looks like this. To make this work inside of Heads so that a make BOARD=qemu-linuxboot run works out of the box, we need:
- swtpm setuped on host calling qemu
- usb passthrough (HTOP) needs to be figured out
- disk images needs to be mapped from host so that Heads finds a workable /dev/sda1 and can boot from USB/DISK

tlaurion · 2020-08-01T14:51:41Z

Interesting USB usage tests were done here

tlaurion · 2020-11-05T15:30:44Z

@ReLIFE9527 @itay-grudev @pietrushnic : important breakthrough in code on the safeboot side for swtpm+qemu communication and board adjustments.

Reimplementing dependencies on qemu boards, Makefile and modules will lead to be able to test heads with TPM (and some additional tinkering would make USB pass-through of USB Security dongle work)

pietrushnic · 2020-11-08T23:32:07Z

@tlaurion thanks. We discussed that architecture and potential use cases extensively during QubesOS minisummit SRTM talk. Slides are here

tlaurion · 2020-11-09T14:07:57Z

That was implemented as draft into #893 (doesn't build correctly as of now)

tlaurion · 2022-03-25T21:34:11Z

For USB support (HOTP support) https://qemu.readthedocs.io/en/latest/system/devices/usb.html
Building instructions for swtpm missing in modules, so that qemu can ask for it and call it directly from make run.

itay-grudev closed this as completed Feb 16, 2019

tlaurion added a commit that referenced this issue May 11, 2019

Update qemu-coreboot.config

2544ff1

add "-nic none" to fix #516 (comment)

tlaurion mentioned this issue May 11, 2019

Fix qemu-coreboot board to boot by default #561

Closed

tlaurion reopened this May 15, 2019

tlaurion mentioned this issue Apr 8, 2020

e1000e: probe of 0000:00:19:0 failed with error -3 on x230 (when gbe.bin is removed) #700

Closed

This was referenced Apr 23, 2020

qemu-coreboot: finally a useable debug/test board #707

Merged

Some QEMU debug successful guidelines? #701

Closed

tlaurion mentioned this issue May 4, 2020

Add xx30-maximized and xx30-hotp-maximized boards (11.5mb flashable BIOS regions, reproducible me.bin and generated gbe.bin and totally externally and internally flashable roms) #703

Merged

tlaurion closed this as completed in #707 May 4, 2020

tlaurion mentioned this issue May 5, 2020

qemu-coreboot-fbwhiptail board addition #708

Merged

tlaurion reopened this May 5, 2020

tlaurion mentioned this issue May 14, 2020

Is TPM or Intel hardware required for heads to work? #717

Closed

tlaurion mentioned this issue Jun 18, 2020

how to boot debian by linuxboot in qemu #688

Open

This was referenced Jul 29, 2020

Various Problems of TPM Simulation on QEMU #354

Closed

qemu-coreboot scsi not working as expected linuxboot/heads-wiki#32

Closed

tlaurion mentioned this issue Jul 12, 2021

TPM2 support #893

Closed

tlaurion mentioned this issue Aug 5, 2022

QEMU testing with OS, TPM, USB token #1188

Merged

tlaurion closed this as completed Aug 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How do you test heads with QEMU? #516

How do you test heads with QEMU? #516

itay-grudev commented Feb 8, 2019

ThatLurker commented Feb 8, 2019

tlaurion commented Feb 8, 2019 •

edited

Loading

itay-grudev commented Feb 8, 2019

tlaurion commented Feb 8, 2019

tlaurion commented Feb 8, 2019 •

edited

Loading

tlaurion commented Feb 15, 2019

itay-grudev commented Feb 15, 2019

paulmenzel commented Feb 15, 2019 via email

itay-grudev commented Feb 15, 2019

itay-grudev commented Feb 15, 2019

tlaurion commented Feb 16, 2019

tlaurion commented Feb 16, 2019 •

edited

Loading

itay-grudev commented Feb 16, 2019

tlaurion commented May 12, 2019 •

edited

Loading

tlaurion commented May 12, 2019

tlaurion commented May 15, 2019 •

edited

Loading

tlaurion commented May 15, 2019 •

edited

Loading

tlaurion commented Oct 6, 2019

tlaurion commented Apr 13, 2020 •

edited

Loading

tlaurion commented Apr 23, 2020 •

edited

Loading

tlaurion commented May 5, 2020

pietrushnic commented May 14, 2020

tlaurion commented Jun 18, 2020 •

edited

Loading

ReLIFE9527 commented Jun 23, 2020

tlaurion commented Jul 18, 2020 •

edited

Loading

tlaurion commented Aug 1, 2020

tlaurion commented Nov 5, 2020

pietrushnic commented Nov 8, 2020

tlaurion commented Nov 9, 2020

tlaurion commented Mar 25, 2022

How do you test heads with QEMU? #516

How do you test heads with QEMU? #516

Comments

itay-grudev commented Feb 8, 2019

ThatLurker commented Feb 8, 2019

tlaurion commented Feb 8, 2019 • edited Loading

itay-grudev commented Feb 8, 2019

tlaurion commented Feb 8, 2019

tlaurion commented Feb 8, 2019 • edited Loading

tlaurion commented Feb 15, 2019

itay-grudev commented Feb 15, 2019

paulmenzel commented Feb 15, 2019 via email

itay-grudev commented Feb 15, 2019

itay-grudev commented Feb 15, 2019

tlaurion commented Feb 16, 2019

tlaurion commented Feb 16, 2019 • edited Loading

itay-grudev commented Feb 16, 2019

tlaurion commented May 12, 2019 • edited Loading

tlaurion commented May 12, 2019

tlaurion commented May 15, 2019 • edited Loading

tlaurion commented May 15, 2019 • edited Loading

tlaurion commented Oct 6, 2019

tlaurion commented Apr 13, 2020 • edited Loading

tlaurion commented Apr 23, 2020 • edited Loading

tlaurion commented May 5, 2020

pietrushnic commented May 14, 2020

tlaurion commented Jun 18, 2020 • edited Loading

ReLIFE9527 commented Jun 23, 2020

tlaurion commented Jul 18, 2020 • edited Loading

tlaurion commented Aug 1, 2020

tlaurion commented Nov 5, 2020

pietrushnic commented Nov 8, 2020

tlaurion commented Nov 9, 2020

tlaurion commented Mar 25, 2022

tlaurion commented Feb 8, 2019 •

edited

Loading

tlaurion commented Feb 8, 2019 •

edited

Loading

tlaurion commented Feb 16, 2019 •

edited

Loading

tlaurion commented May 12, 2019 •

edited

Loading

tlaurion commented May 15, 2019 •

edited

Loading

tlaurion commented May 15, 2019 •

edited

Loading

tlaurion commented Apr 13, 2020 •

edited

Loading

tlaurion commented Apr 23, 2020 •

edited

Loading

tlaurion commented Jun 18, 2020 •

edited

Loading

tlaurion commented Jul 18, 2020 •

edited

Loading