server.VerifyAttestation fails on gce debian10+secureboot

[salrashid123]

Attempting to use server.VerifyAttestation using the snippet provided below
server.VerifyAttestation fails on debian 10 with

gcloud compute instances create shielded-sb  --zone=us-central1-a --machine-type=e2-medium --no-service-account --no-scopes \
   --create-disk=auto-delete=yes,boot=yes,device-name=shielded-sb,image=projects/debian-cloud/global/images/debian-10-buster-v20211104,mode=rw,size=10,type=projects/mineral-minutia-820/zones/us-central1-a/diskTypes/pd-balanced \
  --shielded-vtpm --shielded-secure-boot --shielded-integrity-monitoring


# go run main.go
failed to validate the event log: failed to replay event log: event log failed to verify: 
the following registers failed to replay: [5 4]
exit status 1

though it works when using confidential compute instances

• main.go

package main

import (
	"crypto"
	"flag"
	"log"

	"github.com/google/go-tpm-tools/client"
	"github.com/google/go-tpm-tools/server"
	"github.com/google/go-tpm/tpm2"
	//"github.com/google/go-attestation/attest"
)

var (
	tpmPath = flag.String("tpm-path", "/dev/tpm0", "Path to the TPM device (character device or a Unix socket).")
)

func main() {
	flag.Parse()

	var err error

	rwc, err := tpm2.OpenTPM(*tpmPath)
	if err != nil {
		log.Fatalf("can't open TPM %q: %v", *tpmPath, err)
	}
	defer func() {
		if err := rwc.Close(); err != nil {
			log.Fatalf("\ncan't close TPM %q: %v", tpmPath, err)
		}
	}()

	ekk, err := client.EndorsementKeyRSA(rwc)
	if err != nil {
		log.Fatalf("ERROR:  could not get EndorsementKeyRSA: %v", err)
	}
	defer ekk.Close()

	ak, err := client.AttestationKeyRSA(rwc)
	if err != nil {
		log.Fatalf("ERROR:  could not get AttestationKeyRSA: %v", err)
	}
	defer ak.Close()

	nonce := []byte("noncevalue")

	attestation, err := ak.Attest(client.AttestOpts{Nonce: nonce})
	if err != nil {
		log.Fatalf("failed to attest: %v", err)
	}

	ims, err := server.VerifyAttestation(attestation, server.VerifyOpts{
		Nonce:      nonce,
		TrustedAKs: []crypto.PublicKey{ak.PublicKey()},
	})
	if err != nil {
		log.Fatalf("failed to verify: %v", err)
	}
	log.Printf("Machine State: %v", ims.RawEvents)

}

• go.mod

module main

go 1.17

require (
	github.com/google/go-tpm v0.3.3-0.20210409082102-d3310770bfec
	github.com/google/go-tpm-tools v0.3.1
	golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 // indirect

)

require (
	github.com/google/certificate-transparency-go v1.1.1 // indirect
	github.com/google/go-attestation v0.3.2 // indirect
	github.com/google/go-tspi v0.2.1-0.20190423175329-115dea689aad // indirect
	golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1 // indirect
	google.golang.org/protobuf v1.27.1 // indirect
)

---

root@shielded-sb:~/sb_test# dmesg |grep -i secure
[    0.000000] secureboot: Secure boot enabled
[    0.000000] Kernel is locked down from EFI secure boot; see https://wiki.debian.org/SecureBoot
[    1.583131] Loaded X.509 cert 'Debian Secure Boot CA: 6ccece7e4c6c0d1f6149f3dd27dfcc5cbb419ea1'
[    1.585830] Loaded X.509 cert 'Debian Secure Boot Signer 2021 - linux: 4b6ef5abca669825178e052c84667ccbc0531f8c'
[    1.604722] Lockdown: Hibernation is restricted; see https://wiki.debian.org/SecureBoot
[    2.936099] Lockdown: BPF is restricted; see https://wiki.debian.org/SecureBoot

# tpm2_pcrread sha1:7+sha256:7
  sha1:
    7 : 0xACFD7EACCC8F855AA27B2C05B8B1C7C982BFBBFA
  sha256:
    7 : 0x3D91599581F7A3A3A1BB7C7A55A7B8A50967BE6506A5F47A9E89FEF756FAB07A

# tpm2_eventlog /sys/kernel/security/tpm0/binary_bios_measurements
pcrs:
  sha1:
    0  : 0x0f2d3a2a1adaa479aeeca8f5df76aadc41b862ea
    1  : 0xb1676439cac1531683990fefe2218a43239d6fe8
    2  : 0xb2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
    3  : 0xb2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
    4  : 0x4474b686153d01c26b4455a3eb9e3632f355a2cb
    5  : 0xeb3dd3eb12b3d3df65d868757c87030b083bac06
    6  : 0xb2a83b0ebf2f8374299a5b2bdfc31ea955ad7236
    7  : 0xacfd7eaccc8f855aa27b2c05b8b1c7c982bfbbfa
    14 : 0x7c067190e738329a729aebd84709a7063de9219c

[alexmwu]

Thanks for the issue! Would you be able to attach the event log as well as the tpm2_pcrread outputs of 4 and 5?

[salrashid123]

# tpm2_pcrread sha1:4,5,7+sha256:4,5,7
  sha1:
    4 : 0xBF2FA4828D4613C8C1CD4B8E6F926ED7B2081BE0
    5 : 0x08FDB10D5F95A1157DA2B8D7A11E3FD59FC3A747
    7 : 0xACFD7EACCC8F855AA27B2C05B8B1C7C982BFBBFA
  sha256:
    4 : 0xF3AEA2992B888666F02B13C066FD35C67335D3FB77510214EF5C991522C5B2DF
    5 : 0x6F95FCD8211BB4FE5B0196FEE4DAFFACB4A3175ED22D6554C2656F5CA4AEE5C8
    7 : 0x3D91599581F7A3A3A1BB7C7A55A7B8A50967BE6506A5F47A9E89FEF756FAB07A

the log is at

gs://mineral-minutia-820-tpm/binary_bios_measurements

note, a debian-11-bullseye-v20211105 worksfine

[josephlr]

I was able to reproduce this issue specifically on Debian 10, maybe it's a kernel bug?

I confirmed that tpm2-tools also encountered issues replaying PCRs 4 and 5. I also confirmed that this issue is not fixed by either updating the guest VM or restarting the image.

I've uploaded the relevent files for a first boot of Debian 10

debian10_parsed_log.txt
debian10_pcr_log.txt
debian10_binary_bios_measurements.log

[josephlr]

I confirmed that using the exact same setting but going from Debian 10 to Debian 11, the issue is fixed. So I'm guessing this is a bug in Debian 10's kernel version.

Looking at the events, this does seem to be a kernel or bootloader configuration bug in Debian 10. There are three events missing from the Debian 10 log that a present in the Debian 11 log:

• An EV_EFI_BOOT_SERVICES_APPLICATION measurement into PCR 4 of the kernel itself (i.e. vmlinuz)
• Two EV_EFI_ACTION measurements into PCR 5 indicating that Exit Boot Services Returned with Success

I know the PCR 5 bug is the same issue tracked in https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1838796
The PCR 4 issues are probably also related to the same problem (judging by the comments there). Essentially, this patch which was added in kernel 5.3 fixed the problem.

Per Tyler Hick's comments, it seems like Ubuntu had a special backport to their older kernel (4.15), but the maintainer didn't "target these patches for linux-stable", so Debian 10 never got them.

Debian 10 version: 4.19.0-18-cloud-amd64 (from October 2018)
Debian 11 version: 5.10.0-10-cloud-amd64 (from December 2020)

Here are the relevant files for a first boot of Debian 11

debian11_parsed_log.txt
debian11_pcr_log.txt
debian11_binary_bios_measurements.log

[josephlr]

We either need to get that patch series backported to Debian 10 (which sounds annoying), or just tell users of this library "Don't use Debian 10, it's insecure w.r.t. the TPM anyway due to depending on SHA1)".

[salrashid123]

IMO, its fine to close this out w/ a note about debian10 (since its not just an issue with this library; i'd call it not in scope to hunt down missing patches in debian10!)

[josephlr]

@alexmwu or @jkl73 can we just add a note about Debian 10 (perhaps in the VerifyAttestation/VerifyOps docs where we already mention it). Then we should close this issue.

[alexmwu]

Sure, I'll add it sometime this week.