WIP - Z690 A DDR5 / Msi ms7d25 TPM2

[ThePlexus]

This is my attempt so far to bring the Z690-A DDR5 into Heads. This is a WIP. The board seems to have all regions unlocked for internal programmer.

status
It boots to serial console only and we see hello world;

connections

• Flash chip is MX25U25635F WSON8, soldered on (though has been seen with a Winbond too, per @miczyg1 )
• External flashing is not amazingly easy, but possible with a WSON8 clip
• External flashing is a bit less fiddly by using the SPI bus via the TPM2 header
• Dasharo also show how to get read only ttyS0 console

Requirements

• copied 1251 to use Dasharo flashrom for MX25U25635F compatibility
• Have to use coreboot 4.19 for board support, though i expect 4.20 will be a better choice given all the work since 4.19
• Patch made to back port a fix for some Alderlake iGPU https://review.coreboot.org/c/coreboot/+/70101/4#message-0ac2afa1c9d02452721469c837caabb01ccd86b2

ME

• Cant me_clean and reduce ME firmware on this generation of cpu. just how it is.
• We can use ME hap bit to disable ME 6d6178667269747a/me_cleaner@d6b313a
• I set HAP the bit in download_BIOS_clean.sh by flipping 0x1DE to 0x11 - validated that setting this bit on factory firmware shows ME version 0.0.0.0 in factory BIOS
• as we cant shrink ME, it may be an idea to use VSCC mod by default on this board
• could we make a patch and have coreboot measure the ME region for changes?

Findings/WIP stuff

• Board config not using its own kernel, i just stole the librem common as it was the same version and a reasonably nearby CPU family. WIP
• No graphics working yet, console output only. Coreboot uses FSP GOP, so I assume this needs some kernel modules.
• theres a worrying TPM error message in the console output that needs looking at (see below)
• Coreboot VBOOT seems to use the onboard fTPM even with a dTPM attached. The above mentioned error still remains. This is going to take a bit more work
• network at boot is untested and unlikely. dashro extract a binary blob to use in a UEFI payload, and currently this WIP is CSM mode https://github.com/Dasharo/coreboot/blob/msi_ms7d25/release/build.sh#L16

** TPM error**

[DEBUG]  TPM: CRB Interface is not active. System needs reboot in order to active TPM.
[ERROR]  TPM: Probe failed.
[ERROR]  tlcl_lib_init: tis_init returned error
[ERROR]  TPM: Can't initialize.

even though it seems coreboot creates a TPM log and TPM measurements do present.

[DEBUG]  ACPI:    * TPM2
[DEBUG]  TPM2 log created at 0x7699b000
-- SNIP
[INFO ]  coreboot TPM log measurements:
[INFO ]   PCR-2 ae47351f1afcf7df124e3b547b8d2a96dc43da705e432bc1ba08942ef6822f89 SHA256 [FMAP: FMAP]
[INFO ]   PCR-2 ea2696180cda0ce19438a62f0a1bdc86007908e825f4dec108147aea1aa58fb5 SHA256 [CBFS: bootblock]
[INFO ]   PCR-2 eeca2517dbe532bab8ceb7c7612338e4e72d9bd7aaa86f518b6ee9feaae52ee9 SHA256 [CBFS: fallback/romstage]
[INFO ]   PCR-2 25ff8837f740d51fe99acf1cb85582df6f97c1425e58b4cd67d14bfb426deec0 SHA256 [CBFS: fspm.bin]
[INFO ]   PCR-2 74eba1e41dc4e310666224177fbefa9fa723952ff058c6948805e35cc7ee2ed4 SHA256 [CBFS: fallback/postcar]
[INFO ]   PCR-2 29de505911e6181cb2564a486fcdb77c9b62d22afb7eba7e75a1adaf20dfbca9 SHA256 [CBFS: fallback/ramstage]
[INFO ]   PCR-2 1dc207481a8570abc93295543da654b24eaceb1b17efaf536a0db07617ff3d15 SHA256 [CBFS: cpu_microcode_blob.bin]
[INFO ]   PCR-2 7ec6adf33596c9051cad0ef295675bdf0b02c78d89675d7a3bfd7e74f677d7d0 SHA256 [CBFS: fsps.bin]
[INFO ]   PCR-2 2502ce2ed0172d622f0eb24aaa7c173689e54719eab258a161422f5409bd3b2e SHA256 [CBFS: vbt.bin]
[INFO ]   PCR-2 5ae7b8ee21400362658243091c1dd30044d5392ca725db4d7837b5a4c32d2ddd SHA256 [CBFS: fallback/dsdt.aml]
[INFO ]   PCR-2 1e0433b680a0deba810e265b0e29680d6c67c3d47203bfeaa52bc332f4087a75 SHA256 [CBFS: fallback/payload]

[miczyg1]

@ThePlexus just a few moments ago I pushed slightly better approach to flash the board: https://docs.dasharo.com/variants/msi_z690/development/

WSON8 clip is not needed anymore. The BIOS flash CS pin is exposed on JTPM1 pin5 (reserved), so basically any 1.8V level capable programmer should do the work by connecting 2mm to 2.54mm dupont wires to JTPM1 header.

Also the board may come with Winbond chip too, not that it is relevant for internal programmer

dTPM is not supported for now, because coreboot can work only with one TPM type, either fTPM or dTPM and will need code modifications and fiddling with ME to switch to dTPM.

[ThePlexus]

Thanks for the knowledge share @miczyg1 . Thats great news to hear - I shall modify the top post here to record the change in header status.

Re dTPM - thanks for the info I suspected that was the case. I had to do something similar for the P8H61 (though a lot less effort :P )

[tlaurion]

@ThePlexus some notes from what I can see

• Intel_iommu=igfx_off is not applied from coreboot command line down to heads kernel iommu and passed right through to env to be picked up by userland. Edit: it is applied (kernel output confirming it), unknown why init sees it in userland.
• No idea of how graphics are dealt with by FSP GOP, but serial only shows that either kernel's i915 is not recent enough, that coreboot kernel command line doesn't specify right options for Heads to init DRM+GPU to enable framebuffer
• coreboot enabling and using TPM but not Linux shows something missing in kernel to use fTPM correctly here.

[tlaurion]

Remove this for release (as other debug and trace output) This one only useful to have pcap traces under /tmp

[tlaurion]

This is tpm1 enablement. Remove otherwise unsure of Heads behavior.

[tlaurion]

console=tty needs to he the last one to tell what is main console (VGA).
KERNEL_ADD options are passed to Heads' kexec'ed kernel

[tlaurion]

Same here. Swap consoles so last is desired main console

[miczyg1]

• No idea of how graphics are dealt with by FSP GOP, but serial only shows that either kernel i915 is not recent enough, that coreboot kernel command line doest specify right options for Heads to init DRM+GPU to enable framebuffer

FSP GOP works the same way as libgfxinit. The result is a coreboot framebuffer saved in LB tags for use by payload. If heads Linux kernel does not initialize i915 properly, then maybe the Linux version is too old?

[tlaurion]

We might need to separate flashrom module from dasharo-flashrom. Not all older boards will be able to have this new flashrom version which is bigger. Good for PoC, but this is why it's not merged to replace currently used flashrom version: boards that would benefit from WP (without TPM: x200 etc) are also unfortunately really size constrained in term of available SPI space.

EDIT: conclusion is that passing to newer dasharo/flashrom (ast1100 + WP support) would cost 0.01mb. Will leave point open for now, but not really a concern with even t520-hotp-maximized board havig still 0.87mb of free space after the merge (still concerning after newer gpg2 PR merge etc) but not an emergency, where kernel modules could be trimmed a bit more, as planned.

[miczyg1]

Z690 chipset is available in upstream flashrom already: https://github.com/flashrom/flashrom/blob/master/chipset_enable.c#L2178

[tlaurion]

@miczyg1 : build of master flashrom happening under https://app.circleci.com/pipelines/github/tlaurion/heads/1604/workflows/f7a74cfc-72f2-4de7-aedc-e8a7c24cc5aa for commit tlaurion@0d49979 (building Heads flashrom to be as flashrom flashrom/flashrom@f3c21c6) EDIT: test was bogus for size comparison since both DUMMY and AST1100 were removed.

[tlaurion]

Building on flashrom master at https://app.circleci.com/pipelines/github/tlaurion/heads/1606/workflows/e7adbb7b-4112-4017-b9c4-e419397386c9 for tlaurion@9b03e72 which only removes AST1100 to be comparable to https://app.circleci.com/pipelines/github/tlaurion/heads/1605/workflows/26753865-ac5a-4097-8e06-eccc85d748bf which builds for dasharo/flashrom with ast1100

Will edit this comment to compare free space for single board to have clear comparison ground.

[tlaurion]

@miczyg1 : Clearer and direct builds are happening here:

• flashrom/flashrom master on top of heads master: https://app.circleci.com/pipelines/github/tlaurion/heads/1607/workflows/23f6c9f3-926f-478f-a709-70be6d87bae4
• https://app.circleci.com/pipelines/github/tlaurion/heads/1607/workflows/23f6c9f3-926f-478f-a709-70be6d87bae4/jobs/20838?invite=true#step-103-745 (empty space: 4910808)
• dasharo/flashrom (WP+ast1100 support) on top of heads master: https://app.circleci.com/pipelines/github/tlaurion/heads/1605/workflows/26753865-ac5a-4097-8e06-eccc85d748bf
  • https://app.circleci.com/pipelines/github/tlaurion/heads/1605/workflows/26753865-ac5a-4097-8e06-eccc85d748bf/jobs/20837?invite=true#step-103-790 (empty space: 4910808)

I'm now even more confused. Is ast1100 merged in master? How come adding more does not increase size?

---

EDIT:
Unbelievable until tested, but downloading both initrd.cpio.xz have the exact same size.

flashrom master:

user@heads-tests:/tmp/flashrom_master$ wget https://output.circle-artifacts.com/output/job/1a4f9a85-50c1-4fc9-be79-0fdf3a432d75/artifacts/0/build/x86/x230-hotp-maximized/initrd.cpio.xz
user@heads-tests:/tmp/flashrom_master$ ls -al initrd.cpio.xz 
-rw-r--r-- 1 user user 4125696 Apr 13 11:01 initrd.cpio.xz
user@heads-tests:/tmp/flashrom_master$ xz -d initrd.cpio.xz 
user@heads-tests:/tmp/flashrom_master$ cpio -i < initrd.cpio
user@heads-tests:/tmp/flashrom_master$ ls -al bin/flashrom
-rwx------ 1 user user 674552 Apr 13 11:41 bin/flashrom
user@heads-tests:/tmp/flashrom_master$ strings bin/flashrom | grep ast1100
user@heads-tests:/tmp/flashrom_master$

dasharo/flashrom from other PR:

user@heads-tests:/tmp/dasharo_flashrom$ wget https://output.circle-artifacts.com/output/job/6a254451-086a-4521-a2ad-8834a699a7d2/artifacts/0/build/x86/x230-hotp-maximized/initrd.cpio.xz
user@heads-tests:/tmp/dasharo_flashrom$ ls -al initrd.cpio.xz 
-rw-r--r-- 1 user user 4125696 Apr 13 10:52 initrd.cpio.xz
user@heads-tests:/tmp/dasharo_flashrom$ xz -d initrd.cpio.xz 
user@heads-tests:/tmp/dasharo_flashrom$ cpio -i < initrd.cpio
user@heads-tests:/tmp/dasharo_flashrom$ ls -al bin/flashrom
-rwx------ 1 user user 703224 Apr 13 11:41 bin/flashrom
user@heads-tests:/tmp/dasharo_flashrom$ strings bin/flashrom | grep ast1100
ast1100
ast1100_spi_send_command

So xz is just doing an amazing job at compressing heads.cpio, tools.cpio and modules.cpio where no difference at all is shown when only flashrom is involved as of today.

[tlaurion]

So let's take a clean view now into space differences comparing actual heads master using old flashrom/flashrom and dasharo/flashrom.

• Last master commit's build log https://app.circleci.com/pipelines/github/osresearch/heads/573/workflows/cab20f48-dfb8-43ca-a7a9-cc9d3c3ee5e8/jobs/9088?invite=true#step-103-819 (empty: 4922008)
• dasharo/flashrom (WP+ast1100 support) last buildlog https://app.circleci.com/pipelines/github/tlaurion/heads/1605/workflows/26753865-ac5a-4097-8e06-eccc85d748bf/jobs/20837?invite=true#step-103-790 (empty space: 4910808)

cost of passing to dasharo/flashrom: 11200bytes

[simonbunny]

• Intel_iommu=off is not applied from coreboot command line down to heads kernel iommu and passed right through to env to be picked up by userland.

@tlaurion interesting. As its present in both the corebot config and board config linux lines. When im next home ill dive into this. hopefully just something ive done somewhere in the ports. But, if not, well...things to look into.

• coreboot enabling and using TPM but not Linux shows something missing in kernel to use fTPM correctly here.

Agree. I would personally advise we use a dTPM, which is going to take some work. ive not really given much effort to using the fTPM at the moment, as, well, i prefer something discreet over something on-die that i cant stick a SPI probe in between and verify. Maybe being too paranoid? i dunno

[tlaurion]

@ThePlexus @simonbunny

When expending (oldconfig) current defconfig saved linux config from librem_common, the following expends:

#
# Generic IOMMU Pagetable Support
#
# end of Generic IOMMU Pagetable Support

# CONFIG_IOMMU_DEFAULT_PASSTHROUGH is not set
# CONFIG_AMD_IOMMU is not set
CONFIG_DMAR_TABLE=y
CONFIG_INTEL_IOMMU=y
CONFIG_INTEL_IOMMU_SVM=y
CONFIG_INTEL_IOMMU_DEFAULT_ON=y
CONFIG_INTEL_IOMMU_FLOPPY_WA=y
# CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON is not set
# CONFIG_IRQ_REMAP is not set

As written (sorry for all the debugging info there) under #1351 and under other issues, I am totally not convinced of saving either coreboot nor linux in defconfig format anymore. As one can see today under #1373, the coreboot config there are plain make savedefconfig back in place for all coreboot configs there. I'm not really sure Kconfig changes upstream on neither linux nor coreboot are doing a really good job at saving only differences under defconfig format anymore, but I know that on version bumps and to compare what is actually enabled between two boards, we need full oldconfig format config files otherwise its hard to understand whatever is really happening.

@ThePlexus I would recommend saving in oldconfig format.

@JonathonHall-Purism i'm being confused though, since one quick check of config/coreboot-librem_14.config shows:
CONFIG_LINUX_COMMAND_LINE="intel_iommu=igfx_off quiet loglevel=2" while this board port reuses librem_common.cfg at https://github.com/osresearch/heads/pull/1371/files#diff-231d238bcf1aae5dec8f4bf6ca1fcf71fc5d5c64e7fae5a3e82ed26467120762R5

So are you seeing intel_iommu=igfx_off being passed as environement variable to init as shown under #1371 (comment) picture:

Edit: @JonathonHall-Purism confirms he sees the same on init ouput, but also confirms that he sees "DMAR: Disable GFX device mapping" in the logs, confirming that kernel is applying the option.

Again sorry for the noise.

[tlaurion]

@ThePlexus can you provide full serial console log please?

[tlaurion]

Intel_iommu=igfx_off is not applied from coreboot command line down to heads kernel iommu and passed right through to env to be picked up by userland. Edit: it is applied (kernel output confirming it), unknown why init sees it in userland.

@simonbunny I edited previous posts.

[ThePlexus]

Closing this out for now, as it will require (at minimum) some significant coreboot modifications and has some less desirable outputs. I will open anew PR with work so far (it does post and gets display) but there are some significant upstream blockers to this

1. We cannot use HAP bit to disable ME as coreboot on this board. It requires us to use HECI to disable PTT and switch to dTPM. This is method is using cse_enable_ptt(false);, however this function will not operate with HAP bit 0x11 only with HAP bit 0x10. This would leave only 'soft disable of ME via HECI' as a way to disable ME on this board.

• @miczyg1 do you think there is any way to use a dTPM without HECI? For example, if I were to commission a coder to work on this upstream in coreboot. Or are we looking at a situation where we cannot use a dTPM with HAP bit enabled, no matter what we do?
• @tlaurion Is soft disable of ME via HECI only a viable option? Im leaning towards no as im not sure how much it would be trusted to not simply be able to be switched back on.

2. When using HECI cse_enable_ptt(false) in the main board silicon init, the system does global reset, and on the next boot it does show PTT as 'already disabled'. However, there is still not dTPM as addressing this is not yet added to the board upstream in coreboot.

3. The upstream mainboard Kconfig has TPM_CRB, so it looks like coreboot vboot/measure still just fallbacks to this even with cse_enable_ptt (fTPM PCR2 has measurements in it)

@tlaurion what is the view on using the fTPM in these families, assuming a dead end on dTPM and ME?

Ill do a new PR with the work so far at some point.

[tlaurion]

Can you turn that into an issue? @ThePlexus ?

[miczyg1]

• @miczyg1 do you think there is any way to use a dTPM without HECI? For example, if I were to commission a coder to work on this upstream in coreboot. Or are we looking at a situation where we cannot use a dTPM with HAP bit enabled, no matter what we do?

TBH I haven't verified how chipset behaves when ME is disabled with HAP (I have no MSI compliant SPI dTPM to check if TPM can be switched from PTT to SPI dTPM). I know for sure that fTPM stops working with whatever method you use to disable ME.

2. When using HECI cse_enable_ptt(false) in the main board silicon init, the system does global reset, and on the next boot it does show PTT as 'already disabled'. However, there is still not dTPM as addressing this is not yet added to the board upstream in coreboot.

Yes, we need the patches that unify the TPM drivers in coreboot: https://review.coreboot.org/q/topic:%22tpm-unification%22

3. The upstream mainboard Kconfig has TPM_CRB, so it looks like coreboot vboot/measure still just fallbacks to this even with cse_enable_ptt (fTPM PCR2 has measurements in it)

@tlaurion what is the view on using the fTPM in these families, assuming a dead end on dTPM and ME?

Ill do a new PR with the work so far at some point.

With the patches above, there should be no need for TPM_CRB option anymore.

[miczyg1]

@tlaurion I took a quick look at what happens to the PTT state when HAP is enabled aaand... It looks like ME/chipset automatically switches to the next available and enabled/configured TPM interface. In case of MSI, I have checked this register: https://github.com/coreboot/coreboot/blob/master/src/security/intel/txt/txt_register.h#L98 which indicates currently active TPM bus/interface and it indicated SPI TPM. So that's a good news, that we don't have to use HECI to disable PTT explicitly if we use HAP. However, we still need that patches unifying TPM drivers in coreboot to land.

[tlaurion]

@ThePlexus most URLs in OP are 404

[tlaurion]

However, we still need that patches unifying TPM drivers in coreboot to land.

@miczyg1 something to track?

[miczyg1]

However, we still need that patches unifying TPM drivers in coreboot to land.

@miczyg1 something to track?

@tlaurion see my earlier comment, there is a link to gerrit

[tlaurion]

@ThePlexus I see that last force push removed the PR content.
Another user would be willing to participate here, discussion happened at https://matrix.to/#/!pAlHOfxQNPXOgFGTmo:matrix.org/$hO6weloLmOFA4pb3RlWbWqGSQF920igx2R1nG54qS-E?via=matrix.org&via=nitro.chat&via=fairydust.space yesterday.

Would be nice if you forced push your PoC state here so this PR could be used to collaborate into creating this community driven port!

[ThePlexus]

@tlaurion the branch was deleted, i tried a force push using the same branch name and it didnt action.. there has also been some changes since my work (modules/coreboot layout for example) so ive adapted.#1489