AEM boot option causes hard reboot/partial shutdown #2155
Comments
andrewdavidwong
added
T: bug
|
For the record, I've tried various kernels including 4.1 and 4.2 copied from R3.1. Those copied kernels can boot past the AEM target and there is no reset. This suggests that a kernel 4.2 released for R3.2 could be a solution to the problem. |
|
Per my thread post: But now I cannot get AEM to seal the secret. Nothing at all about AEM is displayed during startup, even though rd.antievilmaid is on the kernel options line. @andrewdavidwong Can you try the min_ram option? You only need to add |
I think this might be fixed by QubesOS/qubes-core-admin-linux@fe6846d (which hasn't been released in a new package version yet) |
|
@ttasket: Sorry, forgot to say: You could try |
|
@rustybird Thanks, but that service is already enabled. Status shows it ran with failure code: |
|
This is already fixed by HW42. Just building&uploading fixed package (v3.0.3). Best Regards, |
|
AEM now works again on my system. The keys were to update to anti-evil-maid 3.0.3 from the testing repo so it would seal, and adding the |
|
I wonder whether |
|
That would be a good question for upstream. I've CC'd gang.wei@intel.com with no response so far; I'd like to post it to the tboot mailing list after others ( @andrewdavidwong , Todd Lasman) try it out. ML might say its better to upgrade tboot. I started a fresh thread in qubes-users... https://groups.google.com/d/msgid/qubes-users/25ec3d4f-17cb-32a9-01b9-8a30c0150fe4%40openmailbox.org |
|
that is platform specific, try different vendors platforms to see if this issue can be seen on all the platforms |
|
Todd confirmed that the |
|
@ttasket: I've now tested this. Unfortunately, there is no change on my end. I tested with |
|
@andrewdavidwong My AEM version is 3.0.4-1.fc23 not fc20. If you're still running Qubes 3.1 you may want to upgrade to 3.2 rc2; that's what I'm using. tboot-devel thread here: https://sourceforge.net/p/tboot/mailman/tboot-devel/thread/005196f3-3afd-eca1-787b-841e3953b39f%40openmailbox.org/#msg35257679 Newer versions of tboot do work with Linux 4.4, at least Ubuntu's. @marmarek Are there any specific guidelines for compiling and installing tboot for Qubes? |
Look at fedora package - especially spec file. It should just work to As you've probably seen, I'm looking for a way to get somehow verified Best Regards, |
|
@ttasket: Ok, I'll try again after I install 3.2* on this laptop. |
|
@marmarek We got a signed copy of tboot 1.9.4 from ning.sun@intel.com (see inbox). Do you want to package it for Qubes and test, or test it first? |
|
I had the same issue with TBOOT causing a reboot after GETSEC[SENTER]... I am using Qubes R3.2, anti-evil-maid 3.0.4 and have a 6th gen i7 SINIT file. You can manually upgrade it via Ubuntu: https://launchpad.net/ubuntu/yakkety/amd64/tboot/1.9.4-0ubuntu1 just unpack it from there and drop the "tboot.gz" and "tboot-syms" into your boot folder. Would be nice to have this by default or AT LEAST have a huge troubleshooter box over AEM which tells you to take this step manually for the time being. Searching for this issue is a freakin nightmare (and it was just one issue in a long list of avoidable problems) to be honest and in total I probably spent two full days on getting AEM to work just because the install process is so fundamentally broken and badly documented. But in my opinion Anti Evil Maid on an SD card is pretty much necessary for running a system like Qubes OS, and some work should be done to make AEM more seamless. Qubes without AEM is terribly unsafe. This is why I think the whole process needs urgent streamlining. So many points of failure:
|
|
Thanks for the input @oldblob666 ... I did look for tboot in debian and couldn't find it... assumed incorrectly that Ubuntu didn't have it either. Of course, verifying the Ubuntu tboot package with their distro key is recommended before using it as a replacement for the fedora tboot.
Agreed. I think what many Qubes users lose sight of is that AEM isn't just a mitigation for physical attacks. It could also warn you if a remote attack somehow got to your firmware. So, yes, the AEM type of protection or warning system should be considered necessary for security in general. @andrewdavidwong : Numbers 1) and 2) should be made into their own issues... these are essentially bugs addressable directly by Qubes. Though, I'll add that AEM has already streamlined somewhat over the years (re-sealing is now automatic, for instance). As for tboot version, I think the assumption so far is that AEM is an 'extra' feature and its OK to defer to Fedora's decisions about updating it. But Qubes can address that, too, by supplying updated version directly. TPM unfortunately is idiosyncratic with insufficient documentation of the ownership flow or the significance/mechanics of physical presence. The AEM documentation might be a good place to shed light on that topic, though one would expect the computer's documentation to supply the right info. |
|
@tasket Thanks for your answer :).
You can verify correct unseal operation by using: tpm_unsealdata -z -i "/boot/aem/tpms/%some-long-hex-name%/aem/secret.txt.sealed" It should print the secret you just sealed with "anti-evil-maid-seal". THIS should also be part of the README, because rebooting over SD card is painfully slow + the normal reboot time and lack of debugging facilities... This is a nice and simple test you can perform to predict the outcome of the next reboot ^^. |
|
FINALLY I got it to work, see my updated comment above! I may try to fix the scripts for supporting SRK & Owner Key at some later date but right now I am really so done with all this low level boot stuff that I will probably not look at a PC for a few days and rather just tend to my pretty iPad, which just works ^^. |
|
@nsun1 : Using the tboot 1.9.4 that I compiled doesn't seem to help. If I remove the min_ram parameter, the system restarts much like before. With min_ram, the system unseals the secret and boots normally-- but wake from suspend no longer works. When its suspended and I press the power button, the HD light flashes like it normally does, but the screen never powers on. The power button keeps pulsing as if the system were still asleep, but after a minute the CPU fan starts running fast. @chris-hacker-news : Does tboot 1.9.4 allow your system to sleep and wake up? |
|
It looks like your machine needs min_ram parameter, so far we cannot tell if S3 was failed. One way is to collect from your serial port during S3 transition to identify this issue root cause. Do you have chance to get another vPro brand PC or laptop to run tboot on it, like HP, Dell all have this kind of machines? -ning From: tasket [mailto:notifications@github.com] @nsun1https://github.com/nsun1 : Using the tboot 1.9.4 that I compiled doesn't seem to help. If I remove the min_ram parameter, the system restarts much like before. With min_ram, the system unseals the secret and boots normally-- but wake from suspend no longer works. When its suspended and I press the power button, the HD light flashes like it normally does, but the screen never powers on. The power button keeps pulsing as if the system were still asleep, but after a minute the CPU fan starts running fast. @chris-hacker-newshttps://github.com/chris-hacker-news : Does tboot 1.9.4 allow your system to sleep and wake up? — |
|
@nsun1 : AFAIK there are no recent Thinkpad models with rs232 ports or with docks bearing them. The other computers here are AMD or otherwise don't have TXT. I get the same behavior using the rtcwake command. Is there a log file that might be recording info during the wake attempt? |
|
I am using the HP EliteDesk 800 and Dell T430 server for tboot dev. From: tasket [mailto:notifications@github.com] @nsun1https://github.com/nsun1 : AFAIK there are no recent Thinkpad models with rs232 ports or with docks bearing them. The other computers here are AMD or otherwise don't have TXT. I get the same behavior using the rtcwake command. Is there a log file that might be recording info during the wake attempt? — Intel(r) TXT Configuration Registers: TBOOT log: |
|
@andrewdavidwong @chris-hacker-news : Is it possible to test tboot 1.9.4 on your systems to see how sleep/wake work? It would be good to have the extra input and get a sense for how common the waking problem is. |
|
@tasket: Not sure when exactly I'll have time to do this, but I'll try! |
|
Thanks Andrew! @nsun1 : If that log is from a serial port, can I get similar output on the vga display somehow? Unfortunately, the display doesn't seem to turn on when resuming from S3, even if tboot was loaded with the vga logging option. I somewhat doubt an in-memory log would survive a reboot on my system... is this even an option? |
|
Update: |
|
Usually, we try linux kernel w/ tboot, and do s3 resume with command: rtcwake –u –s 10 -m mem. From: tasket [mailto:notifications@github.com] Update: — |
|
@nsun1 : The rtcwake command in this case doesn't behave any differently than other methods of going into sleep/wake modes. If you know of some way to retrieve a boot log from memory then I can try that. However, I'm assuming recent Intel systems scramble RAM after a system reset, making log recovery impossible. |
|
@tasket: I just re-tested the |
|
@andrewdavidwong Thanks :) Let us know if you test it also with tboot 1.9.4, which does boot for me; Its available from https://sourceforge.net/projects/tboot/files/tboot/ and IIRC you should have a cc of email from nsun1@intel.com 9/1/2016 with tboot signature. This newer tboot does have an issue where my system can't wake from sleep. |
|
@tasket Can you paste the tboot signature into a gist? Maybe that will get more testers. |
|
i guys, I would like to try too, I have a t450s and anti-evil-maid doesn't work since my last bios update. |
|
@rustybird @cyrinux : You can download it here... https://sourceforge.net/projects/tboot/files/tboot/ The signature is attached to this post (unzip it before using it to |
|
Forgot to mention that is source code so you will have to use Its also possible to manually download a binary deb package from ubuntu's repository then verify it using the ubuntu keys that can be installed in a debian template. |
Thanks! Do you know if the signing key (which was created on the same day as the signature) is mentioned anywhere online? The keyserver had 4 more keys for his email address: (FWIW, the signature's data payload was identical to the tboot-1.9.4.tar.gz I downloaded from one of SourceForge's HTTP mirrors over Tor. Hurray) |
|
@rustybird - You might feel better about verifying and unpacking the Ubuntu package instead? |
|
If it is for try out or test purpose, tboot source code can be found from here: https://sourceforge.net/p/tboot/code/ci/default/tree/
You can download it with command: hg clone http://hg.code.sf.net/p/tboot/code tboot-code
From: tasket [mailto:notifications@github.com]
Sent: Monday, November 28, 2016 1:53 PM
To: QubesOS/qubes-issues <qubes-issues@noreply.github.com>
Cc: Sun, Ning <ning.sun@intel.com>; Mention <mention@noreply.github.com>
Subject: Re: [QubesOS/qubes-issues] AEM boot option causes hard reboot/partial shutdown (#2155)
@rustybird<https://github.com/rustybird> -
@nsun1<https://github.com/nsun1> can comment about the key that was used (seems to have prefered making a new key for that purpose). I get the impression trusted boot isn't a high priority project for Intel. I don't know what else to make of it.
You might feel better about verifying and unpacking the Ubuntu package instead?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub<#2155 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/APgjixf8sQ6NVXU_W2gsGSmw3C2xMRBrks5rC01BgaJpZM4JFmD4>.
|
|
I had the same crash/reboot after
(Yes, it's a weird computer. I know. Was mostly intended for gaming, decided to pick some specs to allow running Qubes and AEM too though. Need to manage to add a second USB controller for the keyboard somehow though.) |
|
I recently experienced this same issue though I'm not sure how. I'm pretty new to linux so my diagnosis skills are limited. About 3 months ago I installed qubes 3.2 on my x230 thinkpad, no major issues, almost everything works. AEM working fine. I'm not sure what caused the change. Best I can figure is that whenever I saw a dom0 update, I updated (rather carelessly, it turns out), without really testing or even restarting. In fact I almost never shut my machine down below s3 sleep. About 3 days ago I got tired of waiting for my laptop to become responsive and I hard restarted. The AEM grub option goes into a boot loop, which I think people in this thread are familiar with. After several days of penitent googling and restarts, it turns out that AEM now only works with 1) a well known owner, 2) the min_ram parameter set, 3) tboot 1.9.4 files (blobs?). As of my last restart, none of this was the case. So there you have it. I have no idea what was updated or why it would cause tboot to stop working. FWIW, AEM is installed to my boot sector and I haven't tried it on a usb device. It's hard enough to get it working as it is. Lessons: I need to figure out how to roll back updates with dnf. I need to watch what gets updated, and test, or at least restart, after updating. If my boot sequence were to be tampered with, I'd probably just think it was something I did, and ham-fistedly clear tpm in my attempt to fix it. In fact, I may have just done that. They do tell me my context is high threat, which is why I'm going through this in the first place. |
|
@earque It was a Xen or Linux upgrade that triggered the problem for me... wish I could be more specific. If you are using removable media for the boot volume, you also have to be mindful about which package updates will require that volume to be mounted as /boot (during the update). That includes xen* packages, kernel, tboot, grub and anything that gets included in the initramfs. |
|
Hi, The AEM setup (2FA with TOTP and usb stick), as described on the README doc, has been pretty straightforward. No problem with the boot/sealing process. However, when put in sleep state, my laptop reboots. I've tried to add several options to grub.cfg (i.e. |
andrewdavidwong
added
the
eol-4.0
|
This issue is being closed because:
If anyone believes that this issue should be reopened and reassigned to an active milestone, please leave a brief comment. |
Qubes OS version (e.g.,
R3.1):R3.1,R3.2-rc1Full discussion thread
Brief Problem Description:
Everything goes smoothly with the AEM installation up to step 5 (reboot and select the "AEM Qubes" GRUB option). I select that option (or allow it to be auto-selected, or select the one in the "advanced" submenu). It gets about 4 lines in (up to "loading initial ramdisk"), then the laptop appears to do a hard reboot/partial shutdown. Instead of a normal reboot with the BIOS and normal boot process, the screen is blank, but the system retains power. (Power button is lit and keyboard backlight brightness can be changed.)
More precisely, here are the physical symptoms, in order:
The text was updated successfully, but these errors were encountered: