-
Notifications
You must be signed in to change notification settings - Fork 53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
--enable-modernity results in instacrash for xen unikernels #70
Comments
This is on x86_64; haven't yet tested ARM but will do so if helpful. |
Ouch, ouch and ouch. This means it is crashing in the new SSE/AES-NI code that What's the behavior if you use the same flags on the same machine but with Unix underneath? Alternatively, what flags do you see in Explanation 1: Your CPU does not know AES-NI. That would be OK, I could rename the flag to something less presumptuous, and maybe disable it by default. Dynamic detection is yet to be done. Explanation 2: SSE registers are unavailable on Xen/MiniOS. This is the stuff of nightmares. Re: ARM, |
No problem running on Unix; process runs and serves files over HTTPS just fine. Here's the relevant-looking stuff from /proc/cpuinfo:
|
Is it xen (does AES-NI work in a Linux domU?)? or is it minios/mirage running in supervisor mode? |
SSE is available but stack alignment must be preserved. This should just work though, because builtins like |
@yomimono Thank you! I just wanted to check if it's the CPU feature set. It is not. The next thing that would be helpful is to add printfs to the beginning of these two functions, because I think the first brush with this code is in the early RNG seeding. Another helpful printf-case is confirming that the In more detail, for the purposes of my question, it makes no difference whatsoever whether you ran the Unix test without Xen in sight, or in dom0, or domU. It especially does not matter if it didn't explode, because we can immediately rule out the CPU limitation on your own computer. Xen itself obviously "supports" SSE in that these are not privileged instructions, are not trapped in HVM and are not hindered by having no access to ring0 in PVM. There is no difference between dom0 and domU in this respect, as both are actually virtualized. dom0 might be allowed access to memory regions peripherals are mapped to, but runs under the same regime of CPU virtualization. And as @avsm alluded to, actual lack of support for SSE under Xen would make Linux bomb out immediately. SSE requires OS cooperation at least in order to preserve the extended register set on context-switch. I suspect, but at this point without knowing for a fact, that the OS also needs to enable them, and that MiniOS never does it. I also think there is no other SSE code running in MiniOS as it doesn't use the standard libc, so this is the first time MiniOS was prodded into running SSE. @avsm It's specifically using SSE2 unaligned-access Off to untar my Xen-debugging rig... |
@pqwy - thanks for the explanation. printfs in the suggested places don't produce any output before the VM crashes. Happy untaring... |
Ok, I have a repro. But no clue so far. |
It's alignment! Spot on @avsm ! 🎊 Most traffic between memory and SSE registers goes via unaligned load/store, except when it doesn't -- AES round keys are just referenced directly, by mistake. I managed to remain completely oblivious to this because on Unix the allocator returns 16-byte aligned bigarrays. Not on Xen. mirage/ocaml-cstruct#38 would probably help for a nice, long-term solution. |
See #70. RKs are not explicitly loaded and stored so the pointers need to be aligned. A better solution than allocating a little more and aligning on access would be to make a utility to allocate them aligned in the first place. The best solution would be to have a reusable one.
@pqwy - Yes, it boots and serves nicely in Xen with that patch. Hooray! |
I'm building unikernels with
mirage-seal
version 0.3.1. I have the following pins set:$ opam pin list
nocrypto.0.4.0 path /home/dothraki/ocaml-nocrypto
tls.0.5.0 git https://github.com/mirleft/ocaml-tls#naming-updates
x509.0.3.1 git https://github.com/mirleft/ocaml-x509#naming
The local FS pin of nocrypto is master as of 61023ea . Running mirage-seal with this configuration results in a unikernel which crashes on boot (the results of
xl dump-core
for such a run are available if helpful):Rebuilding against a nocrypto built with with
--disable-modernity
results in a normally booting unikernel.Please let me know if there's more information I can provide, or any other way I can help.
The text was updated successfully, but these errors were encountered: