[RISC-V | TEEs] Crashes when memory is protected by PMP.

[neiling]

Setup
Device: QEMU emulator version 7.1.0
OS: Debian Sid
Kernel: 5.18.1

Issue
I have encountered a problem regarding RISC-V and Physical Memory Protection (PMP),
which is used to protect memory space from the OS. This is mainly used for Trusted
Execution Environments (TEEs) such as Keystone. If LiME encounters a read protected
area in memory while creating the memory dump, a "load access fault" exception occurs
under Linux, but there is no further handling by LiME, which leads to the abort of
the dump and actually the whole system.

A possible workaround where zero bytes are written to the dump for the read protected
areas can be found in the Pull request 106 .

[  553.010774] [LiME] Parameters
[  553.010951] [LiME]   PATH: /root/dump.lime
[  553.011107] [LiME]   DIO: 0
[  553.011217] [LiME]   FORMAT: lime
[  553.011337] [LiME]   LOCALHOSTONLY: 0
[  553.011471] [LiME]   DIGEST: (null)
[  553.011599] [LiME]   TIMEOUT: 1000
[  553.011702] [LiME]   COMPRESS: 0
[  553.011844] [LiME] Initializing Dump...
[  553.011981] [LiME] Direct IO Disabled
[  553.018235] [LiME] Writing range 80200000 - 17fffffff.
[  576.195305] Oops - load access fault [#1]
[  576.195649] Modules linked in: lime(O+) keystone_driver(O) fuse
[  576.196243] CPU: 0 PID: 518 Comm: insmod Tainted: G           O      5.18.1 #4
[  576.196703] Hardware name: riscv-virtio,qemu (DT)
[  576.197052] epc : __memcpy+0x3c/0xf8
[  576.197346]  ra : copy_page_from_iter_atomic+0x4fa/0x5e6
[  576.197854] epc : ffffffff80692580 ra : ffffffff805fd376 sp : ff20000010b037b0
[  576.198409]  gp : ffffffff81a3d398 tp : ff6000008272d400 t0 : ffffffffe9bd66e7
[  576.198911]  t1 : ff600000722e5020 t2 : ffffffffb57d105c s0 : ff20000010b03870
[  576.199456]  s1 : ff600000722e5020 a0 : ff600000722e5020 a1 : ff60000082000000
[  576.199868]  a2 : 0000000000000fe0 a3 : ff60000082000f80 a4 : 0000000000000f80
[  576.200366]  a5 : 0004000000000000 a6 : ff20000000000000 a7 : 0000000000000001
[  576.200738]  s2 : ff20000010b039f8 s3 : 0000000000000fe0 s4 : ff20000010b039f8
[  576.201070]  s5 : 0000000000000000 s6 : ff20000010b039f8 s7 : ff20000010b039e8
[  576.201377]  s8 : 0000000000001000 s9 : 0000000000000fe0 s10: 0000000000000000
[  576.201841]  s11: ffffffff81418e08 t3 : ff60000000000000 t4 : 0000000000001000
[  576.202345]  t5 : 0000000000000001 t6 : ff600000722e5020
[  576.202714] status: 0000000200000120 badaddr: ff60000082000000 cause: 0000000000000005
[  576.203376] [<ffffffff8014977a>] generic_perform_write+0xd8/0x196
[  576.203630] [<ffffffff80269b92>] ext4_buffered_write_iter+0x80/0xee
[  576.203853] [<ffffffff8026a0f2>] ext4_file_write_iter+0xd6/0x64c
[  576.204057] [<ffffffff801b941e>] __kernel_write+0xdc/0x202
[  576.204249] [<ffffffff801b9594>] kernel_write+0x50/0x16e
[  576.204564] [<ffffffff01c573f2>] write_vaddr_disk+0x42/0x5c [lime]
[  576.204992] [<ffffffff01c57998>] try_write.part.0+0x58/0xac [lime]
[  576.205210] [<ffffffff01c57a7e>] write_vaddr+0x92/0xa2 [lime]
[  576.205471] [<ffffffff01c5801c>] init_module+0x502/0x6c0 [lime]
[  576.205930] [<ffffffff800026ca>] do_one_initcall+0x44/0x1a4
[  576.206302] [<ffffffff8009a04e>] do_init_module+0x54/0x260
[  576.206667] [<ffffffff8009c200>] load_module+0x1e76/0x255e
[  576.207033] [<ffffffff8009ca74>] __do_sys_finit_module+0x8a/0xb0
[  576.207392] [<ffffffff8009cabe>] sys_finit_module+0x24/0x2c
[  576.207683] [<ffffffff800037b8>] ret_from_syscall+0x0/0x2
[  576.208450] ---[ end trace 0000000000000000 ]---

Message from syslogd@debian at Oct 20 11:40:50 ...
 kernel:[  576.195305] Oops - load access fault [#1]

[JumboMite]

@neiling I'm using an ARM Android device, and think I'm experiencing a similar issue

#117

The RAM dump always stops after 82MB and then crashes and reboots the device. The TEE is at 0x5100000, and I think it's crashing when it reaches this area. The device reboots when trying to read this region of secmon in uboot too.

Is it possible to apply your fix more generally to ARM? Or to specific a memory range to skip/write with zeros?

Thank you

[Panchajanya1999]

@neiling I'm using an ARM Android device, and think I'm experiencing a similar issue

#117

The RAM dump always stops after 82MB and then crashes and reboots the device. The TEE is at 0x5100000, and I think it's crashing when it reaches this area. The device reboots when trying to read this region of secmon in uboot too.

Is it possible to apply your fix more generally to ARM? Or to specific a memory range to skip/write with zeros?

Thank you

Hi, I am facing same issue on a samsung 5.4 device. It is s21fe 888. I am attaching the error

<4>[ 175.270930] [5: insmod:14791] CPU: 5 PID: 14791 Comm: insmod Tainted: G W 5.4.259-AQUA-perf@v2.1 #3 <4>[ 175.270935] [5: insmod:14791] Hardware name: Samsung R9Q PROJECT (board-id,14) (DT) <4>[ 175.270941] [5: insmod:14791] Call trace: <4>[ 175.270952] [5: insmod:14791] dump_backtrace+0x0/0x2a0 <4>[ 175.270959] [5: insmod:14791] show_stack+0x14/0x20 <4>[ 175.270967] [5: insmod:14791] dump_stack+0xc0/0xf8 <4>[ 175.270976] [5: insmod:14791] panic+0x198/0x3cc <4>[ 175.270983] [5: insmod:14791] do_sched_setscheduler+0x0/0x110 <4>[ 175.270990] [5: insmod:14791] __schedule+0x584/0x7e0 <4>[ 175.270996] [5: insmod:14791] schedule+0x88/0xc0 <4>[ 175.271004] [5: insmod:14791] fuse_wait_on_page_writeback+0x144/0x1a0 <4>[ 175.271011] [5: insmod:14791] fuse_write_begin+0x6c/0x1e0 <4>[ 175.271019] [5: insmod:14791] generic_perform_write+0x9c/0x1b0 <4>[ 175.271025] [5: insmod:14791] __generic_file_write_iter+0xb8/0x170 <4>[ 175.271031] [5: insmod:14791] generic_file_write_iter+0x104/0x170 <4>[ 175.271037] [5: insmod:14791] fuse_file_write_iter+0x214/0x3b0 <4>[ 175.271044] [5: insmod:14791] __vfs_write+0x178/0x1b0 <4>[ 175.271050] [5: insmod:14791] vfs_write+0xe0/0x1b0 <4>[ 175.271056] [5: insmod:14791] kernel_write+0x3c/0xb0 <4>[ 175.271071] [5: insmod:14791] write_vaddr_disk+0x44/0x80 [lime] <4>[ 175.271080] [5: insmod:14791] write_vaddr+0xe4/0x150 [lime] <4>[ 175.271088] [5: insmod:14791] init_module+0x3ac/0x580 [lime] <4>[ 175.271095] [5: insmod:14791] do_one_initcall+0xd4/0x390 <4>[ 175.271103] [5: insmod:14791] do_init_module+0x44/0x1f0 <4>[ 175.271110] [5: insmod:14791] load_module+0x1870/0x1a70 <4>[ 175.271117] [5: insmod:14791] __arm64_sys_finit_module+0xb0/0xf0 <4>[ 175.271124] [5: insmod:14791] el0_svc_common+0xa4/0x170 <4>[ 175.271131] [5: insmod:14791] el0_svc_handler+0x20/0x80 <4>[ 175.271137] [5: insmod:14791] el0_svc+0x8/0x200 <2>[ 175.271152] [5: insmod:14791] SMP: stopping secondary CPUs