Panic on loading module for /proc/meminfo example on vanilla 4.1 #497
Comments
|
This is a weird one. I'm not able to recreate.
You can do something like: |
|
❦ 17 août 2015 15:04 -0700, Josh Poimboeuf notifications@github.com :
I have no problem with the kpatch module. It's the other one that makes I am using gcc 4.9.3 from Debian: My .config is quite small: Habit is habit, and not to be flung out of the window by any man, but coaxed |
|
Looking at the the oops, it appears the faulting instruction was: which corresponds to: The invalid mod pointer (rbx) was The surrounding code was: It's very odd that the mod pointer (0xffffffffa0010cc0) seemed to be valid for the earlier @vincentbernat are you still able to recreate? |
|
There's one more thing that doesn't make sense to me, from looking at the assembly listing: After the call to do_one_initcall(), it does a |
|
(that assumes my assembly listing is the same as yours) |
|
Can I try again with a 4.2? |
|
Sure. BTW was this on a VM? |
|
Yes, this is in a VM. I now have gcc-5.2 on my system and I run into additional difficulties. When I try to run I could try to use gcc-4.9 but it seems that kpatch-build doesn't honor the I am a bit low on time currently. Feel free to postpone/close the issue if things don't add up. It is likely that I screw up at some point. |
|
Gah, sorry. I've made a lot of fixes in the last week and I probably broke Debian. If you have a chance to try, I think reverting commit 1704498 will fix your new problem. |
|
You could also try doing |
|
I have the same issue when CONFIG_DEBUG_SET_MODULE_RONX is not set. When it's set, the bug goes away. I'm using livepatch. Tested on 4.1, 4.2 and 4.3, on Debian 8 (with a custom kernel). By default, Debian has CONFIG_DEBUG_SET_MODULE_RONX=y. |
|
@cbay, thanks for isolating the problem. I should have a fix soon. |
|
@cbay any chance you can test with the following patch? |
|
Oh, you're using livepatch? Guess we need a fix there as well. |
|
(and this patch won't help you) |
|
Yes, I'm using livepatch. If I remember correctly, I did test with the kpatch module and it didn't cause the bug. |
|
Assuming I understand the problem correctly, it affects both kpatch.ko and livepatch, but it's intermittent, and may be dependent on what patch you use as well as how memory is laid out. It requires a dynamic relocation write being performed on the same page of memory that holds the struct module. After writing the dynamic relocation, we set the page to be read-only, which causes the next write to the module struct "mod->state = MODULE_STATE_LIVE" to fail. |
|
OK. As far as I'm concerned, I was testing the example patch (modifying /proc/meminfo). Let me know if you want me to test a patch for livepatch. |
|
@cbay can you try this livepatch fix? |
|
@cbay Also, if you care to send me your email address, I can credit you with "Reported-by" and "Tested-by" in the upstream livepatch commit log. |
|
The patch fixes the issue for me. I tested it on Linux 4.3 (the patch didn't apply cleanly, though). Thanks! My email is cbay at alwaysdata.com. |
jpoimboe
added a commit
to jpoimboe/kpatch
that referenced
this issue
Nov 3, 2015
When loading a patch module on a kernel with !CONFIG_DEBUG_SET_MODULE_RONX, the following crash occurs: loading core module: /root/src/kpatch/kpatch/../kmod/core/kpatch.ko loading patch module: kpatch-meminfo-string.ko BUG: unable to handle kernel paging request at ffffffffa0010cc0 IP: [<ffffffff8125ecb0>] do_init_module+0x84/0x1af PGD 13d3067 PUD 13d4063 PMD 1e1ee067 PTE 1e1a0161 Oops: 0003 [#1] Modules linked in: kpatch_meminfo_string(O+) kpatch(O) CPU: 0 PID: 149 Comm: insmod Tainted: G O K 4.1.0+ #1 task: ffff88001e17b810 ti: ffff88001e1cc000 task.ti: ffff88001e1cc000 RIP: 0010:[<ffffffff8125ecb0>] [<ffffffff8125ecb0>] do_init_module+0x84/0x1af RSP: 0018:ffff88001e1cfda8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffffffffa0010cc0 RCX: 0000000080a02001 RDX: 0000000000000024 RSI: 0000000000000000 RDI: ffffffff813fabe0 RBP: 0000000000000000 R08: 0000000000000000 R09: 00000000d0000000 R10: ffffffffa000e000 R11: 0000000000000001 R12: ffff88001eb58638 R13: ffffffffa0010d10 R14: 0000000000000001 R15: 0000000000000000 FS: 00007f0ae00aa700(0000) GS:ffffffff813e1000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffa0010cc0 CR3: 000000001e181000 CR4: 00000000000006b0 Stack: ffff88001e1cfed8 0000000000000001 ffffffffa0010cc0 ffffffff81058aac ffff88001e207680 00000000810a462f ffffc90000096890 0000000000000e00 ffffffff00000016 ffffffff8126cd40 ffff88001eaa6a08 ffff88001e1cfe48 Call Trace: [<ffffffff81058aac>] ? load_module+0x18ad/0x18e9 [<ffffffff81056290>] ? copy_module_from_fd+0x86/0xdf [<ffffffff81058c1e>] ? SyS_finit_module+0x56/0x61 [<ffffffff81261854>] ? system_call_fastpath+0x12/0x6a Code: f8 00 00 00 74 23 49 c7 c0 80 ca 26 81 48 8d 53 18 89 c1 4c 89 c6 48 c7 c7 6d ef 36 81 31 c0 e8 16 fb ff ff e8 18 06 00 00 31 f6 <c7> 03 00 00 00 00 48 89 da 48 c7 c7 c0 c9 3f 81 e8 7e b3 dd ff RIP [<ffffffff8125ecb0>] do_init_module+0x84/0x1af RSP <ffff88001e1cfda8> CR2: ffffffffa0010cc0 With !CONFIG_DEBUG_SET_MODULE_RONX, module text and rodata pages are writable, and the debug_align() macro allows the module struct to share a page with executable text. When klp_write_module_reloc() calls set_memory_ro() on the page, it effectively turns the module struct into a read-only structure, resulting in a page fault when load_module() does "mod->state = MODULE_STATE_LIVE". Fixes: dynup#497
arges
pushed a commit
to arges/kpatch
that referenced
this issue
Dec 10, 2015
When loading a patch module on a kernel with !CONFIG_DEBUG_SET_MODULE_RONX, the following crash occurs: loading core module: /root/src/kpatch/kpatch/../kmod/core/kpatch.ko loading patch module: kpatch-meminfo-string.ko BUG: unable to handle kernel paging request at ffffffffa0010cc0 IP: [<ffffffff8125ecb0>] do_init_module+0x84/0x1af PGD 13d3067 PUD 13d4063 PMD 1e1ee067 PTE 1e1a0161 Oops: 0003 [dynup#1] Modules linked in: kpatch_meminfo_string(O+) kpatch(O) CPU: 0 PID: 149 Comm: insmod Tainted: G O K 4.1.0+ dynup#1 task: ffff88001e17b810 ti: ffff88001e1cc000 task.ti: ffff88001e1cc000 RIP: 0010:[<ffffffff8125ecb0>] [<ffffffff8125ecb0>] do_init_module+0x84/0x1af RSP: 0018:ffff88001e1cfda8 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffffffffa0010cc0 RCX: 0000000080a02001 RDX: 0000000000000024 RSI: 0000000000000000 RDI: ffffffff813fabe0 RBP: 0000000000000000 R08: 0000000000000000 R09: 00000000d0000000 R10: ffffffffa000e000 R11: 0000000000000001 R12: ffff88001eb58638 R13: ffffffffa0010d10 R14: 0000000000000001 R15: 0000000000000000 FS: 00007f0ae00aa700(0000) GS:ffffffff813e1000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: ffffffffa0010cc0 CR3: 000000001e181000 CR4: 00000000000006b0 Stack: ffff88001e1cfed8 0000000000000001 ffffffffa0010cc0 ffffffff81058aac ffff88001e207680 00000000810a462f ffffc90000096890 0000000000000e00 ffffffff00000016 ffffffff8126cd40 ffff88001eaa6a08 ffff88001e1cfe48 Call Trace: [<ffffffff81058aac>] ? load_module+0x18ad/0x18e9 [<ffffffff81056290>] ? copy_module_from_fd+0x86/0xdf [<ffffffff81058c1e>] ? SyS_finit_module+0x56/0x61 [<ffffffff81261854>] ? system_call_fastpath+0x12/0x6a Code: f8 00 00 00 74 23 49 c7 c0 80 ca 26 81 48 8d 53 18 89 c1 4c 89 c6 48 c7 c7 6d ef 36 81 31 c0 e8 16 fb ff ff e8 18 06 00 00 31 f6 <c7> 03 00 00 00 00 48 89 da 48 c7 c7 c0 c9 3f 81 e8 7e b3 dd ff RIP [<ffffffff8125ecb0>] do_init_module+0x84/0x1af RSP <ffff88001e1cfda8> CR2: ffffffffa0010cc0 With !CONFIG_DEBUG_SET_MODULE_RONX, module text and rodata pages are writable, and the debug_align() macro allows the module struct to share a page with executable text. When klp_write_module_reloc() calls set_memory_ro() on the page, it effectively turns the module struct into a read-only structure, resulting in a page fault when load_module() does "mod->state = MODULE_STATE_LIVE". Fixes: dynup#497
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello!
Still on the same vanilla 4.1 kernel, I get this when loading the patch with
kpatch load:I have tried to debug a bit, but I have no clue on how to load the symbols from the module while I can't get
.textand.datafrom sysfs. Is there another way to get the address the.textsection was loaded?If I casually trace with gdb, the kernel doesn't panic when initializing the module per-se, but a bit latter:
The text was updated successfully, but these errors were encountered: