Skip to content
This repository has been archived by the owner on Jun 18, 2024. It is now read-only.

WARNING in scx_ops_bypass #153

Closed
xingyiz opened this issue Mar 5, 2024 · 1 comment
Closed

WARNING in scx_ops_bypass #153

xingyiz opened this issue Mar 5, 2024 · 1 comment

Comments

@xingyiz
Copy link

xingyiz commented Mar 5, 2024

Fuzzing with Syzkaller produces a WARNING in scx_ops_bypass that depth <= 0. Both warnings at line 3198 and 3203 are invoked. The line numbers don't match up in the reports because I have modified ext.c to include a new BPF syscall to call scx_bpf_kick_cpu, but I don't think it should affect scx_ops_bypass (in any case, I have included the modified ext.c file below).

Commit version: 047f5c2
Config: config.txt
Modified ext.c file (Uploaded as .txt): ext.txt
log0: log0.txt

The sequence of syscalls that led up to the WARNING are in log0. The file is a bit long, but in particular, the second report seems to be related to the openat$snapshot syscall.

r1 = openat$snapshot(0xffffffffffffff9c, &(0x7f00000000c0), 0x2040, 0x0)

These warnings used to appear quite a bit (before I modified the ext.c file), and they recently popped up again so I thought I would flag it out. Thank you for your time!

Full reports (there are 2):

------------[ cut here ]------------
WARNING: CPU: 0 PID: 12865 at kernel/sched/ext.c:3251 scx_ops_bypass+0x309/0x330 kernel/sched/ext.c:3247
Modules linked in:
CPU: 0 PID: 12865 Comm: syz-executor.0 Not tainted 6.8.0-rc6-gc09338b048f9-dirty #2
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Sched_ext: serialise (enabled), task: runnable_at=-11ms
RIP: 0010:scx_ops_bypass+0x309/0x330 kernel/sched/ext.c:3251
Code: ff ff f3 48 0f bc c0 89 44 24 34 83 f8 3f 0f 86 45 fe ff ff e9 de fd ff ff e8 b3 64 20 00 e9 72 fe ff ff 0f 0b e9 cd fd ff ff <0f> 0b e9 c6 fd ff ff 48 89 de 48 c7 c7 a0 15 bb 84 e8 e1 24 ba 00
RSP: 0018:ffff8880246f79a0 EFLAGS: 00010297
RAX: 00000000ffffffff RBX: ffffffff858e90a0 RCX: ffffffff811df0b6
RDX: fffffbfff0d81c58 RSI: 0000000000000004 RDI: ffffffff86c0e2c0
RBP: ffff8880246f7a60 R08: 0000000000000001 R09: fffffbfff0d81c58
R10: 0000000000000003 R11: 725f6b636f6c205d R12: 00000000fffffff8
R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff888036000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000004a9e003 CR4: 0000000000370ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 scx_pm_handler kernel/sched/ext.c:4547 [inline]
 scx_pm_handler+0x2b/0x40 kernel/sched/ext.c:4526
 notifier_call_chain+0x78/0x260 kernel/notifier.c:93
 blocking_notifier_call_chain kernel/notifier.c:388 [inline]
 blocking_notifier_call_chain+0x5c/0x80 kernel/notifier.c:376
 snapshot_release+0x101/0x140 kernel/power/user.c:125
 __fput+0x1a3/0x6c0 fs/file_table.c:376
 task_work_run+0x132/0x1d0 kernel/task_work.c:180
 exit_task_work include/linux/task_work.h:38 [inline]
 do_exit+0x75c/0x1790 kernel/exit.c:871
 do_group_exit+0x83/0x160 kernel/exit.c:1020
 get_signal+0x1be5/0x1bf0 kernel/signal.c:2893
 arch_do_signal_or_restart+0x7f/0x590 arch/x86/kernel/signal.c:310
 exit_to_user_mode_loop kernel/entry/common.c:105 [inline]
 exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
 __syscall_exit_to_user_mode_work kernel/entry/common.c:201 [inline]
 syscall_exit_to_user_mode+0x119/0x1d0 kernel/entry/common.c:212
 do_syscall_64+0x7d/0x140 arch/x86/entry/common.c:89
 entry_SYSCALL_64_after_hwframe+0x6e/0x76
RIP: 0033:0x7f102c71e70c
Code: Unable to access opcode bytes at 0x7f102c71e6e2.
RSP: 002b:00007f102b9e7b10 EFLAGS: 00000287 ORIG_RAX: 0000000000000101
RAX: 0000000000000007 RBX: 0000000000000001 RCX: 00007f102c71e70c
RDX: 0000000000080000 RSI: 00007f102c7a6a80 RDI: 00000000ffffff9c
RBP: 000000000000000c R08: 0000000000080000 R09: 00007f102c7a6a80
R10: 0000000000000000 R11: 0000000000000287 R12: 0000000000000000
R13: 000000000000006e R14: 00007f102c6de2d0 R15: 00007f102b9c8000
 </TASK>
irq event stamp: 4395
hardirqs last  enabled at (4407): [<ffffffff8124eee5>] __up_console_sem kernel/printk/printk.c:341 [inline]
hardirqs last  enabled at (4407): [<ffffffff8124eee5>] __console_unlock kernel/printk/printk.c:2706 [inline]
hardirqs last  enabled at (4407): [<ffffffff8124eee5>] console_unlock+0x285/0x290 kernel/printk/printk.c:3038
hardirqs last disabled at (4420): [<ffffffff8124eeca>] __up_console_sem kernel/printk/printk.c:339 [inline]
hardirqs last disabled at (4420): [<ffffffff8124eeca>] __console_unlock kernel/printk/printk.c:2706 [inline]
hardirqs last disabled at (4420): [<ffffffff8124eeca>] console_unlock+0x26a/0x290 kernel/printk/printk.c:3038
softirqs last  enabled at (3814): [<ffffffff81137804>] invoke_softirq kernel/softirq.c:427 [inline]
softirqs last  enabled at (3814): [<ffffffff81137804>] __irq_exit_rcu kernel/softirq.c:632 [inline]
softirqs last  enabled at (3814): [<ffffffff81137804>] irq_exit_rcu+0x84/0xb0 kernel/softirq.c:644
softirqs last disabled at (4441): [<ffffffff81137804>] invoke_softirq kernel/softirq.c:427 [inline]
softirqs last disabled at (4441): [<ffffffff81137804>] __irq_exit_rcu kernel/softirq.c:632 [inline]
softirqs last disabled at (4441): [<ffffffff81137804>] irq_exit_rcu+0x84/0xb0 kernel/softirq.c:644
---[ end trace 0000000000000000 ]---

and

------------[ cut here ]------------
WARNING: CPU: 0 PID: 12868 at kernel/sched/ext.c:3246 _raw_spin_rq_lock_irqsave kernel/sched/sched.h:1473 [inline]
WARNING: CPU: 0 PID: 12868 at kernel/sched/ext.c:3246 rq_lock_irqsave kernel/sched/sched.h:1753 [inline]
WARNING: CPU: 0 PID: 12868 at kernel/sched/ext.c:3246 scx_ops_bypass+0x302/0x330 kernel/sched/ext.c:3270
Modules linked in:
CPU: 0 PID: 12868 Comm: syz-executor.0 Tainted: G        W          6.8.0-rc6-gc09338b048f9-dirty #2
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Sched_ext: serialise (enabled), task: runnable_at=-7ms
RIP: 0010:scx_ops_bypass+0x302/0x330 kernel/sched/ext.c:3246
Code: 12 77 04 0f 84 f5 fd ff ff f3 48 0f bc c0 89 44 24 34 83 f8 3f 0f 86 45 fe ff ff e9 de fd ff ff e8 b3 64 20 00 e9 72 fe ff ff <0f> 0b e9 cd fd ff ff 0f 0b e9 c6 fd ff ff 48 89 de 48 c7 c7 a0 15
RSP: 0018:ffff8880110a77a8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffff858e90a0 RCX: ffffffff811df092
RDX: fffffbfff0d81c58 RSI: 0000000000000004 RDI: ffffffff86c0e2c0
RBP: ffff8880110a7868 R08: 0000000000000001 R09: fffffbfff0d81c58
R10: 0000000000000003 R11: 725f6b636f6c205d R12: 00000000fffffff8
R13: ffff8880110a78f0 R14: 0000000000000001 R15: 0000000000000000
FS:  00007f102ba8d640(0000) GS:ffff888036000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f102c7922d8 CR3: 000000000e3d2001 CR4: 0000000000370ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 scx_pm_handler kernel/sched/ext.c:4542 [inline]
 scx_pm_handler+0x3b/0x40 kernel/sched/ext.c:4526
 notifier_call_chain+0x78/0x260 kernel/notifier.c:93
 notifier_call_chain_robust kernel/notifier.c:128 [inline]
 blocking_notifier_call_chain_robust kernel/notifier.c:353 [inline]
 blocking_notifier_call_chain_robust+0xbd/0x150 kernel/notifier.c:341
 pm_notifier_call_chain_robust+0x2a/0x60 kernel/power/main.c:115
 snapshot_open+0x133/0x220 kernel/power/user.c:77
 misc_open+0x244/0x2a0 drivers/char/misc.c:165
 chrdev_open+0x1c5/0x420 fs/char_dev.c:414
 do_dentry_open+0x499/0xd20 fs/open.c:953
 do_open fs/namei.c:3645 [inline]
 path_openat+0x1349/0x17a0 fs/namei.c:3802
 do_filp_open+0x166/0x2c0 fs/namei.c:3829
 do_sys_openat2+0x150/0x1b0 fs/open.c:1404
 do_sys_open fs/open.c:1419 [inline]
 __do_sys_openat fs/open.c:1435 [inline]
 __se_sys_openat fs/open.c:1430 [inline]
 __x64_sys_openat+0xef/0x180 fs/open.c:1430
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0x71/0x140 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x6e/0x76
RIP: 0033:0x7f102c71ecfd
Code: c3 e8 97 2b 00 00 0f 1f 80 00 00 00 00 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f102ba8d028 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 00007f102c859f80 RCX: 00007f102c71ecfd
RDX: 0000000000002040 RSI: 00000000200000c0 RDI: ffffffffffffff9c
RBP: 00007f102c77f4bd R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000006 R14: 00007f102c859f80 R15: 00007f102ba6d000
 </TASK>
irq event stamp: 2597
hardirqs last  enabled at (2609): [<ffffffff8124eee5>] __up_console_sem kernel/printk/printk.c:341 [inline]
hardirqs last  enabled at (2609): [<ffffffff8124eee5>] __console_unlock kernel/printk/printk.c:2706 [inline]
hardirqs last  enabled at (2609): [<ffffffff8124eee5>] console_unlock+0x285/0x290 kernel/printk/printk.c:3038
hardirqs last disabled at (2620): [<ffffffff8124eeca>] __up_console_sem kernel/printk/printk.c:339 [inline]
hardirqs last disabled at (2620): [<ffffffff8124eeca>] __console_unlock kernel/printk/printk.c:2706 [inline]
hardirqs last disabled at (2620): [<ffffffff8124eeca>] console_unlock+0x26a/0x290 kernel/printk/printk.c:3038
softirqs last  enabled at (2574): [<ffffffff81137804>] invoke_softirq kernel/softirq.c:427 [inline]
softirqs last  enabled at (2574): [<ffffffff81137804>] __irq_exit_rcu kernel/softirq.c:632 [inline]
softirqs last  enabled at (2574): [<ffffffff81137804>] irq_exit_rcu+0x84/0xb0 kernel/softirq.c:644
softirqs last disabled at (2569): [<ffffffff81137804>] invoke_softirq kernel/softirq.c:427 [inline]
softirqs last disabled at (2569): [<ffffffff81137804>] __irq_exit_rcu kernel/softirq.c:632 [inline]
softirqs last disabled at (2569): [<ffffffff81137804>] irq_exit_rcu+0x84/0xb0 kernel/softirq.c:644
---[ end trace 0000000000000000 ]---
htejun added a commit that referenced this issue Mar 5, 2024
scx_pm_handler() skips calling scx_ops_bypass() if !scx_enabled(). This
means that if a SCX scheduler is loaded while PM operation is in progress,
we can get an unbalanced scx_ops_bypass(%false) and vice-versa, which is
easy to trigger using /dev/snapshot. Let's fix it by:

- Make scx_pm_handler() always call scx_ops_bypass().

- Synchronize scx_ops_bypass() against enable/disable path using
  scx_ops_enable_mutex and skip only the task update part when
  !scx_enabled().

Reported by @xingyiz in #153.
@htejun
Copy link
Collaborator

htejun commented Mar 5, 2024

Closing for now. Please reopen if the issue isn't resolved the the merged fix.

@htejun htejun closed this as completed Mar 5, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants