BUG: soft lockup in aoecmd_cfg (2)

Status: auto-obsoleted due to no activity on 2025/03/02 19:34
Subsystems: block
[Documentation on labels]
Reported-by: syzbot+5a4ee86bafe1500974eb@syzkaller.appspotmail.com
First crash: 654d, last: 618d
▶ ▼ Similar bugs (8)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-6.1 INFO: rcu detected stall in aoecmd_cfg 1 2 713d 803d 0/3 auto-obsoleted due to no activity on 2024/12/07 10:13
upstream BUG: soft lockup in aoecmd_cfg (3) prio:low block 1 27 125d 469d 0/29 auto-obsoleted due to no activity on 2026/07/19 06:30
upstream BUG: soft lockup in aoecmd_cfg block 1 1 937d 933d 0/29 auto-obsoleted due to no activity on 2024/04/17 16:28
upstream INFO: rcu detected stall in aoecmd_cfg (2) usb block 1 syz, C done 7 713d 826d 28/29 fixed on 2024/10/22 11:57
linux-5.15 INFO: rcu detected stall in aoecmd_cfg 1 1 360d 360d 0/3 auto-obsoleted due to no activity on 2025/11/25 17:18
linux-6.1 INFO: rcu detected stall in aoecmd_cfg (2) 1 1 170d 170d 0/3 auto-obsoleted due to no activity on 2026/06/03 19:48
linux-5.15 INFO: rcu detected stall in aoecmd_cfg (2) 1 2 200d 259d 0/3 auto-obsoleted due to no activity on 2026/05/05 06:23
linux-6.6 INFO: rcu detected stall in aoecmd_cfg 1 1 165d 165d 0/2 auto-obsoleted due to no activity on 2026/06/09 04:24

Sample crash report:
watchdog: BUG: soft lockup - CPU#0 stuck for 143s! [syz.6.184:6925]
Modules linked in:
irq event stamp: 11284457
hardirqs last  enabled at (11284456): [<ffffffff8bc6e753>] irqentry_exit+0x63/0x90 kernel/entry/common.c:357
hardirqs last disabled at (11284457): [<ffffffff8bc6c2fe>] sysvec_apic_timer_interrupt+0xe/0xc0 arch/x86/kernel/apic/apic.c:1049
softirqs last  enabled at (10979966): [<ffffffff81578ce4>] __do_softirq kernel/softirq.c:588 [inline]
softirqs last  enabled at (10979966): [<ffffffff81578ce4>] invoke_softirq kernel/softirq.c:428 [inline]
softirqs last  enabled at (10979966): [<ffffffff81578ce4>] __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637
softirqs last disabled at (10979969): [<ffffffff81578ce4>] __do_softirq kernel/softirq.c:588 [inline]
softirqs last disabled at (10979969): [<ffffffff81578ce4>] invoke_softirq kernel/softirq.c:428 [inline]
softirqs last disabled at (10979969): [<ffffffff81578ce4>] __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637
CPU: 0 UID: 0 PID: 6925 Comm: syz.6.184 Not tainted 6.12.0-rc4-syzkaller-00161-gae90f6a6170d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
RIP: 0010:preempt_count_sub+0x47/0x170 kernel/sched/core.c:5828
Code: df 42 0f b6 04 30 84 c0 0f 85 d0 00 00 00 83 3d 1e eb d9 18 00 75 22 65 8b 05 9d a9 a0 7e 89 c1 81 e1 ff ff ff 7f 39 d9 7c 27 <81> fb fe 00 00 00 77 07 0f b6 c0 85 c0 74 5f 65 8b 05 7b a9 a0 7e
RSP: 0018:ffffc90000007418 EFLAGS: 00000202
RAX: 0000000000000102 RBX: 0000000000000001 RCX: 0000000000000102
RDX: ffffffff91485a00 RSI: 0000000000000000 RDI: 0000000000000001
RBP: ffffc90000007500 R08: 0000000000000018 R09: ffffc900000075f0
R10: ffffc90000007550 R11: ffffffff818090c0 R12: dffffc0000000000
R13: ffffc90000007500 R14: dffffc0000000000 R15: ffffc900035ef5c0
FS:  00007f5b1ab6a6c0(0000) GS:ffff8880b8600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b3401bff8 CR3: 000000003633a000 CR4: 00000000003526f0
DR0: 0100000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
Call Trace:
 <IRQ>
 unwind_next_frame+0x18e6/0x22d0 arch/x86/kernel/unwind_orc.c:672
 arch_stack_walk+0x11c/0x150 arch/x86/kernel/stacktrace.c:25
 stack_trace_save+0x118/0x1d0 kernel/stacktrace.c:122
 kasan_save_stack mm/kasan/common.c:47 [inline]
 kasan_save_track+0x3f/0x80 mm/kasan/common.c:68
 unpoison_slab_object mm/kasan/common.c:319 [inline]
 __kasan_slab_alloc+0x66/0x80 mm/kasan/common.c:345
 kasan_slab_alloc include/linux/kasan.h:247 [inline]
 slab_post_alloc_hook mm/slub.c:4085 [inline]
 slab_alloc_node mm/slub.c:4134 [inline]
 kmem_cache_alloc_node_noprof+0x16b/0x320 mm/slub.c:4186
 kmalloc_reserve+0xa8/0x2a0 net/core/skbuff.c:587
 __alloc_skb+0x1f3/0x440 net/core/skbuff.c:678
 alloc_skb include/linux/skbuff.h:1322 [inline]
 new_skb drivers/block/aoe/aoecmd.c:66 [inline]
 aoecmd_cfg_pkts drivers/block/aoe/aoecmd.c:430 [inline]
 aoecmd_cfg+0x2d3/0xa30 drivers/block/aoe/aoecmd.c:1370
 call_timer_fn+0x18e/0x650 kernel/time/timer.c:1794
 expire_timers kernel/time/timer.c:1845 [inline]
 __run_timers kernel/time/timer.c:2419 [inline]
 __run_timer_base+0x66a/0x8e0 kernel/time/timer.c:2430
 run_timer_base kernel/time/timer.c:2439 [inline]
 run_timer_softirq+0xb7/0x170 kernel/time/timer.c:2449
 handle_softirqs+0x2c5/0x980 kernel/softirq.c:554
 __do_softirq kernel/softirq.c:588 [inline]
 invoke_softirq kernel/softirq.c:428 [inline]
 __irq_exit_rcu+0xf4/0x1c0 kernel/softirq.c:637
 irq_exit_rcu+0x9/0x30 kernel/softirq.c:649
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1049 [inline]
 sysvec_apic_timer_interrupt+0xa6/0xc0 arch/x86/kernel/apic/apic.c:1049
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:preempt_schedule_irq+0xf6/0x1c0 kernel/sched/core.c:7012
Code: 89 f5 49 c1 ed 03 eb 0d 48 f7 03 08 00 00 00 0f 84 8b 00 00 00 bf 01 00 00 00 e8 a5 31 94 f5 e8 f0 a3 cd f5 fb bf 01 00 00 00 <e8> e5 ab ff ff 43 80 7c 3d 00 00 74 08 4c 89 f7 e8 85 9b 2f f6 48
RSP: 0018:ffffc900035ef5c0 EFLAGS: 00000282
RAX: 6c24f20b4d4edf00 RBX: 1ffff920006bdec0 RCX: ffffffff8170be0a
RDX: dffffc0000000000 RSI: ffffffff8c0acb40 RDI: 0000000000000001
RBP: ffffc900035ef670 R08: ffffffff942ca807 R09: 1ffffffff2859500
R10: dffffc0000000000 R11: fffffbfff2859501 R12: 1ffff920006bdeb8
R13: 1ffff920006bdebc R14: ffffc900035ef5e0 R15: dffffc0000000000
 irqentry_exit+0x5e/0x90 kernel/entry/common.c:354
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:702
RIP: 0010:preempt_schedule+0x0/0xf0 kernel/sched/core.c:6886
Code: 00 00 e8 e3 35 94 f5 c3 cc cc cc cc 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 <f3> 0f 1e fa 55 48 89 e5 41 57 41 56 41 54 53 48 83 e4 e0 48 83 ec
RSP: 0018:ffffc900035ef730 EFLAGS: 00000287
RAX: ffffffff8188fc84 RBX: 0000000000000000 RCX: 0000000000040000
RDX: ffffc9001918f000 RSI: 00000000000045ea RDI: 00000000000045eb
RBP: ffffc900035ef780 R08: ffffffff8188fc69 R09: 1ffffffff2859500
R10: dffffc0000000000 R11: fffffbfff2859501 R12: 0000000000000000
R13: ffffffff931bda4a R14: 0000000000000000 R15: ffffffff901d0268
 preempt_schedule_thunk+0x1a/0x30 arch/x86/entry/thunk.S:12
 on_each_cpu_cond_mask+0x79/0x80 kernel/smp.c:1052
 on_each_cpu include/linux/smp.h:71 [inline]
 text_poke_sync arch/x86/kernel/alternative.c:2085 [inline]
 text_poke_bp_batch+0x726/0xb30 arch/x86/kernel/alternative.c:2357
 text_poke_flush arch/x86/kernel/alternative.c:2486 [inline]
 text_poke_finish+0x30/0x50 arch/x86/kernel/alternative.c:2493
 arch_jump_label_transform_apply+0x1c/0x30 arch/x86/kernel/jump_label.c:146
 static_key_enable_cpuslocked+0x136/0x260 kernel/jump_label.c:210
 static_key_enable+0x1a/0x20 kernel/jump_label.c:223
 tracepoint_add_func+0x953/0x9e0 kernel/tracepoint.c:361
 tracepoint_probe_register_prio kernel/tracepoint.c:511 [inline]
 tracepoint_probe_register+0x105/0x160 kernel/tracepoint.c:531
 perf_trace_event_reg kernel/trace/trace_event_perf.c:129 [inline]
 perf_trace_event_init+0x478/0x930 kernel/trace/trace_event_perf.c:202
 perf_trace_init+0x243/0x2e0 kernel/trace/trace_event_perf.c:226
 perf_tp_event_init+0x8d/0x110 kernel/events/core.c:10357
 perf_try_init_event+0x146/0x810 kernel/events/core.c:11891
 perf_init_event kernel/events/core.c:11977 [inline]
 perf_event_alloc+0x135f/0x2310 kernel/events/core.c:12259
 __do_sys_perf_event_open kernel/events/core.c:12766 [inline]
 __se_sys_perf_event_open+0xb1f/0x3870 kernel/events/core.c:12657
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f5b19d7e719
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f5b1ab6a038 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 00007f5b19f35f80 RCX: 00007f5b19d7e719
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000020000040
RBP: 00007f5b19df132e R08: 0000000000000000 R09: 0000000000000000
R10: ffffffffffffffff R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007f5b19f35f80 R15: 00007fffc3e5d6b8
 </TASK>
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 UID: 0 PID: 6938 Comm: syz-executor Not tainted 6.12.0-rc4-syzkaller-00161-gae90f6a6170d #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/13/2024
RIP: 0010:rcu_is_watching+0x44/0xb0 kernel/rcu/tree.c:737
Code: 73 7a 49 bf 00 00 00 00 00 fc ff df 4c 8d 34 dd 50 fa 31 8e 4c 89 f0 48 c1 e8 03 42 80 3c 38 00 74 08 4c 89 f7 e8 6c cf 83 00 <48> c7 c3 98 7e 03 00 49 03 1e 48 89 d8 48 c1 e8 03 42 0f b6 04 38
RSP: 0018:ffffc90002fef4e0 EFLAGS: 00000046
RAX: 1ffffffff1c63f4b RBX: 0000000000000001 RCX: ffffffff81706d60
RDX: 0000000000000000 RSI: ffffffff8c610340 RDI: ffffffff8c610300
RBP: ffffc90002fef640 R08: ffffffff901d026f R09: 1ffffffff203a04d
R10: dffffc0000000000 R11: fffffbfff203a04e R12: 1ffff920005fdeac
R13: ffffffff81f31ed0 R14: ffffffff8e31fa58 R15: dffffc0000000000
FS:  0000555566ceb500(0000) GS:ffff8880b8700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3614535f40 CR3: 00000000589d6000 CR4: 00000000003526f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <NMI>
 </NMI>
 <TASK>
 trace_lock_release include/trace/events/lock.h:69 [inline]
 lock_release+0xbf/0xa30 kernel/locking/lockdep.c:5836
 local_lock_release include/linux/local_lock_internal.h:38 [inline]
 put_cpu_partial+0x13a/0x250 mm/slub.c:3217
 __slab_free+0x2ea/0x3d0 mm/slub.c:4449
 qlink_free mm/kasan/quarantine.c:163 [inline]
 qlist_free_all+0x9a/0x140 mm/kasan/quarantine.c:179
 kasan_quarantine_reduce+0x14f/0x170 mm/kasan/quarantine.c:286
 __kasan_slab_alloc+0x23/0x80 mm/kasan/common.c:329
 kasan_slab_alloc include/linux/kasan.h:247 [inline]
 slab_post_alloc_hook mm/slub.c:4085 [inline]
 slab_alloc_node mm/slub.c:4134 [inline]
 kmem_cache_alloc_noprof+0x135/0x2a0 mm/slub.c:4141
 vm_area_alloc+0x24/0x1d0 kernel/fork.c:472
 mmap_region+0x11c0/0x2a30 mm/mmap.c:1429
 do_mmap+0x8f0/0x1000 mm/mmap.c:496
 vm_mmap_pgoff+0x1dd/0x3d0 mm/util.c:588
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7f8d21b7e753
Code: f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 41 89 ca 41 f7 c1 ff 0f 00 00 75 14 b8 09 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 25 c3 0f 1f 40 00 48 c7 c0 a8 ff ff ff 64 c7
RSP: 002b:00007ffed88ad688 EFLAGS: 00000246 ORIG_RAX: 0000000000000009
RAX: ffffffffffffffda RBX: 00007f8d21d35f40 RCX: 00007f8d21b7e753
RDX: 0000000000000000 RSI: 0000000000802000 RDI: 0000000000000000
RBP: 00007f8d21d36a38 R08: 00000000ffffffff R09: 0000000000000000
R10: 0000000000000022 R11: 0000000000000246 R12: 000000000000000c
R13: 0000000000000003 R14: 0000000000000009 R15: 0000000000000000
 </TASK>

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/10/27 22:18 bpf ae90f6a6170d 65e8686b .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-kasan-gce BUG: soft lockup in aoecmd_cfg
2024/12/02 19:34 bpf-next 3bfb49d73fc2 bb326ffb .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
2024/11/21 14:27 bpf-next 2c8b09ac2537 4b25d554 .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-bpf-next-kasan-gce BUG: soft lockup in aoecmd_cfg
* Struck through repros no longer work on HEAD.