INFO: rcu detected stall in process_one_work

Status: auto-closed as invalid on 2019/11/14 14:39
Subsystems: kernel
[Documentation on labels]
Reported-by: syzbot+0efa75891ba0cdaa377b@syzkaller.appspotmail.com
First crash: 2986d, last: 2539d
▶ ▼ Similar bugs (10)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in process_one_work 1 3 770d 873d 0/3 auto-obsoleted due to no activity on 2024/09/27 05:22
linux-5.15 INFO: rcu detected stall in process_one_work (3) origin:upstream 1 syz error 3 203d 319d 0/3 upstream: reported syz repro on 2025/09/13 13:36
upstream INFO: rcu detected stall in process_one_work (2) kernel 1 1 2300d 2300d 0/29 auto-closed as invalid on 2020/07/10 22:43
upstream INFO: rcu detected stall in process_one_work (4) kernel 1 1 1774d 1774d 0/29 auto-closed as invalid on 2021/12/18 04:26
upstream INFO: rcu detected stall in process_one_work (8) kernel 1 3 953d 1002d 0/29 auto-obsoleted due to no activity on 2024/03/18 23:07
linux-5.15 INFO: rcu detected stall in process_one_work (2) 1 1 589d 589d 0/3 auto-obsoleted due to no activity on 2025/03/27 21:48
upstream INFO: rcu detected stall in process_one_work (9) prio:high kernel 1 syz, C error error 21 18d 786d 0/29 upstream: reported C repro on 2024/06/03 12:55
upstream INFO: rcu detected stall in process_one_work (5) kernel 1 1 1420d 1420d 0/29 auto-obsoleted due to no activity on 2022/12/07 18:00
linux-6.1 INFO: rcu detected stall in process_one_work origin:upstream 1 syz, C error 3 1d05h 932d 0/3 upstream: reported C repro on 2024/01/09 18:19
upstream INFO: rcu detected stall in process_one_work (3) kernel 1 syz, C done done 1 2121d 2163d 15/29 fixed on 2020/11/16 12:12

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
	(detected by 1, t=11091 jiffies, g=139073, q=481)
rcu: All QSes seen, last rcu_preempt kthread activity 11091 (4295019003-4295007912), jiffies_till_next_fqs=1, root ->qsmask 0x0
kworker/1:3     R  running task    27160  9619      2 0x80004008
Workqueue: rcu_gp process_srcu
Call Trace:
 <IRQ>
 sched_show_task+0x348/0x400 kernel/sched/core.c:5755
 print_other_cpu_stall kernel/rcu/tree_stall.h:410 [inline]
 check_cpu_stall kernel/rcu/tree_stall.h:536 [inline]
 rcu_pending kernel/rcu/tree.c:2736 [inline]
 rcu_sched_clock_irq+0x1597/0x16d0 kernel/rcu/tree.c:2183
 update_process_times+0x134/0x190 kernel/time/timer.c:1639
 tick_sched_handle kernel/time/tick-sched.c:167 [inline]
 tick_sched_timer+0x263/0x420 kernel/time/tick-sched.c:1296
 __run_hrtimer kernel/time/hrtimer.c:1389 [inline]
 __hrtimer_run_queues+0x403/0x850 kernel/time/hrtimer.c:1451
 hrtimer_interrupt+0x38c/0xda0 kernel/time/hrtimer.c:1509
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1068 [inline]
 smp_apic_timer_interrupt+0x109/0x280 arch/x86/kernel/apic/apic.c:1093
 apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:830
 </IRQ>
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x5f/0x80 kernel/locking/spinlock.c:199
Code: 00 00 00 00 fc ff df 80 3c 08 00 74 0c 48 c7 c7 90 87 8a 88 e8 a2 5a 31 fa 48 83 3d 6a 6a 0f 01 00 74 26 fb 66 0f 1f 44 00 00 <bf> 01 00 00 00 e8 67 61 d5 f9 65 8b 05 8c e0 86 78 85 c0 74 03 5b
RSP: 0018:ffff888060ee7bb0 EFLAGS: 00000282 ORIG_RAX: ffffffffffffff13
RAX: 1ffffffff11150f2 RBX: ffffffff88901500 RCX: dffffc0000000000
RDX: 0000000000000000 RSI: ffffffff8160b72a RDI: ffffffff877b1cf7
RBP: ffff888060ee7bb8 R08: ffffffff81790484 R09: fffffbfff11202a1
R10: fffffbfff11202a1 R11: 0000000000000000 R12: ffffffff88901198
R13: dffffc0000000000 R14: 000000000000142a R15: ffffffff88901500
 srcu_reschedule+0x9a/0xf0 kernel/rcu/srcutree.c:1220
 process_srcu+0x566/0xd20 kernel/rcu/srcutree.c:1238
 process_one_work+0x7ef/0x10e0 kernel/workqueue.c:2269
 worker_thread+0xc01/0x1630 kernel/workqueue.c:2415
 kthread+0x332/0x350 kernel/kthread.c:255
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
rcu: rcu_preempt kthread starved for 11091 jiffies! g139073 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: RCU grace-period kthread stack dump:
rcu_preempt     R  running task    28896    10      2 0x80004000
Call Trace:
 context_switch kernel/sched/core.c:3254 [inline]
 __schedule+0x877/0xc50 kernel/sched/core.c:3880
 schedule+0x131/0x1e0 kernel/sched/core.c:3944
 schedule_timeout+0x14f/0x240 kernel/time/timer.c:1807
 rcu_gp_fqs_loop kernel/rcu/tree.c:1611 [inline]
 rcu_gp_kthread+0xef8/0x1790 kernel/rcu/tree.c:1768
 kthread+0x332/0x350 kernel/kthread.c:255
 ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352

Crashes (8):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2019/08/16 14:38 upstream 329120423947 8fd428a1 .config console log report ci-upstream-kasan-gce-smack-root
2019/08/10 03:47 upstream 7f20fd23377a acb51638 .config console log report ci-upstream-kasan-gce-smack-root
2019/08/02 21:20 upstream 1e78030e5e5b 835dffe7 .config console log report ci-upstream-kasan-gce-smack-root
2019/03/20 03:27 upstream b7a42146dca3 2458c1c6 .config console log report ci-upstream-kasan-gce-root
2019/03/02 11:46 upstream a215ce8f0e00 1c0e457a .config console log report ci-upstream-kasan-gce-root
2019/02/26 23:58 upstream 7d762d69145a f2468c12 .config console log report ci-upstream-kasan-gce-selinux-root
2019/01/12 18:49 upstream 4b3c31c8d4dd c3f3344c .config console log report ci-upstream-kasan-gce-selinux-root
2018/05/26 19:43 upstream bc2dbc5420e8 f48c20b8 .config console log report ci-upstream-kasan-gce-root
* Struck through repros no longer work on HEAD.