INFO: rcu detected stall in process_one_work (3)

Status: upstream: reported syz repro on 2025/09/13 13:36
Bug presence: origin:upstream
[Documentation on labels]
Reported-by: syzbot+873957f63b11eeb615f4@syzkaller.appspotmail.com
First crash: 319d, last: 203d
Fix bisection: failed (error log, bisect log)
  
▶ ▼ Bug presence (1)
Date Name Commit Repro Result
2025/09/13 upstream (ToT) 5cd64d4f9268 syz [report] INFO: rcu detected stall in tc_modify_qdisc
▶ ▼ Similar bugs (10)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-5.15 INFO: rcu detected stall in process_one_work 1 3 770d 873d 0/3 auto-obsoleted due to no activity on 2024/09/27 05:22
upstream INFO: rcu detected stall in process_one_work (2) kernel 1 1 2299d 2299d 0/29 auto-closed as invalid on 2020/07/10 22:43
upstream INFO: rcu detected stall in process_one_work (4) kernel 1 1 1774d 1774d 0/29 auto-closed as invalid on 2021/12/18 04:26
upstream INFO: rcu detected stall in process_one_work (8) kernel 1 3 952d 1002d 0/29 auto-obsoleted due to no activity on 2024/03/18 23:07
linux-5.15 INFO: rcu detected stall in process_one_work (2) 1 1 588d 588d 0/3 auto-obsoleted due to no activity on 2025/03/27 21:48
upstream INFO: rcu detected stall in process_one_work kernel 1 8 2539d 2981d 0/29 auto-closed as invalid on 2019/11/14 14:39
upstream INFO: rcu detected stall in process_one_work (9) prio:high kernel 1 syz, C error error 21 17d 786d 0/29 upstream: reported C repro on 2024/06/03 12:55
upstream INFO: rcu detected stall in process_one_work (5) kernel 1 1 1419d 1419d 0/29 auto-obsoleted due to no activity on 2022/12/07 18:00
linux-6.1 INFO: rcu detected stall in process_one_work origin:upstream 1 syz, C error 3 19h08m 931d 0/3 upstream: reported C repro on 2024/01/09 18:19
upstream INFO: rcu detected stall in process_one_work (3) kernel 1 syz, C done done 1 2121d 2163d 15/29 fixed on 2020/11/16 12:12
▶ ▼ Last patch testing requests (4)
Created Duration User Patch Repo Result
2026/07/04 07:04 22m retest repro linux-5.15.y report log
2026/04/17 14:36 2h22m retest repro linux-5.15.y report log
2025/12/07 00:28 15m retest repro linux-5.15.y report log
2025/09/27 16:22 14m retest repro linux-5.15.y report log

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	0-...!: (2 ticks this GP) idle=ca1/1/0x4000000000000000 softirq=6852/6852 fqs=69 
	(detected by 1, t=10502 jiffies, g=7673, q=284)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 1325 Comm: kworker/0:2 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/18/2025
Workqueue: rcu_gp process_srcu
RIP: 0010:__lock_acquire+0x189/0x7c60 kernel/locking/lockdep.c:4911
Code: ff 49 b8 00 00 00 00 00 fc ff df 49 89 c7 48 85 c0 0f 84 79 58 00 00 44 89 64 24 68 4d 8d b5 e8 0a 00 00 4c 89 f0 48 c1 e8 03 <48> 89 84 24 88 00 00 00 42 0f b6 04 00 84 c0 0f 85 91 64 00 00 45
RSP: 0018:ffffc900000078c0 EFLAGS: 00000802
RAX: 1ffff110044a7515 RBX: ffff8880221c5300 RCX: 1ffff92000000f3c
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8880221c5300
RBP: ffffc90000007b10 R08: dffffc0000000000 R09: 0000000000000001
R10: fffffbfff1ad323e R11: 1ffffffff1ad323d R12: 0000000000000001
R13: ffff888022539dc0 R14: ffff88802253a8a8 R15: ffffffff8f9bc940
FS:  0000000000000000(0000) GS:ffff8880b9000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055ca1f33e8e8 CR3: 000000007cfbe000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <IRQ>
 lock_acquire+0x197/0x3f0 kernel/locking/lockdep.c:5623
 __raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
 _raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:154
 spin_lock include/linux/spinlock.h:363 [inline]
 advance_sched+0x47/0x940 net/sched/sch_taprio.c:716
 __run_hrtimer kernel/time/hrtimer.c:1690 [inline]
 __hrtimer_run_queues+0x53d/0xc40 kernel/time/hrtimer.c:1754
 hrtimer_interrupt+0x3bb/0x8d0 kernel/time/hrtimer.c:1816
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1097 [inline]
 __sysvec_apic_timer_interrupt+0x137/0x4a0 arch/x86/kernel/apic/apic.c:1114
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1108 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1108
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:676
RIP: 0010:__raw_spin_unlock_irq include/linux/spinlock_api_smp.h:169 [inline]
RIP: 0010:_raw_spin_unlock_irq+0x25/0x40 kernel/locking/spinlock.c:202
Code: f6 ff 0f 1f 00 53 48 89 fb 48 83 c7 18 48 8b 74 24 08 e8 3e 1d aa f7 48 89 df e8 06 f2 aa f7 e8 c1 8c cb f7 fb bf 01 00 00 00 <e8> 46 15 9f f7 65 8b 05 27 13 50 76 85 c0 74 02 5b c3 e8 d4 1d 4e
RSP: 0018:ffffc90005377bf8 EFLAGS: 00000282
RAX: a6d3ffe641777600 RBX: ffffffff9612d6d8 RCX: a6d3ffe641777600
RDX: dffffc0000000000 RSI: ffffffff8a0b1820 RDI: 0000000000000001
RBP: 00000000000000c6 R08: dffffc0000000000 R09: fffffbfff1ff781b
R10: fffffbfff1ff781b R11: 1ffffffff1ff781a R12: ffffffff9612d7b8
R13: dffffc0000000000 R14: ffffffff9612d5a0 R15: ffffffff9612d6d8
 spin_unlock_irq include/linux/spinlock.h:413 [inline]
 srcu_reschedule+0x9b/0xf0 kernel/rcu/srcutree.c:1310
 process_one_work+0x863/0x1000 kernel/workqueue.c:2310
 worker_thread+0xaa8/0x12a0 kernel/workqueue.c:2457
 kthread+0x436/0x520 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>
rcu: rcu_preempt kthread starved for 10364 jiffies! g7673 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:R  running task     stack:27200 pid:   15 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5049 [inline]
 __schedule+0x11bb/0x4390 kernel/sched/core.c:6395
 schedule+0x11b/0x1e0 kernel/sched/core.c:6478
 schedule_timeout+0x15c/0x280 kernel/time/timer.c:1914
 rcu_gp_fqs_loop+0x29e/0x11b0 kernel/rcu/tree.c:1972
 rcu_gp_kthread+0x98/0x350 kernel/rcu/tree.c:2145
 kthread+0x436/0x520 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
NMI backtrace for cpu 1
CPU: 1 PID: 1107 Comm: kworker/1:2 Not tainted syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 08/18/2025
Workqueue: events fqdir_free_fn
Call Trace:
 <IRQ>
 dump_stack_lvl+0x168/0x230 lib/dump_stack.c:106
 nmi_cpu_backtrace+0x397/0x3d0 lib/nmi_backtrace.c:111
 nmi_trigger_cpumask_backtrace+0x163/0x280 lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:166 [inline]
 rcu_check_gp_kthread_starvation+0x1cd/0x250 kernel/rcu/tree_stall.h:487
 print_other_cpu_stall+0x10c8/0x1220 kernel/rcu/tree_stall.h:592
 check_cpu_stall kernel/rcu/tree_stall.h:745 [inline]
 rcu_pending kernel/rcu/tree.c:3936 [inline]
 rcu_sched_clock_irq+0x831/0x1110 kernel/rcu/tree.c:2619
 update_process_times+0x193/0x200 kernel/time/timer.c:1818
 tick_sched_handle kernel/time/tick-sched.c:254 [inline]
 tick_sched_timer+0x37d/0x560 kernel/time/tick-sched.c:1473
 __run_hrtimer kernel/time/hrtimer.c:1690 [inline]
 __hrtimer_run_queues+0x4fe/0xc40 kernel/time/hrtimer.c:1754
 hrtimer_interrupt+0x3bb/0x8d0 kernel/time/hrtimer.c:1816
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1097 [inline]
 __sysvec_apic_timer_interrupt+0x137/0x4a0 arch/x86/kernel/apic/apic.c:1114
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1108 [inline]
 sysvec_apic_timer_interrupt+0x9b/0xc0 arch/x86/kernel/apic/apic.c:1108
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x16/0x20 arch/x86/include/asm/idtentry.h:676
RIP: 0010:csd_lock_wait kernel/smp.c:440 [inline]
RIP: 0010:smp_call_function_single+0x212/0x490 kernel/smp.c:758
Code: 48 44 89 f6 83 e6 01 31 ff e8 ca 5e 0b 00 41 83 e6 01 49 bc 00 00 00 00 00 fc ff df 75 0a e8 55 5b 0b 00 e9 a3 00 00 00 f3 90 <f7> 44 24 48 01 00 00 00 0f 84 8e 00 00 00 e8 3b 5b 0b 00 eb e9 e8
RSP: 0018:ffffc90004907a40 EFLAGS: 00000293
RAX: ffffffff816c6fe5 RBX: 0000000000000000 RCX: ffff888020cfbb80
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000000
RBP: ffffc90004907b30 R08: dffffc0000000000 R09: ffffed1017207681
R10: ffffed1017207681 R11: 1ffff11017207680 R12: dffffc0000000000
R13: 0000000000000001 R14: 0000000000000001 R15: 1ffff92000920f4c
 rcu_barrier+0x25d/0x4b0 kernel/rcu/tree.c:4082
 fqdir_free_fn+0x2f/0x130 net/ipv4/inet_fragment.c:166
 process_one_work+0x863/0x1000 kernel/workqueue.c:2310
 worker_thread+0xaa8/0x12a0 kernel/workqueue.c:2457
 kthread+0x436/0x520 kernel/kthread.c:334
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:287
 </TASK>

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2025/09/13 13:35 linux-5.15.y 43bb85222e53 e2beed91 .config console log report syz / log [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan INFO: rcu detected stall in process_one_work
2026/01/07 13:16 linux-5.15.y 68efe5a6c16a d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in process_one_work
2026/01/01 11:57 linux-5.15.y 68efe5a6c16a d6526ea3 .config console log report info [disk image] [vmlinux] [kernel image] ci2-linux-5-15-kasan-perf INFO: rcu detected stall in process_one_work
* Struck through repros no longer work on HEAD.