INFO: rcu detected stall in cleanup_net (5)

Status: closed as invalid on 2022/02/08 10:00
Subsystems: net
[Documentation on labels]
First crash: 1910d, last: 1650d
▶ ▼ Similar bugs (16)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream INFO: rcu detected stall in cleanup_net (8) prio:normal kernfs net 1 syz, C error inconclusive 34 2d12h 210d 0/29 upstream: reported C repro on 2026/01/13 16:49
upstream INFO: rcu detected stall in cleanup_net (3) net 1 1 2522d 2521d 0/29 auto-closed as invalid on 2019/12/15 02:54
upstream INFO: rcu detected stall in cleanup_net (6) net 1 63 1069d 1312d 0/29 closed as invalid on 2023/10/27 14:39
linux-4.19 INFO: rcu detected stall in cleanup_net (2) 1 4 1871d 2003d 0/1 auto-closed as invalid on 2021/10/25 03:43
upstream INFO: rcu detected stall in cleanup_net kernel 1 5 2953d 3021d 0/29 closed as invalid on 2018/09/05 12:51
upstream INFO: rcu detected stall in cleanup_net (4) kernel 1 syz, C done 5 2166d 2162d 15/29 fixed on 2020/10/10 01:52
linux-4.19 INFO: rcu detected stall in cleanup_net 1 1 2394d 2394d 0/1 auto-closed as invalid on 2020/05/21 02:17
linux-6.1 INFO: rcu detected stall in cleanup_net origin:lts-only 1 syz inconclusive 143 171d 1126d 0/3 upstream: reported syz repro on 2023/07/12 14:45
upstream INFO: rcu detected stall in cleanup_net (2) kernel 1 1 2865d 2861d 0/29 auto-closed as invalid on 2019/04/05 22:14
upstream INFO: rcu detected stall in cleanup_net (7) kernfs rdma bridge 1 15 622d 999d 0/29 auto-obsoleted due to no activity on 2025/02/25 20:50
linux-5.15 INFO: rcu detected stall in cleanup_net origin:lts-only 1 syz, C inconclusive 193 1d02h 1127d 0/3 upstream: reported C repro on 2023/07/11 05:04
android-5-10 BUG: soft lockup in cleanup_net 1 1 460d 460d 0/2 auto-obsoleted due to no activity on 2025/08/06 14:41
linux-4.19 BUG: soft lockup in cleanup_net (2) 1 syz, C 3 1255d 1367d 0/1 upstream: reported C repro on 2022/11/13 07:04
android-5-15 BUG: soft lockup in cleanup_net origin:lts 1 syz 6 701d 828d 0/2 auto-obsoleted due to no activity on 2024/12/08 05:58
android-6-1 BUG: soft lockup in cleanup_net 1 1 571d 571d 0/2 auto-obsoleted due to no activity on 2025/04/17 10:35
linux-4.19 BUG: soft lockup in cleanup_net 1 4 1501d 1714d 0/1 auto-obsoleted due to no activity on 2022/10/30 18:15

Sample crash report:
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 	Tasks blocked on level-0 rcu_node (CPUs 0-1): P50/1:b..l
	(detected by 0, t=10502 jiffies, g=22369, q=192)
task:kworker/u4:2    state:R  running task     stack:25376 pid:   50 ppid:     2 flags:0x00004000
Workqueue: netns cleanup_net
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:4986 [inline]
 __schedule+0xab2/0x4db0 kernel/sched/core.c:6295
 preempt_schedule_irq+0x4e/0x90 kernel/sched/core.c:6710
 irqentry_exit+0x31/0x80 kernel/entry/common.c:425
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:preempt_count arch/x86/include/asm/preempt.h:27 [inline]
RIP: 0010:check_kcov_mode kernel/kcov.c:166 [inline]
RIP: 0010:__sanitizer_cov_trace_pc+0x0/0x60 kernel/kcov.c:200
Code: 48 89 ef 5d e9 61 b2 46 00 5d be 03 00 00 00 e9 06 17 66 02 66 0f 1f 44 00 00 48 8b be b0 01 00 00 e8 b4 ff ff ff 31 c0 c3 90 <65> 8b 05 29 f7 89 7e 89 c1 48 8b 34 24 81 e1 00 01 00 00 65 48 8b
RSP: 0018:ffffc900012ffb48 EFLAGS: 00000202
RAX: 0000000000000000 RBX: 00000000000e44bd RCX: 0000000000000001
RDX: 0000000000000000 RSI: ffff888015e88000 RDI: 0000000000000003
RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000001
R10: ffffffff87affc45 R11: 0000000000000000 R12: 0000000000000006
R13: ffff88807bdc9a80 R14: 0000000000000002 R15: dffffc0000000000
 get_nulls_value include/linux/list_nulls.h:56 [inline]
 inet_twsk_purge+0x4a8/0x7d0 net/ipv4/inet_timewait_sock.c:297
 ops_exit_list+0x125/0x170 net/core/net_namespace.c:173
 cleanup_net+0x4ea/0xb00 net/core/net_namespace.c:597
 process_one_work+0x9ac/0x1650 kernel/workqueue.c:2307
 worker_thread+0x657/0x1110 kernel/workqueue.c:2454
 kthread+0x2e9/0x3a0 kernel/kthread.c:377
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: rcu_preempt kthread timer wakeup didn't happen for 10499 jiffies! g22369 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
rcu: 	Possible timer handling issue on cpu=1 timer-softirq=8302
rcu: rcu_preempt kthread starved for 10500 jiffies! g22369 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
rcu: 	Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
rcu: RCU grace-period kthread stack dump:
task:rcu_preempt     state:I stack:28736 pid:   14 ppid:     2 flags:0x00004000
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:4986 [inline]
 __schedule+0xab2/0x4db0 kernel/sched/core.c:6295
 schedule+0xd2/0x260 kernel/sched/core.c:6368
 schedule_timeout+0x14a/0x2a0 kernel/time/timer.c:1881
 rcu_gp_fqs_loop+0x186/0x810 kernel/rcu/tree.c:1963
 rcu_gp_kthread+0x1de/0x320 kernel/rcu/tree.c:2136
 kthread+0x2e9/0x3a0 kernel/kthread.c:377
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
 </TASK>
rcu: Stack dump where RCU GP kthread last ran:
Sending NMI from CPU 0 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 7169 Comm: syz-executor.3 Not tainted 5.17.0-rc2-syzkaller-00071-g1f2cfdd349b7 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:native_irq_disable arch/x86/include/asm/irqflags.h:40 [inline]
RIP: 0010:arch_local_irq_disable arch/x86/include/asm/irqflags.h:75 [inline]
RIP: 0010:arch_local_irq_save arch/x86/include/asm/irqflags.h:107 [inline]
RIP: 0010:lock_is_held_type+0x54/0x140 kernel/locking/lockdep.c:5678
Code: c0 0f 85 ca 00 00 00 65 4c 8b 24 25 00 70 02 00 41 8b 94 24 5c 0a 00 00 85 d2 0f 85 b1 00 00 00 48 89 fd 41 89 f6 9c 8f 04 24 <fa> 48 c7 c7 60 61 ac 89 31 db e8 fd 0d 00 00 41 8b 84 24 58 0a 00
RSP: 0018:ffffc90000dc0e00 EFLAGS: 00000046
RAX: 0000000000000000 RBX: ffff88801e481340 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 00000000ffffffff RDI: ffff8880b9d2a618
RBP: ffff8880b9d2a618 R08: 0000000000000000 R09: 0000000000000000
R10: ffffffff8167d08c R11: 0000000000000000 R12: ffff8880214e0000
R13: 00000000ffffffff R14: 00000000ffffffff R15: 0000000000000001
FS:  00007f088267f700(0000) GS:ffff8880b9d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000001b2dc2b000 CR3: 0000000019324000 CR4: 0000000000350ee0
Call Trace:
 <IRQ>
 lock_is_held include/linux/lockdep.h:283 [inline]
 __run_hrtimer kernel/time/hrtimer.c:1651 [inline]
 __hrtimer_run_queues+0x95a/0xe50 kernel/time/hrtimer.c:1749
 hrtimer_interrupt+0x31c/0x790 kernel/time/hrtimer.c:1811
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1086 [inline]
 __sysvec_apic_timer_interrupt+0x146/0x530 arch/x86/kernel/apic/apic.c:1103
 sysvec_apic_timer_interrupt+0x8e/0xc0 arch/x86/kernel/apic/apic.c:1097
 </IRQ>
 <TASK>
 asm_sysvec_apic_timer_interrupt+0x12/0x20 arch/x86/include/asm/idtentry.h:638
RIP: 0010:__raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:152 [inline]
RIP: 0010:_raw_spin_unlock_irqrestore+0x38/0x70 kernel/locking/spinlock.c:194
Code: 74 24 10 e8 3a 41 10 f8 48 89 ef e8 f2 b6 10 f8 81 e3 00 02 00 00 75 25 9c 58 f6 c4 02 75 2d 48 85 db 74 01 fb bf 01 00 00 00 <e8> 23 6a 03 f8 65 8b 05 9c 5d b5 76 85 c0 74 0a 5b 5d c3 e8 70 ab
RSP: 0018:ffffc9000b1a7d98 EFLAGS: 00000206
RAX: 0000000000000002 RBX: 0000000000000200 RCX: 1ffffffff1b27221
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001
RBP: ffffffff8bba3700 R08: 0000000000000001 R09: 0000000000000001
R10: ffffffff817ebdd8 R11: 0000000000000000 R12: 1ffff92001634fbc
R13: 0000000000000000 R14: ffffffff9e03a5ee R15: 0000000015393909
 do_settimeofday64 kernel/time/timekeeping.c:1323 [inline]
 do_settimeofday64+0x3af/0x5b0 kernel/time/timekeeping.c:1293
 do_sys_settimeofday64 kernel/time/time.c:195 [inline]
 do_sys_settimeofday64+0x1de/0x260 kernel/time/time.c:169
 __do_sys_clock_settime kernel/time/posix-timers.c:1079 [inline]
 __se_sys_clock_settime kernel/time/posix-timers.c:1067 [inline]
 __x64_sys_clock_settime+0x1a1/0x280 kernel/time/posix-timers.c:1067
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7f0883d0a059
Code: ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007f088267f168 EFLAGS: 00000246 ORIG_RAX: 00000000000000e3
RAX: ffffffffffffffda RBX: 00007f0883e1cf60 RCX: 00007f0883d0a059
RDX: 0000000000000000 RSI: 00000000200000c0 RDI: 0000000000000000
RBP: 00007f0883d6408d R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 00007ffedb4cfcbf R14: 00007f088267f300 R15: 0000000000022000
 </TASK>
----------------
Code disassembly (best guess):
   0:	48 89 ef             	mov    %rbp,%rdi
   3:	5d                   	pop    %rbp
   4:	e9 61 b2 46 00       	jmpq   0x46b26a
   9:	5d                   	pop    %rbp
   a:	be 03 00 00 00       	mov    $0x3,%esi
   f:	e9 06 17 66 02       	jmpq   0x266171a
  14:	66 0f 1f 44 00 00    	nopw   0x0(%rax,%rax,1)
  1a:	48 8b be b0 01 00 00 	mov    0x1b0(%rsi),%rdi
  21:	e8 b4 ff ff ff       	callq  0xffffffda
  26:	31 c0                	xor    %eax,%eax
  28:	c3                   	retq
  29:	90                   	nop
* 2a:	65 8b 05 29 f7 89 7e 	mov    %gs:0x7e89f729(%rip),%eax        # 0x7e89f75a <-- trapping instruction
  31:	89 c1                	mov    %eax,%ecx
  33:	48 8b 34 24          	mov    (%rsp),%rsi
  37:	81 e1 00 01 00 00    	and    $0x100,%ecx
  3d:	65                   	gs
  3e:	48                   	rex.W
  3f:	8b                   	.byte 0x8b

Crashes (24):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2022/02/03 22:47 upstream 1f2cfdd349b7 30646bfe .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in cleanup_net
2021/11/12 06:21 upstream 5833291ab6de 75b04091 .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in cleanup_net
2021/10/27 19:31 upstream 1fc596a56b33 373bf66b .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in cleanup_net
2021/10/12 06:41 upstream fa5878760579 838e7e2c .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in cleanup_net
2021/10/11 13:18 upstream 64570fbc14f8 838e7e2c .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in cleanup_net
2021/10/03 16:16 upstream 7b66f4393ad4 db0f5787 .config console log report info ci-upstream-kasan-gce-root INFO: rcu detected stall in cleanup_net
2021/08/09 00:42 upstream 36a21d51725a 6972b106 .config console log report info ci-upstream-kasan-gce INFO: rcu detected stall in cleanup_net
2021/08/05 23:03 upstream 902e7f373fff d2d6e680 .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in cleanup_net
2021/07/19 04:42 upstream 2734d6c1b1a0 f115ae98 .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in cleanup_net
2021/07/11 20:16 upstream 98f7fdced2e0 8f5a7b8c .config console log report info ci-upstream-kasan-gce-smack-root INFO: rcu detected stall in cleanup_net
2021/05/20 01:35 upstream 293837b9ac8d a343ba6b .config console log report info ci-upstream-kasan-gce-selinux-root INFO: rcu detected stall in cleanup_net
2021/12/07 20:00 net-old 2a62df369271 0230ba3e .config console log report info ci-upstream-net-this-kasan-gce INFO: rcu detected stall in cleanup_net
2021/11/21 02:14 net-old 262ae1f9de4e 4eb20a4e .config console log report info ci-upstream-net-this-kasan-gce INFO: rcu detected stall in cleanup_net
2021/11/17 05:51 net-old 848e5d66fa31 cafff8b6 .config console log report info ci-upstream-net-this-kasan-gce INFO: rcu detected stall in cleanup_net
2021/11/09 11:23 net-old c45231a7668d 8ab17e57 .config console log report info ci-upstream-net-this-kasan-gce INFO: rcu detected stall in cleanup_net
2021/10/22 11:47 net-old 397430b50a36 55f90bc6 .config console log report info ci-upstream-net-this-kasan-gce INFO: rcu detected stall in cleanup_net
2021/08/09 21:38 net-old 3125f26c5148 6972b106 .config console log report info ci-upstream-net-this-kasan-gce INFO: rcu detected stall in cleanup_net
2021/07/31 21:03 net-old 8d67041228ac 6c236867 .config console log report info ci-upstream-net-this-kasan-gce INFO: rcu detected stall in cleanup_net
2022/01/15 06:31 net-next-old fe8152b38d3a 723cfaf0 .config console log report info ci-upstream-net-kasan-gce INFO: rcu detected stall in cleanup_net
2021/11/16 15:03 net-next-old 3ad4b7c81a99 600426bd .config console log report info ci-upstream-net-kasan-gce INFO: rcu detected stall in cleanup_net
2021/11/12 20:57 net-next-old 5833291ab6de 83f5c9b5 .config console log report info ci-upstream-net-kasan-gce INFO: rcu detected stall in cleanup_net
2021/10/21 17:09 net-next-old dfcb63ce1de6 c5cb7da8 .config console log report info ci-upstream-net-kasan-gce INFO: rcu detected stall in cleanup_net
2021/10/01 11:32 linux-next a25006a77348 1d849ab4 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in cleanup_net
2021/08/09 17:36 linux-next da454ebf578f 6972b106 .config console log report info ci-upstream-linux-next-kasan-gce-root INFO: rcu detected stall in cleanup_net
* Struck through repros no longer work on HEAD.