INFO: rcu detected stall in __perf_sw_event

Status: upstream: reported C repro on 2020/07/27 06:04
Reported-by: syzbot+22fa47646f6ba9350797@syzkaller.appspotmail.com
First crash: 2207d, last: 1262d
▶ ▼ Similar bugs (5)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
linux-4.19 INFO: rcu detected stall in __perf_sw_event 1 2 2525d 2592d 0/1 auto-closed as invalid on 2020/01/11 01:03
linux-4.19 INFO: rcu detected stall in __perf_sw_event (2) 1 syz, C error 133 1259d 2396d 0/1 upstream: reported C repro on 2020/01/20 09:59
android-414 INFO: rcu detected stall in __perf_sw_event 1 13 2783d 2678d 0/1 auto-closed as invalid on 2019/06/27 14:55
android-49 INFO: rcu detected stall in __perf_sw_event 1 syz, C 23 2681d 2678d 0/3 public: reported C repro on 2019/04/13 00:00
upstream INFO: rcu detected stall in __perf_sw_event perf 1 syz done error 99 1924d 2743d 0/29 auto-obsoleted due to no activity on 2023/01/16 15:10
▶ ▼ Fix bisection attempts (25)
Created Duration User Patch Repo Result
2023/02/27 02:58 34m bisect fix linux-4.14.y OK (0) job log log
2023/01/25 15:30 32m bisect fix linux-4.14.y OK (0) job log log
2022/11/14 11:39 30m bisect fix linux-4.14.y OK (0) job log log
2022/10/15 02:31 29m bisect fix linux-4.14.y OK (0) job log log
2022/09/14 07:37 28m bisect fix linux-4.14.y OK (0) job log log
2022/08/15 07:03 34m bisect fix linux-4.14.y OK (0) job log log
2022/07/16 06:33 29m bisect fix linux-4.14.y OK (0) job log log
2022/06/16 05:59 33m bisect fix linux-4.14.y OK (0) job log log
2022/05/17 05:16 43m bisect fix linux-4.14.y OK (0) job log log
2022/04/17 04:20 38m bisect fix linux-4.14.y OK (0) job log log
2022/03/17 08:05 33m bisect fix linux-4.14.y OK (0) job log log
2022/02/15 07:34 30m bisect fix linux-4.14.y OK (0) job log log
2022/01/16 07:00 34m bisect fix linux-4.14.y OK (0) job log log
2021/12/17 06:26 33m bisect fix linux-4.14.y OK (0) job log log
2021/11/17 05:48 38m bisect fix linux-4.14.y OK (0) job log log
2021/10/18 05:08 39m bisect fix linux-4.14.y OK (0) job log log
2021/09/18 04:36 31m bisect fix linux-4.14.y OK (0) job log log
2021/08/19 03:57 33m bisect fix linux-4.14.y OK (0) job log log
2021/07/20 03:16 40m bisect fix linux-4.14.y OK (0) job log log
2021/06/20 02:38 38m bisect fix linux-4.14.y OK (0) job log log
2021/05/21 01:49 48m bisect fix linux-4.14.y OK (0) job log log
2021/04/21 00:43 37m bisect fix linux-4.14.y OK (0) job log log
2021/03/21 23:01 36m bisect fix linux-4.14.y OK (0) job log log
2021/02/19 18:09 35m bisect fix linux-4.14.y OK (0) job log log
2021/02/17 12:51 18m bisect fix linux-4.14.y error job log

Sample crash report:
INFO: rcu_preempt self-detected stall on CPU
	0-...: (10499 ticks this GP) idle=176/140000000000001/0 softirq=15476/15477 fqs=5234 
	 (t=10501 jiffies g=4384 c=4383 q=28604)
NMI backtrace for cpu 0
CPU: 0 PID: 9987 Comm: syz-executor212 Not tainted 4.14.198-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
 <IRQ>
 __dump_stack lib/dump_stack.c:17 [inline]
 dump_stack+0x1b2/0x283 lib/dump_stack.c:58
 nmi_cpu_backtrace.cold+0x57/0x93 lib/nmi_backtrace.c:101
 nmi_trigger_cpumask_backtrace+0x13a/0x17f lib/nmi_backtrace.c:62
 trigger_single_cpu_backtrace include/linux/nmi.h:158 [inline]
 rcu_dump_cpu_stacks+0x15f/0x19c kernel/rcu/tree.c:1396
 print_cpu_stall kernel/rcu/tree.c:1542 [inline]
 check_cpu_stall kernel/rcu/tree.c:1610 [inline]
 __rcu_pending kernel/rcu/tree.c:3390 [inline]
 rcu_pending kernel/rcu/tree.c:3452 [inline]
 rcu_check_callbacks.cold+0x464/0xd99 kernel/rcu/tree.c:2792
 update_process_times+0x28/0xa0 kernel/time/timer.c:1591
 tick_sched_handle+0x7d/0x150 kernel/time/tick-sched.c:165
 tick_sched_timer+0x92/0x200 kernel/time/tick-sched.c:1223
 __run_hrtimer kernel/time/hrtimer.c:1223 [inline]
 __hrtimer_run_queues+0x30b/0xc80 kernel/time/hrtimer.c:1287
 hrtimer_interrupt+0x1e6/0x5e0 kernel/time/hrtimer.c:1321
 local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1075 [inline]
 smp_apic_timer_interrupt+0x117/0x5e0 arch/x86/kernel/apic/apic.c:1100
 apic_timer_interrupt+0x93/0xa0 arch/x86/entry/entry_64.S:793
 </IRQ>
RIP: 0010:check_memory_region_inline mm/kasan/kasan.c:251 [inline]
RIP: 0010:check_memory_region+0x17/0x170 mm/kasan/kasan.c:267
RSP: 0018:ffff8880a0727268 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff10
RAX: ffff7fffffffffff RBX: 0000000000000000 RCX: ffffffff8169efd9
RDX: 0000000000000000 RSI: 0000000000000010 RDI: ffff8880a0727360
RBP: ffff88808f2041b0 R08: 0000000000000000 R09: ffff88821fff700f
R10: ffff88821fff7017 R11: 0000003efa49bf0a R12: 0000000000000010
R13: ffff8880a0727360 R14: ffff8880a0727528 R15: 0000000000000010
 memcpy+0x20/0x50 mm/kasan/kasan.c:302
 memcpy include/linux/string.h:372 [inline]
 memcpy_common kernel/events/internal.h:172 [inline]
 __output_copy kernel/events/internal.h:176 [inline]
 perf_output_read_group kernel/events/core.c:5895 [inline]
 perf_output_read+0x6c9/0x1090 kernel/events/core.c:5928
 perf_output_sample+0xa28/0x16f0 kernel/events/core.c:5970
 __perf_event_output kernel/events/core.c:6283 [inline]
 perf_event_output_forward+0xf8/0x1f0 kernel/events/core.c:6296
 __perf_event_overflow+0x113/0x310 kernel/events/core.c:7541
 perf_swevent_overflow kernel/events/core.c:7617 [inline]
 perf_swevent_event+0x299/0x460 kernel/events/core.c:7645
 do_perf_sw_event kernel/events/core.c:7758 [inline]
 ___perf_sw_event+0x2a1/0x480 kernel/events/core.c:7789
 __perf_sw_event+0x4f/0x100 kernel/events/core.c:7801
 perf_sw_event include/linux/perf_event.h:1046 [inline]
 __do_page_fault+0x6c1/0xb50 arch/x86/mm/fault.c:1483
 page_fault+0x25/0x50 arch/x86/entry/entry_64.S:1123
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x20 arch/x86/lib/copy_user_64.S:181
RSP: 0018:ffff8880a0727a78 EFLAGS: 00010206
RAX: ffffed1013b0a000 RBX: 0000000000008000 RCX: 0000000000000acb
RDX: 0000000000008000 RSI: 0000000020e6d000 RDI: ffff88809d84f535
RBP: 0000000020e65acb R08: 0000000000000001 R09: ffffed1013b09fff
R10: ffff88809d84ffff R11: 0000000000000000 R12: ffff88809d848000
R13: 00007ffffffff000 R14: 0000000020e6dacb R15: ffff88809d848000
 copy_user_generic arch/x86/include/asm/uaccess_64.h:37 [inline]
 raw_copy_from_user arch/x86/include/asm/uaccess_64.h:55 [inline]
 copyin+0x99/0xc0 lib/iov_iter.c:146
 _copy_from_iter_full+0x197/0x690 lib/iov_iter.c:608
 copy_from_iter_full include/linux/uio.h:126 [inline]
 skb_do_copy_data_nocache include/net/sock.h:1889 [inline]
 skb_copy_to_page_nocache include/net/sock.h:1915 [inline]
 tcp_sendmsg_locked+0x23fc/0x2ef0 net/ipv4/tcp.c:1360
 tcp_sendmsg+0x2b/0x40 net/ipv4/tcp.c:1457
 inet_sendmsg+0x11a/0x4e0 net/ipv4/af_inet.c:762
 sock_sendmsg_nosec net/socket.c:646 [inline]
 sock_sendmsg+0xb5/0x100 net/socket.c:656
 SYSC_sendto net/socket.c:1763 [inline]
 SyS_sendto+0x1c7/0x2c0 net/socket.c:1731
 do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x46/0xbb
RIP: 0033:0x4498a9
RSP: 002b:00007f919d024da8 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 00000000006dec48 RCX: 00000000004498a9
RDX: ffffffffffffffef RSI: 0000000020d7cfcb RDI: 0000000000000004
RBP: 00000000006dec40 R08: 0000000000000000 R09: 0000000000000053
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dec4c
R13: 00007ffc0d5347ff R14: 00007f919d0259c0 R15: 0000000000000064
INFO: rcu_sched detected stalls on CPUs/tasks:
	0-...: (10497 ticks this GP) idle=176/140000000000000/0 softirq=15477/15477 fqs=5233 
	(detected by 1, t=10548 jiffies, g=2309, c=2308, q=1238)
Sending NMI from CPU 1 to CPUs 0:
NMI backtrace for cpu 0
CPU: 0 PID: 9987 Comm: syz-executor212 Not tainted 4.14.198-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
task: ffff88808974e540 task.stack: ffff8880a0720000
RIP: 0010:__pvclock_read_cycles arch/x86/include/asm/pvclock.h:87 [inline]
RIP: 0010:pvclock_clocksource_read+0x1e1/0x4f0 arch/x86/kernel/pvclock.c:87
RSP: 0018:ffff8880a0727190 EFLAGS: 00000246
RAX: 0000003f07a621c1 RBX: ffff88821fff7000 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000001 RDI: 0000000000000001
RBP: dffffc0000000000 R08: ffff88821fff7018 R09: ffff88821fff700f
R10: ffff88821fff7017 R11: ffff88821fff701b R12: ffff88821fff7003
R13: ffffed1043ffee01 R14: ffff88821fff701d R15: ffff88821fff7010
FS:  00007f919d025700(0000) GS:ffff8880aea00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000800000001 CR3: 000000008ddc0000 CR4: 00000000001406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 kvm_clock_read+0x1f/0x30 arch/x86/kernel/kvmclock.c:88
 kvm_sched_clock_read+0x5/0x10 arch/x86/kernel/kvmclock.c:100
 paravirt_sched_clock arch/x86/include/asm/paravirt.h:175 [inline]
 sched_clock+0x2a/0x40 arch/x86/kernel/tsc.c:228
 sched_clock_cpu+0x18/0x1b0 kernel/sched/clock.c:363
 local_clock include/linux/sched/clock.h:84 [inline]
 cpu_clock_event_update kernel/events/core.c:8828 [inline]
 cpu_clock_event_read+0x15/0x30 kernel/events/core.c:8861
 perf_output_read_group kernel/events/core.c:5889 [inline]
 perf_output_read+0xc28/0x1090 kernel/events/core.c:5928
 perf_output_sample+0xa28/0x16f0 kernel/events/core.c:5970
 __perf_event_output kernel/events/core.c:6283 [inline]
 perf_event_output_forward+0xf8/0x1f0 kernel/events/core.c:6296
 __perf_event_overflow+0x113/0x310 kernel/events/core.c:7541
 perf_swevent_overflow kernel/events/core.c:7617 [inline]
 perf_swevent_event+0x299/0x460 kernel/events/core.c:7645
 do_perf_sw_event kernel/events/core.c:7758 [inline]
 ___perf_sw_event+0x2a1/0x480 kernel/events/core.c:7789
 __perf_sw_event+0x4f/0x100 kernel/events/core.c:7801
 perf_sw_event include/linux/perf_event.h:1046 [inline]
 __do_page_fault+0x6c1/0xb50 arch/x86/mm/fault.c:1483
 page_fault+0x25/0x50 arch/x86/entry/entry_64.S:1123
RIP: 0010:copy_user_enhanced_fast_string+0xe/0x20 arch/x86/lib/copy_user_64.S:181
RSP: 0018:ffff8880a0727a78 EFLAGS: 00010206
RAX: ffffed1013b0a000 RBX: 0000000000008000 RCX: 0000000000000acb
RDX: 0000000000008000 RSI: 0000000020e6d000 RDI: ffff88809d84f535
RBP: 0000000020e65acb R08: 0000000000000001 R09: ffffed1013b09fff
R10: ffff88809d84ffff R11: 0000000000000000 R12: ffff88809d848000
R13: 00007ffffffff000 R14: 0000000020e6dacb R15: ffff88809d848000
 copy_user_generic arch/x86/include/asm/uaccess_64.h:37 [inline]
 raw_copy_from_user arch/x86/include/asm/uaccess_64.h:55 [inline]
 copyin+0x99/0xc0 lib/iov_iter.c:146
 _copy_from_iter_full+0x197/0x690 lib/iov_iter.c:608
 copy_from_iter_full include/linux/uio.h:126 [inline]
 skb_do_copy_data_nocache include/net/sock.h:1889 [inline]
 skb_copy_to_page_nocache include/net/sock.h:1915 [inline]
 tcp_sendmsg_locked+0x23fc/0x2ef0 net/ipv4/tcp.c:1360
 tcp_sendmsg+0x2b/0x40 net/ipv4/tcp.c:1457
 inet_sendmsg+0x11a/0x4e0 net/ipv4/af_inet.c:762
 sock_sendmsg_nosec net/socket.c:646 [inline]
 sock_sendmsg+0xb5/0x100 net/socket.c:656
 SYSC_sendto net/socket.c:1763 [inline]
 SyS_sendto+0x1c7/0x2c0 net/socket.c:1731
 do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
 entry_SYSCALL_64_after_hwframe+0x46/0xbb
RIP: 0033:0x4498a9
RSP: 002b:00007f919d024da8 EFLAGS: 00000246 ORIG_RAX: 000000000000002c
RAX: ffffffffffffffda RBX: 00000000006dec48 RCX: 00000000004498a9
RDX: ffffffffffffffef RSI: 0000000020d7cfcb RDI: 0000000000000004
RBP: 00000000006dec40 R08: 0000000000000000 R09: 0000000000000053
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000006dec4c
R13: 00007ffc0d5347ff R14: 00007f919d0259c0 R15: 0000000000000064
Code: 03 0f b6 14 2a 40 38 f2 40 0f 9e c6 84 d2 0f 95 c2 40 84 d6 0f 85 c1 01 00 00 4c 89 d2 83 e2 07 38 d1 40 0f 9e c6 84 c9 0f 95 c2 <40> 84 d6 0f 85 a7 01 00 00 4c 89 f2 48 8b 7b 10 4c 89 f1 48 c1

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2020/10/02 18:56 linux-4.14.y cbfa1702aaf6 062c9832 .config console log report syz C ci2-linux-4-14
2020/10/02 13:44 linux-4.14.y cbfa1702aaf6 062c9832 .config console log report info ci2-linux-4-14
2020/07/27 06:03 linux-4.14.y 69b94dd6dcd1 cb93dc6a .config console log report ci2-linux-4-14
* Struck through repros no longer work on HEAD.