KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (12)

Status: fixed on 2023/06/08 14:41
Subsystems: net
[Documentation on labels]
Fix commit: 4b397c06cb98 net: tunnels: annotate lockless accesses to dev->needed_headroom
First crash: 1267d, last: 1252d
▶ ▼ Similar bugs (11)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (6) net 6 32 1697d 1862d 0/29 auto-closed as invalid on 2022/01/21 16:19
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit net 6 22 2338d 2470d 0/29 auto-closed as invalid on 2020/05/25 22:33
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (11) net 6 1 1337d 1337d 0/29 auto-obsoleted due to no activity on 2023/01/17 03:08
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (9) net 6 1 1457d 1457d 0/29 auto-closed as invalid on 2022/09/18 04:26
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (10) net 6 4 1375d 1417d 0/29 auto-obsoleted due to no activity on 2022/12/10 01:58
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (2) net 6 11 2134d 2238d 0/29 auto-closed as invalid on 2020/11/10 03:04
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (5) net 6 5 1912d 1876d 0/29 auto-closed as invalid on 2021/07/05 08:43
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (8) net 6 1 1500d 1500d 0/29 auto-closed as invalid on 2022/08/06 22:00
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (4) net 6 2 2001d 2024d 0/29 auto-closed as invalid on 2021/03/23 07:29
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (7) net 6 10 1539d 1649d 0/29 auto-closed as invalid on 2022/06/29 00:38
upstream KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit (3) net 6 5 2060d 2068d 0/29 auto-closed as invalid on 2021/01/23 09:54

Sample crash report:
Dead loop on virtual device gre600, fix it urgently!
==================================================================
BUG: KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit

read to 0xffff88815b9da0ec of 2 bytes by task 888 on cpu 1:
 ip_tunnel_xmit+0x1270/0x1730 net/ipv4/ip_tunnel.c:803
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 neigh_direct_output+0x17/0x20 net/core/neighbour.c:1623
 neigh_output include/net/neighbour.h:546 [inline]
 ip_finish_output2+0x740/0x840 net/ipv4/ip_output.c:228
 ip_finish_output+0xf4/0x240 net/ipv4/ip_output.c:316
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0xe5/0x1b0 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out+0x64/0x80 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x34a/0x4b0 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1451/0x1730 net/ipv4/ip_tunnel.c:813
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 neigh_direct_output+0x17/0x20 net/core/neighbour.c:1623
 neigh_output include/net/neighbour.h:546 [inline]
 ip_finish_output2+0x740/0x840 net/ipv4/ip_output.c:228
 ip_finish_output+0xf4/0x240 net/ipv4/ip_output.c:316
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0xe5/0x1b0 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out+0x64/0x80 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x34a/0x4b0 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1451/0x1730 net/ipv4/ip_tunnel.c:813
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 neigh_direct_output+0x17/0x20 net/core/neighbour.c:1623
 neigh_output include/net/neighbour.h:546 [inline]
 ip_finish_output2+0x740/0x840 net/ipv4/ip_output.c:228
 ip_finish_output+0xf4/0x240 net/ipv4/ip_output.c:316
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0xe5/0x1b0 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out+0x64/0x80 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x34a/0x4b0 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1451/0x1730 net/ipv4/ip_tunnel.c:813
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 neigh_direct_output+0x17/0x20 net/core/neighbour.c:1623
 neigh_output include/net/neighbour.h:546 [inline]
 ip_finish_output2+0x740/0x840 net/ipv4/ip_output.c:228
 ip_finish_output+0xf4/0x240 net/ipv4/ip_output.c:316
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0xe5/0x1b0 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out+0x64/0x80 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x34a/0x4b0 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1451/0x1730 net/ipv4/ip_tunnel.c:813
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 neigh_direct_output+0x17/0x20 net/core/neighbour.c:1623
 neigh_output include/net/neighbour.h:546 [inline]
 ip_finish_output2+0x740/0x840 net/ipv4/ip_output.c:228
 ip_finish_output+0xf4/0x240 net/ipv4/ip_output.c:316
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0xe5/0x1b0 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out+0x64/0x80 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x34a/0x4b0 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1451/0x1730 net/ipv4/ip_tunnel.c:813
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 neigh_direct_output+0x17/0x20 net/core/neighbour.c:1623
 neigh_output include/net/neighbour.h:546 [inline]
 ip_finish_output2+0x740/0x840 net/ipv4/ip_output.c:228
 ip_finish_output+0xf4/0x240 net/ipv4/ip_output.c:316
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip_output+0xe5/0x1b0 net/ipv4/ip_output.c:430
 dst_output include/net/dst.h:444 [inline]
 ip_local_out+0x64/0x80 net/ipv4/ip_output.c:126
 iptunnel_xmit+0x34a/0x4b0 net/ipv4/ip_tunnel_core.c:82
 ip_tunnel_xmit+0x1451/0x1730 net/ipv4/ip_tunnel.c:813
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246

write to 0xffff88815b9da0ec of 2 bytes by task 2379 on cpu 0:
 ip_tunnel_xmit+0x1294/0x1730 net/ipv4/ip_tunnel.c:804
 __gre_xmit net/ipv4/ip_gre.c:469 [inline]
 ipgre_xmit+0x516/0x570 net/ipv4/ip_gre.c:661
 __netdev_start_xmit include/linux/netdevice.h:4881 [inline]
 netdev_start_xmit include/linux/netdevice.h:4895 [inline]
 xmit_one net/core/dev.c:3580 [inline]
 dev_hard_start_xmit+0x127/0x400 net/core/dev.c:3596
 __dev_queue_xmit+0x1007/0x1eb0 net/core/dev.c:4246
 dev_queue_xmit include/linux/netdevice.h:3051 [inline]
 neigh_direct_output+0x17/0x20 net/core/neighbour.c:1623
 neigh_output include/net/neighbour.h:546 [inline]
 ip6_finish_output2+0x9bc/0xc50 net/ipv6/ip6_output.c:134
 __ip6_finish_output net/ipv6/ip6_output.c:195 [inline]
 ip6_finish_output+0x39a/0x4e0 net/ipv6/ip6_output.c:206
 NF_HOOK_COND include/linux/netfilter.h:291 [inline]
 ip6_output+0xeb/0x220 net/ipv6/ip6_output.c:227
 dst_output include/net/dst.h:444 [inline]
 NF_HOOK include/linux/netfilter.h:302 [inline]
 mld_sendpack+0x438/0x6a0 net/ipv6/mcast.c:1820
 mld_send_cr net/ipv6/mcast.c:2121 [inline]
 mld_ifc_work+0x519/0x7b0 net/ipv6/mcast.c:2653
 process_one_work+0x3e6/0x750 kernel/workqueue.c:2390
 worker_thread+0x5f2/0xa10 kernel/workqueue.c:2537
 kthread+0x1ac/0x1e0 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:308

value changed: 0x0dd4 -> 0x0e14

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 PID: 2379 Comm: kworker/0:0 Not tainted 6.3.0-rc1-syzkaller-00002-g8ca09d5fa354-dirty #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/02/2023
Workqueue: mld mld_ifc_work
==================================================================
net_ratelimit: 64 callbacks suppressed
Dead loop on virtual device gre607, fix it urgently!
Dead loop on virtual device gre606, fix it urgently!
Dead loop on virtual device gre607, fix it urgently!
Dead loop on virtual device gre606, fix it urgently!
Dead loop on virtual device gre607, fix it urgently!
Dead loop on virtual device gre606, fix it urgently!
Dead loop on virtual device gre606, fix it urgently!
Dead loop on virtual device gre2, fix it urgently!
Dead loop on virtual device gre613, fix it urgently!

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2023/03/07 22:14 upstream 8ca09d5fa354 d7ea8bc4 .config console log report info ci2-upstream-kcsan-gce KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit
2023/02/22 01:30 upstream 9e58df973d22 42a4d508 .config console log report info ci2-upstream-kcsan-gce KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit
2023/02/20 02:48 upstream c9c3395d5e3d bcdf85f8 .config console log report info ci2-upstream-kcsan-gce KCSAN: data-race in ip_tunnel_xmit / ip_tunnel_xmit
* Struck through repros no longer work on HEAD.