KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (12)

Status: moderation: reported on 2026/11/17 17:15
Subsystems: virt
Labels: prio:high
[Documentation on labels]
Reported-by: syzbot+32a3bdf4bf5a41f2b603@syzkaller.appspotmail.com
First crash: 15h51m, last: 7h43m
✨ AI cause bisection: candidate commits (most relevant first) (AI job):
  f277ec42f38f virtio_ring: shadow available ring flags & index
  6c0b057cec5e virtio_ring: don't update event idx on get_buf
  e13b6da7045f virtio-net: tweak for better TX performance in NAPI mode
  
✨ AI-generated root cause summary:

A KCSAN data race occurs on vq->split.avail_flags_shadow between start_xmit() (which calls virtqueue_enable_cb_delayed()) and skb_xmit_done() (which calls virtqueue_disable_cb()). While virtqueue API documentation explicitly specifies that virtqueue_disable_cb() need not be serialized, virtio_ring implements virtqueue_disable_cb_split() and virtqueue_enable_cb_delayed_split() using plain, unsynchronized read-modify-writes on the 16-bit shadow flag avail_flags_shadow. Because start_xmit() runs under the netdev TX lock with hardirqs enabled, an incoming TX completion interrupt can preempt start_xmit() on the same CPU (or run concurrently on another CPU) and invoke skb_xmit_done(), triggering concurrent unsynchronized access to avail_flags_shadow.

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 666539eecdf4:

Root Cause Analysis

1. Functions, Structs, and Fields Involved

2. Triggering Mechanism and API Invariant Breakdown

According to virtqueue API specifications:

  • virtqueue_enable_cb_delayed() and virtqueue_enable_cb_prepare() require callers to avoid concurrent invocation with other virtqueue operations "except where noted".
  • virtqueue_disable_cb() explicitly states: "Unlike other operations, this need not be serialized", and consequently its split and packed implementations intentionally omit START_USE(vq) / END_USE(vq).
  • Network drivers such as virtio_net adhere to this contract by calling virtqueue_disable_cb() directly from the interrupt handler skb_xmit_done() without holding any locks (such as the netdev _xmit_lock).

In virtio_net:

  1. When TX NAPI is enabled, start_xmit() transmits packets and, if kicked (kick), calls virtqueue_enable_cb_delayed(sq->vq) at line 3422.
  2. start_xmit() holds the netdev transmission lock (HARD_TX_LOCK / _xmit_lock), but hard interrupts remain enabled.
  3. An in-flight or newly generated TX completion interrupt triggers vring_interrupt(), which invokes skb_xmit_done(). This can either preempt start_xmit() on the same CPU or execute concurrently on a different CPU.
  4. skb_xmit_done() invokes virtqueue_disable_cb(vq), which reads and modifies vq->split.avail_flags_shadow:
    if (!(vq->split.avail_flags_shadow & VRING_AVAIL_F_NO_INTERRUPT)) {
        vq->split.avail_flags_shadow |= VRING_AVAIL_F_NO_INTERRUPT;
        ...
    }
    
  5. Simultaneously, virtqueue_enable_cb_delayed_split() is modifying the same field without synchronization:
    if (vq->split.avail_flags_shadow & VRING_AVAIL_F_NO_INTERRUPT) {
        vq->split.avail_flags_shadow &= ~VRING_AVAIL_F_NO_INTERRUPT;
        ...
    }
    

Because virtio_ring.c performs plain C read-modify-write operations on vq->split.avail_flags_shadow without atomic access primitives or KCSAN annotations (unlike vq->event_triggered, which is wrapped in data_race(...)), KCSAN flags the conflicting concurrent read and write operations.

▶ ▼ ✨ AI Jobs (3)
ID Workflow Result Correct Bug Created Started Finished Revision
d14f9532 bisect-cause ❓ KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (12) 2026/11/17 16:54 2026/11/17 16:54 2026/11/17 17:31 ca622473
30a223f9 assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ✅ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ✅ UserNamespace: ✅ VMGuestTrigger: ❌ VMHostTrigger: ✅ ❓ KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (12) 2026/11/17 16:52 2026/11/17 16:52 2026/11/17 17:09 ca622473
9e2ff36b assessment-kcsan 🏃 KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (12) 2026/11/17 16:52 2026/11/17 16:52 ca622473
▶ ▼ Similar bugs (11)
Kernel Title Rank 🛈 Repro Cause bisect Fix bisect Count Last Reported Patched Status
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (3) virt 6 1 2283d 2283d 0/29 auto-closed as invalid on 2020/09/20 20:51
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (11) race:harmful virt 6 1 234d 231d 0/29 auto-obsoleted due to no activity on 2026/05/22 19:23
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (10) virt 6 1 361d 361d 0/29 auto-obsoleted due to no activity on 2026/01/15 20:57
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (5) virt 6 3 1627d 1676d 0/29 auto-closed as invalid on 2022/07/09 17:31
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (2) virt 6 3 2456d 2529d 0/29 auto-closed as invalid on 2020/05/05 19:05
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (8) virt 6 2 1245d 1264d 0/29 auto-obsoleted due to no activity on 2023/07/25 21:20
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (9) virt 6 1 623d 622d 28/29 fixed on 2025/06/10 16:19
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (6) virt 6 1 1468d 1468d 0/29 auto-obsoleted due to no activity on 2022/12/15 06:15
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed virt 6 2 2569d 2580d 0/29 closed as invalid on 2019/11/19 13:54
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (7) virt 6 1 1323d 1323d 0/29 auto-obsoleted due to no activity on 2023/05/09 11:44
upstream KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed (4) virt 6 1 2190d 2171d 0/29 auto-closed as invalid on 2020/12/23 03:08

Sample crash report:
==================================================================
BUG: KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed

read to 0xffff888103793788 of 2 bytes by interrupt on cpu 1:
 virtqueue_disable_cb_split drivers/virtio/virtio_ring.c:1093 [inline]
 virtqueue_disable_cb+0x4c/0x2c0 drivers/virtio/virtio_ring.c:3190
 skb_xmit_done+0x67/0x1b0 drivers/net/virtio_net.c:770
 vring_interrupt+0x2d7/0x310 drivers/virtio/virtio_ring.c:3344
 __handle_irq_event_percpu+0x92/0x4a0 kernel/irq/handle.c:209
 handle_irq_event_percpu kernel/irq/handle.c:246 [inline]
 handle_irq_event+0x64/0xf0 kernel/irq/handle.c:263
 handle_edge_irq+0x155/0x440 kernel/irq/chip.c:859
 generic_handle_irq_desc include/linux/irqdesc.h:188 [inline]
 handle_irq arch/x86/kernel/irq.c:209 [inline]
 call_irq_handler arch/x86/kernel/irq.c:-1 [inline]
 __common_interrupt+0x60/0xb0 arch/x86/kernel/irq.c:280
 common_interrupt+0x7e/0x90 arch/x86/kernel/irq.c:273
 asm_common_interrupt+0x26/0x40 arch/x86/include/asm/idtentry.h:665
 native_pause arch/x86/include/asm/vdso/processor.h:13 [inline]
 delay_tsc+0x43/0xa0 arch/x86/lib/delay.c:76
 udelay include/asm-generic/delay.h:64 [inline]
 delay_access kernel/kcsan/core.c:332 [inline]
 kcsan_setup_watchpoint+0x255/0x3f0 kernel/kcsan/core.c:622
 virtqueue_enable_cb_delayed_split drivers/virtio/virtio_ring.c:1149 [inline]
 virtqueue_enable_cb_delayed+0x222/0x6c0 drivers/virtio/virtio_ring.c:3286
 start_xmit+0x1804/0x1dc0 drivers/net/virtio_net.c:3422
 __netdev_start_xmit include/linux/netdevice.h:5457 [inline]
 netdev_start_xmit include/linux/netdevice.h:5466 [inline]
 xmit_one net/core/dev.c:4068 [inline]
 dev_hard_start_xmit+0x107/0x3d0 net/core/dev.c:4084
 sch_direct_xmit+0x1c4/0x360 net/sched/sch_generic.c:411
 __dev_xmit_skb net/core/dev.c:4419 [inline]
 __dev_queue_xmit+0xe7d/0x2020 net/core/dev.c:5058
 dev_queue_xmit include/linux/netdevice.h:3485 [inline]
 neigh_hh_output include/net/neighbour.h:538 [inline]
 neigh_output include/net/neighbour.h:552 [inline]
 ip_finish_output2+0x845/0x920 net/ipv4/ip_output.c:236
 __ip_finish_output net/ipv4/ip_output.c:-1 [inline]
 ip_finish_output+0x112/0x290 net/ipv4/ip_output.c:324
 NF_HOOK_COND include/linux/netfilter.h:314 [inline]
 ip_output+0xbd/0x1c0 net/ipv4/ip_output.c:437
 dst_output include/net/dst.h:472 [inline]
 ip_local_out net/ipv4/ip_output.c:131 [inline]
 __ip_queue_xmit+0xbf3/0xc30 net/ipv4/ip_output.c:533
 ip_queue_xmit+0x39/0x50 net/ipv4/ip_output.c:547
 __tcp_transmit_skb+0x1cad/0x2110 net/ipv4/tcp_output.c:1773
 tcp_transmit_skb net/ipv4/tcp_output.c:1790 [inline]
 tcp_write_xmit+0x13b5/0x30e0 net/ipv4/tcp_output.c:3126
 __tcp_push_pending_frames+0x6d/0x1b0 net/ipv4/tcp_output.c:3309
 tcp_push+0x34d/0x370 net/ipv4/tcp.c:776
 tcp_sendmsg_locked+0x2682/0x2df0 net/ipv4/tcp.c:1428
 tcp_sendmsg+0x2f/0x50 net/ipv4/tcp.c:1465
 inet_sendmsg+0x76/0xd0 net/ipv4/af_inet.c:866
 sock_sendmsg_nosec net/socket.c:801 [inline]
 __sock_sendmsg net/socket.c:818 [inline]
 sock_write_iter+0x2a5/0x300 net/socket.c:1269
 new_sync_write fs/read_write.c:575 [inline]
 vfs_write+0x57a/0x9b0 fs/read_write.c:667
 ksys_write+0xdc/0x1a0 fs/read_write.c:719
 __do_sys_write fs/read_write.c:730 [inline]
 __se_sys_write fs/read_write.c:727 [inline]
 __x64_sys_write+0x40/0x50 fs/read_write.c:727
 x64_sys_call+0x154e/0x2550 arch/x86/include/generated/asm/syscalls_64.h:2
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x11e/0x3a0 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

write to 0xffff888103793788 of 2 bytes by task 3280 on cpu 1:
 virtqueue_enable_cb_delayed_split drivers/virtio/virtio_ring.c:1149 [inline]
 virtqueue_enable_cb_delayed+0x222/0x6c0 drivers/virtio/virtio_ring.c:3286
 start_xmit+0x1804/0x1dc0 drivers/net/virtio_net.c:3422
 __netdev_start_xmit include/linux/netdevice.h:5457 [inline]
 netdev_start_xmit include/linux/netdevice.h:5466 [inline]
 xmit_one net/core/dev.c:4068 [inline]
 dev_hard_start_xmit+0x107/0x3d0 net/core/dev.c:4084
 sch_direct_xmit+0x1c4/0x360 net/sched/sch_generic.c:411
 __dev_xmit_skb net/core/dev.c:4419 [inline]
 __dev_queue_xmit+0xe7d/0x2020 net/core/dev.c:5058
 dev_queue_xmit include/linux/netdevice.h:3485 [inline]
 neigh_hh_output include/net/neighbour.h:538 [inline]
 neigh_output include/net/neighbour.h:552 [inline]
 ip_finish_output2+0x845/0x920 net/ipv4/ip_output.c:236
 __ip_finish_output net/ipv4/ip_output.c:-1 [inline]
 ip_finish_output+0x112/0x290 net/ipv4/ip_output.c:324
 NF_HOOK_COND include/linux/netfilter.h:314 [inline]
 ip_output+0xbd/0x1c0 net/ipv4/ip_output.c:437
 dst_output include/net/dst.h:472 [inline]
 ip_local_out net/ipv4/ip_output.c:131 [inline]
 __ip_queue_xmit+0xbf3/0xc30 net/ipv4/ip_output.c:533
 ip_queue_xmit+0x39/0x50 net/ipv4/ip_output.c:547
 __tcp_transmit_skb+0x1cad/0x2110 net/ipv4/tcp_output.c:1773
 tcp_transmit_skb net/ipv4/tcp_output.c:1790 [inline]
 tcp_write_xmit+0x13b5/0x30e0 net/ipv4/tcp_output.c:3126
 __tcp_push_pending_frames+0x6d/0x1b0 net/ipv4/tcp_output.c:3309
 tcp_push+0x34d/0x370 net/ipv4/tcp.c:776
 tcp_sendmsg_locked+0x2682/0x2df0 net/ipv4/tcp.c:1428
 tcp_sendmsg+0x2f/0x50 net/ipv4/tcp.c:1465
 inet_sendmsg+0x76/0xd0 net/ipv4/af_inet.c:866
 sock_sendmsg_nosec net/socket.c:801 [inline]
 __sock_sendmsg net/socket.c:818 [inline]
 sock_write_iter+0x2a5/0x300 net/socket.c:1269
 new_sync_write fs/read_write.c:575 [inline]
 vfs_write+0x57a/0x9b0 fs/read_write.c:667
 ksys_write+0xdc/0x1a0 fs/read_write.c:719
 __do_sys_write fs/read_write.c:730 [inline]
 __se_sys_write fs/read_write.c:727 [inline]
 __x64_sys_write+0x40/0x50 fs/read_write.c:727
 x64_sys_call+0x154e/0x2550 arch/x86/include/generated/asm/syscalls_64.h:2
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x11e/0x3a0 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

Reported by Kernel Concurrency Sanitizer on:
CPU: 1 UID: 0 PID: 3280 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(lazy) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
==================================================================

Crashes (5):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/17 10:07 upstream 666539eecdf4 43af6cac .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed
2026/11/17 10:06 upstream 666539eecdf4 43af6cac .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed
2026/11/17 06:00 upstream 666539eecdf4 a3e64cfc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed
2026/11/17 01:59 upstream 666539eecdf4 a3e64cfc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed
2026/11/17 01:59 upstream 666539eecdf4 a3e64cfc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in virtqueue_disable_cb / virtqueue_enable_cb_delayed
* Struck through repros no longer work on HEAD.