KCSAN: data-race in queue_delayed_work_on / queue_delayed_work_on

Status: moderation: reported on 2026/11/22 00:48
Subsystems: net
Labels: prio:high race:benign
[Documentation on labels]
Reported-by: syzbot+4d949380380b54ef4564@syzkaller.appspotmail.com
First crash: 4d20h, last: 3h24m
✨ AI cause bisection: introduced by (AI job) :
commit 86898fa6b8cd942505860556f3a0bf52eae57fe8
Author: Tejun Heo <tj@kernel.org>
Date: Mon Mar 25 17:21:03 2024 +0000

  workqueue: Implement disable/enable for (delayed) work items

✨ AI-generated root cause summary:

When queueing a work item via queue_delayed_work_on() or queue_work_on(), the caller atomically sets WORK_STRUCT_PENDING_BIT using test_and_set_bit(..., work_data_bits(work)). If the bit was previously clear, the winner calls clear_pending_if_disabled(), which performs an unmarked plain load (*work_data_bits(work)) from work->data instead of using an atomic accessor (e.g. atomic_long_read). If another thread concurrently calls queue_delayed_work_on() on the same work item, its test_and_set_bit() performs an atomic RMW on work->data concurrently with the unmarked load, triggering a KCSAN data-race report.

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 666539eecdf4:

Technical Analysis of Root Cause

1. Involved Files, Functions, and Data Structures


2. Detailed Mechanism and Trigger Sequence

  1. Workqueue Disabling Feature (clear_pending_if_disabled): Commit 86898fa6b8cd ("workqueue: Implement disable/enable for (delayed) work items") introduced disable/enable semantics for work items by encoding a 16-bit disable depth within work->data while the work item is off-queue. To enforce this when queueing a work item, queue_work_on() and queue_delayed_work_on() perform:

    if (!test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work)) &&
        !clear_pending_if_disabled(work)) {
        __queue_delayed_work(cpu, wq, dwork, delay);
        ret = true;
    }
    

    Inside clear_pending_if_disabled():

    static bool clear_pending_if_disabled(struct work_struct *work)
    {
        unsigned long data = *work_data_bits(work); // Line 2562: Unmarked plain read
        ...
    

    Unlike other places in kernel/workqueue.c that inspect work->data (e.g., get_work_pwq() and get_work_pool(), which use atomic_long_read(&work->data)), line 2562 dereferences work_data_bits(work) directly with a plain, unmarked C read (*work_data_bits(work)).

  2. Triggering the Concurrent Access via Bonding: Concurrent invocations of queue_work() or queue_delayed_work() on the same work item from different contexts are legal and supported by workqueue semantics, designed to be synchronized via test_and_set_bit(WORK_STRUCT_PENDING_BIT, ...).

    • In bond_enslave(), new_slave->notify_work is queued on CPU 0 while holding the rtnl_lock (e.g., via bond_set_slave_link_state() or bond_set_slave_inactive_flags()).
    • A worker thread (Task 13 on CPU 1) picks up notify_work. Before calling the worker callback, workqueue infrastructure clears the WORK_STRUCT_PENDING_BIT. The callback bond_netdev_notify_work() executes and attempts rtnl_trylock(), which fails because CPU 0 is still executing bond_enslave() holding rtnl_lock.
    • Failing rtnl_trylock(), Task 13 calls queue_delayed_work(slave->bond->wq, &slave->notify_work, 1) at line 1818. Task 13 successfully transitions WORK_STRUCT_PENDING_BIT from 0 -> 1 in test_and_set_bit() and proceeds to execute clear_pending_if_disabled(work), performing the unmarked read *work_data_bits(work).
    • Concurrently, CPU 0 reaches the end of bond_enslave() at line 2347 and calls bond_queue_slave_event(new_slave), which executes queue_delayed_work(slave->bond->wq, &slave->notify_work, 0).
    • In queue_delayed_work_on() at line 2765, CPU 0 invokes test_and_set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(work)), executing an atomic read-modify-write (LOCK BTS) on the exact same 8-byte word (work->data).
  3. Violation and Sanitizer Detection: KCSAN detects that clear_pending_if_disabled() performs an uninstrumented, non-atomic plain read from work->data (*work_data_bits(work)) concurrently with another thread executing an atomic read-modify-write (test_and_set_bit()) to the same word. Using atomic_long_read(&work->data) or READ_ONCE(*work_data_bits(work)) in clear_pending_if_disabled() marks the access and avoids the data race report.

▶ ▼ ✨ AI Jobs (3)
ID Workflow Result Correct Bug Created Started Finished Revision
ddfec862 assessment-kcsan Benign: ✅ ❓ KCSAN: data-race in queue_delayed_work_on / queue_delayed_work_on 2026/11/18 00:21 2026/11/18 00:21 2026/11/18 00:39 ca622473
c98f45a8 bisect-cause ❓ KCSAN: data-race in queue_delayed_work_on / queue_delayed_work_on 2026/11/18 00:14 2026/11/18 00:14 2026/11/18 00:30 ca622473
c08e15f1 assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ✅ PeripheralTrigger: ❌ RemoteTrigger: ✅ Unprivileged: ✅ UserNamespace: ✅ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ KCSAN: data-race in queue_delayed_work_on / queue_delayed_work_on 2026/11/18 00:11 2026/11/18 00:11 2026/11/18 00:37 ca622473

Sample crash report:
veth0_vlan: entered promiscuous mode
veth1_vlan: entered promiscuous mode
==================================================================
BUG: KCSAN: data-race in queue_delayed_work_on / queue_delayed_work_on

read-write (marked) to 0xffff88810b93d418 of 8 bytes by task 3447 on cpu 1:
 instrument_atomic_read_write include/linux/instrumented.h:113 [inline]
 test_and_set_bit include/asm-generic/bitops/instrumented-atomic.h:71 [inline]
 queue_delayed_work_on+0x45/0x160 kernel/workqueue.c:2765
 queue_delayed_work include/linux/workqueue.h:715 [inline]
 check_lifetime+0x4b9/0x4d0 net/ipv4/devinet.c:804
 process_one_work kernel/workqueue.c:3495 [inline]
 process_scheduled_works+0x582/0xab0 kernel/workqueue.c:3578
 worker_thread+0x56c/0x770 kernel/workqueue.c:3659
 kthread+0x248/0x2a0 kernel/kthread.c:436
 ret_from_fork+0x146/0x330 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

read to 0xffff88810b93d418 of 8 bytes by task 3328 on cpu 0:
 clear_pending_if_disabled kernel/workqueue.c:2562 [inline]
 queue_delayed_work_on+0x7e/0x160 kernel/workqueue.c:2766
 queue_delayed_work include/linux/workqueue.h:715 [inline]
 __inet_insert_ifa+0x4cd/0x5e0 net/ipv4/devinet.c:550
 inet_rtm_newaddr+0x993/0xcc0 net/ipv4/devinet.c:991
 rtnetlink_rcv_msg+0x64b/0x720 net/core/rtnetlink.c:7169
 netlink_rcv_skb+0x11e/0x220 net/netlink/af_netlink.c:2585
 rtnetlink_rcv+0x1c/0x30 net/core/rtnetlink.c:7196
 netlink_unicast_kernel net/netlink/af_netlink.c:1336 [inline]
 netlink_unicast+0x6fd/0x7e0 net/netlink/af_netlink.c:1362
 netlink_sendmsg+0x605/0x760 net/netlink/af_netlink.c:1921
 sock_sendmsg_nosec net/socket.c:800 [inline]
 __sock_sendmsg net/socket.c:818 [inline]
 __sys_sendto+0x429/0x480 net/socket.c:2283
 __do_sys_sendto net/socket.c:2290 [inline]
 __se_sys_sendto net/socket.c:2286 [inline]
 __x64_sys_sendto+0x76/0x90 net/socket.c:2286
 x64_sys_call+0x200c/0x2550 arch/x86/include/generated/asm/syscalls_64.h:45
 do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
 do_syscall_64+0x11e/0x3a0 arch/x86/entry/syscall_64.c:89
 entry_SYSCALL_64_after_hwframe+0x77/0x7f

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 UID: 0 PID: 3328 Comm: syz-executor Not tainted syzkaller #0 PREEMPT(lazy) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
==================================================================
veth0_macvtap: entered promiscuous mode
veth1_macvtap: entered promiscuous mode
batman_adv: batadv0: Interface activated: batadv_slave_0
batman_adv: batadv0: Interface activated: batadv_slave_1

Crashes (2):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/21 22:31 upstream c44104f3642d b2c69459 .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in queue_delayed_work_on / queue_delayed_work_on
2026/11/17 05:10 upstream 666539eecdf4 a3e64cfc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in queue_delayed_work_on / queue_delayed_work_on
* Struck through repros no longer work on HEAD.