KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit

Status: moderation: reported on 2026/11/19 00:04
Subsystems: trace
Labels: prio:low race:benign
[Documentation on labels]
Reported-by: syzbot+2f56c6bdef6124061cdf@syzkaller.appspotmail.com
First crash: 1d22h, last: 1d13h
✨ AI-generated root cause summary:

The KCSAN data-race report between ring_buffer_lock_reserve() and ring_buffer_unlock_commit() is caused by two compounding factors: (1) kernel/trace/Makefile accidentally placed 'KCSAN_SANITIZE := n' inside an 'ifdef CONFIG_FUNCTION_TRACER' block, which left kernel/trace/ instrumented with KCSAN in configurations where trace events are active but CONFIG_FUNCTION_TRACER is disabled; and (2) ring_buffer_lock_reserve() and ring_buffer_unlock_commit() run with preemption disabled but hardware interrupts enabled, accessing cpu_buffer->current_context in trace_recursive_lock() and trace_recursive_unlock() using plain C operations without READ_ONCE()/WRITE_ONCE() annotations. When KCSAN placed a watchpoint delay on current_context during trace_recursive_unlock(), a local APIC timer interrupt fired on the same CPU and called trace_recursive_lock(), triggering the data-race report.

▶ ▼ ✨ Full AI root cause analysis
Based on kernel commit 666539eecdf4:

Technical Root Cause Analysis

Involved Entities


Mechanism and Root Cause

The reported KCSAN data race between ring_buffer_lock_reserve() and ring_buffer_unlock_commit() is the result of two interacting defects:

  1. Unintended KCSAN Instrumentation in kernel/trace/Makefile: In commit f5d2313bd3c5 ("kcsan, trace: Make KCSAN compatible with tracing"), the intention was to disable KCSAN across the entire tracing subsystem to avoid recursion and watchdog delays during core trace path execution:

    /* kernel/trace/Makefile */
    ifdef CONFIG_FUNCTION_TRACER
    
    # Avoid recursion due to instrumentation.
    KCSAN_SANITIZE := n
    

    Because KCSAN_SANITIZE := n was placed inside ifdef CONFIG_FUNCTION_TRACER, any kernel configuration where general tracing and trace events are enabled (CONFIG_TRACING=y, CONFIG_RING_BUFFER=y) but function tracing is disabled (# CONFIG_FUNCTION_TRACER is not set) leaves all object files in kernel/trace/ (including ring_buffer.o) instrumented by KCSAN. When instrumented, KCSAN sets watchpoints and injects microsecond delays (udelay() / delay_tsc()) during tracing operations.

  2. Unmarked Interrupt-Concurrent Accesses in kernel/trace/ring_buffer.c: Both ring_buffer_lock_reserve() and ring_buffer_unlock_commit() protect ring buffer modifications by disabling preemption via preempt_disable_notrace(), but they intentionally leave hardware interrupts enabled. To detect and prevent re-entrant tracing loops from nested interrupt levels (normal task -> softirq -> hardirq -> NMI), the ring buffer maintains a per-CPU bitmask cpu_buffer->current_context.

    In trace_recursive_unlock():

    static __always_inline void
    trace_recursive_unlock(struct ring_buffer_per_cpu *cpu_buffer)
    {
        cpu_buffer->current_context &=
            cpu_buffer->current_context - (1 << cpu_buffer->nest);
    }
    

    And in trace_recursive_lock():

    static __always_inline bool
    trace_recursive_lock(struct ring_buffer_per_cpu *cpu_buffer)
    {
        unsigned int val = cpu_buffer->current_context;
        ...
        val |= (1 << (bit + cpu_buffer->nest));
        cpu_buffer->current_context = val;
        return false;
    }
    

    These accesses to cpu_buffer->current_context are plain C memory operations rather than volatile or atomic reads/writes (READ_ONCE() / WRITE_ONCE()).


How the Bug is Triggered

  1. Task context on CPU 0 calls kfree(), which invokes the trace_kfree() tracepoint.
  2. trace_kfree() calls ring_buffer_unlock_commit() -> trace_recursive_unlock().
  3. In trace_recursive_unlock(), because kernel/trace/ is compiled with KCSAN instrumentation, KCSAN intercepts the write/read-write to cpu_buffer->current_context and sets up a watchpoint, inserting a delay loop (delay_tsc()).
  4. While the task is stalled inside delay_tsc() with interrupts enabled, an APIC timer interrupt (sysvec_apic_timer_interrupt) fires on CPU 0.
  5. The timer interrupt handler executes trace_local_timer_entry(), which calls ring_buffer_lock_reserve() -> trace_recursive_lock().
  6. trace_recursive_lock() performs a plain read from cpu_buffer->current_context (unsigned int val = cpu_buffer->current_context;).
  7. KCSAN detects that 0xffff88810007a838 was concurrently read by the interrupt handler while a watchpoint was pending for the write by the interrupted task on the same CPU, triggering the data-race report.

Fix Considerations

  • In kernel/trace/Makefile, move KCSAN_SANITIZE := n outside the ifdef CONFIG_FUNCTION_TRACER block so that the tracer subsystem is uniformly exempt from KCSAN instrumentation regardless of whether CONFIG_FUNCTION_TRACER is enabled.
  • In kernel/trace/ring_buffer.c, use READ_ONCE() and WRITE_ONCE() (or appropriate single-load expressions) when reading and writing cpu_buffer->current_context in trace_recursive_lock(), trace_recursive_unlock(), and ring_buffer_swap_cpu() to prevent compiler load tearing or duplicate fetches across nested interrupt boundaries.
▶ ▼ ✨ AI Jobs (2)
ID Workflow Result Correct Bug Created Started Finished Revision
c1c241cf assessment-kcsan Benign: ✅ ❓ KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit 2026/11/18 13:57 2026/11/18 13:57 2026/11/18 14:13 e99b1095
5f60096a assessment-security DenialOfService: ❌ Exploitable: ❌ FilesystemTrigger: ❌ FuzzingArtifact: ❌ MemoryCorruption: ❌ NetworkNeedsSetup: ❌ NetworkTrigger: ❌ PeripheralTrigger: ❌ RemoteTrigger: ❌ Unprivileged: ❌ UserNamespace: ❌ VMGuestTrigger: ❌ VMHostTrigger: ❌ ❓ KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit 2026/11/18 13:16 2026/11/18 13:16 2026/11/18 13:38 eb4895d7

Sample crash report:
==================================================================
BUG: KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit

read to 0xffff88810007a838 of 4 bytes by interrupt on cpu 0:
 trace_recursive_lock kernel/trace/ring_buffer.c:4485 [inline]
 ring_buffer_lock_reserve+0x14e/0x9c0 kernel/trace/ring_buffer.c:5068
 __trace_buffer_lock_reserve kernel/trace/trace.h:1674 [inline]
 trace_event_buffer_lock_reserve+0x14a/0x300 kernel/trace/trace.c:1935
 trace_event_buffer_reserve+0x120/0x180 kernel/trace/trace_events.c:724
 do_trace_event_raw_event_x86_irq_vector arch/x86/include/asm/trace/irq_vectors.h:12 [inline]
 trace_event_raw_event_x86_irq_vector+0x9a/0x160 arch/x86/include/asm/trace/irq_vectors.h:12
 __do_trace_local_timer_entry arch/x86/include/asm/trace/irq_vectors.h:40 [inline]
 trace_local_timer_entry arch/x86/include/asm/trace/irq_vectors.h:40 [inline]
 __sysvec_apic_timer_interrupt+0x15c/0x1f0 arch/x86/kernel/apic/apic.c:1067
 instr_sysvec_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1062 [inline]
 sysvec_apic_timer_interrupt+0x6f/0x80 arch/x86/kernel/apic/apic.c:1062
 asm_sysvec_apic_timer_interrupt+0x1a/0x20 arch/x86/include/asm/idtentry.h:674
 rdtsc_ordered arch/x86/include/asm/tsc.h:57 [inline]
 delay_tsc+0x5b/0xa0 arch/x86/lib/delay.c:70
 udelay include/asm-generic/delay.h:64 [inline]
 delay_access kernel/kcsan/core.c:332 [inline]
 kcsan_setup_watchpoint+0x255/0x3f0 kernel/kcsan/core.c:622
 trace_recursive_unlock kernel/trace/ring_buffer.c:4512 [inline]
 ring_buffer_unlock_commit+0x331/0x3b0 kernel/trace/ring_buffer.c:4584
 __buffer_unlock_commit kernel/trace/trace.h:1695 [inline]
 trace_buffer_unlock_commit_regs+0x43/0x310 kernel/trace/trace.c:2060
 trace_event_buffer_commit+0x149/0x5b0 kernel/trace/trace.c:2035
 do_trace_event_raw_event_kfree include/trace/events/kmem.h:97 [inline]
 trace_event_raw_event_kfree+0x94/0xf0 include/trace/events/kmem.h:97
 __do_trace_kfree include/trace/events/kmem.h:97 [inline]
 trace_kfree include/trace/events/kmem.h:97 [inline]
 kfree+0x351/0x3b0 mm/slub.c:6909
 lo_rw_aio_do_completion+0x5d/0x1e0 drivers/block/loop.c:325
 lo_rw_aio_complete drivers/block/loop.c:338 [inline]
 lo_rw_aio+0x73a/0x7a0 drivers/block/loop.c:406
 do_req_filebacked drivers/block/loop.c:-1 [inline]
 loop_handle_cmd drivers/block/loop.c:1847 [inline]
 loop_process_work+0x569/0xb00 drivers/block/loop.c:1882
 loop_workfn+0x31/0x40 drivers/block/loop.c:1906
 process_one_work kernel/workqueue.c:3495 [inline]
 process_scheduled_works+0x582/0xab0 kernel/workqueue.c:3578
 worker_thread+0x56c/0x770 kernel/workqueue.c:3659
 kthread+0x248/0x2a0 kernel/kthread.c:436
 ret_from_fork+0x146/0x330 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

read-write to 0xffff88810007a838 of 4 bytes by task 111 on cpu 0:
 trace_recursive_unlock kernel/trace/ring_buffer.c:4512 [inline]
 ring_buffer_unlock_commit+0x331/0x3b0 kernel/trace/ring_buffer.c:4584
 __buffer_unlock_commit kernel/trace/trace.h:1695 [inline]
 trace_buffer_unlock_commit_regs+0x43/0x310 kernel/trace/trace.c:2060
 trace_event_buffer_commit+0x149/0x5b0 kernel/trace/trace.c:2035
 do_trace_event_raw_event_kfree include/trace/events/kmem.h:97 [inline]
 trace_event_raw_event_kfree+0x94/0xf0 include/trace/events/kmem.h:97
 __do_trace_kfree include/trace/events/kmem.h:97 [inline]
 trace_kfree include/trace/events/kmem.h:97 [inline]
 kfree+0x351/0x3b0 mm/slub.c:6909
 lo_rw_aio_do_completion+0x5d/0x1e0 drivers/block/loop.c:325
 lo_rw_aio_complete drivers/block/loop.c:338 [inline]
 lo_rw_aio+0x73a/0x7a0 drivers/block/loop.c:406
 do_req_filebacked drivers/block/loop.c:-1 [inline]
 loop_handle_cmd drivers/block/loop.c:1847 [inline]
 loop_process_work+0x569/0xb00 drivers/block/loop.c:1882
 loop_workfn+0x31/0x40 drivers/block/loop.c:1906
 process_one_work kernel/workqueue.c:3495 [inline]
 process_scheduled_works+0x582/0xab0 kernel/workqueue.c:3578
 worker_thread+0x56c/0x770 kernel/workqueue.c:3659
 kthread+0x248/0x2a0 kernel/kthread.c:436
 ret_from_fork+0x146/0x330 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245

Reported by Kernel Concurrency Sanitizer on:
CPU: 0 UID: 0 PID: 111 Comm: kworker/u8:6 Not tainted syzkaller #0 PREEMPT(full) 
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
Workqueue: loop0 loop_workfn
==================================================================
kworker/u8:6: attempt to access beyond end of device
loop0: rw=1, sector=560, nr_sectors = 32 limit=512
kworker/u8:6: attempt to access beyond end of device
loop0: rw=1, sector=624, nr_sectors = 32 limit=512
kworker/u8:6: attempt to access beyond end of device
loop0: rw=1, sector=688, nr_sectors = 16 limit=512

Crashes (3):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2026/11/17 11:05 upstream 666539eecdf4 43af6cac .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit
2026/11/17 11:04 upstream 666539eecdf4 43af6cac .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit
2026/11/17 02:27 upstream 666539eecdf4 a3e64cfc .config console log report info [disk image] [vmlinux] [kernel image] ci2-upstream-kcsan-gce KCSAN: data-race in ring_buffer_lock_reserve / ring_buffer_unlock_commit
* Struck through repros no longer work on HEAD.