Root Cause Analysis: KCSAN Data Race in fpu_flush_thread
1. Functions, Structs, and Files Involved
2. Root Cause & Invariant Violation
The core FPU locking invariant (documented in arch/x86/include/asm/fpu/api.h:52-56) dictates:
"Use fpregs_lock() while editing CPU's FPU registers or fpu->fpstate, or while using the FPU in kernel mode. A context switch will (and softirq might) save CPU's FPU registers to fpu->fpstate.regs and set TIF_NEED_FPU_LOAD leaving CPU's FPU registers in a random state."
During an execve syscall, flush_thread() calls fpu_flush_thread():
void fpu_flush_thread(void)
{
fpstate_reset(x86_task_fpu(current));
fpu_reset_fpstate_regs();
}
In fpstate_reset(struct fpu *fpu):
fpu->fpstate = &fpu->__fpstate;
__fpstate_reset(fpu->fpstate);
fpstate_reset() updates fpu->fpstate to point to the embedded default &fpu->__fpstate and writes several 8-byte configuration fields (xfeatures, xfd, user_xfeatures) via __fpstate_reset(). However:
fpstate_reset() is called without acquiring fpregs_lock().
- The calling task is running in normal preemptible process context with softirqs enabled (
PREEMPT(lazy)).
- The task's pre-execve user FPU register states are still active in hardware registers, and
TIF_NEED_FPU_LOAD is not yet set (it is only set subsequently within fpu_reset_fpstate_regs()).
3. How the Race Occurs and Produces the KCSAN Splat
When __fpstate_reset() performs an 8-byte write to fpstate->xfeatures (0xffff888104809f18), KCSAN sets a watchpoint and executes delay_access() with interrupts and preemption enabled.
A preemption / reschedule occurs on CPU 0:
- The scheduler invokes
__switch_to_asm, which switches %rsp to the incoming task (next_p)'s stack and calls __switch_to(prev_p, next_p).
- Inside
__switch_to(), before raw_cpu_write(current_task, next_p) updates current to next_p, it calls switch_fpu(prev_p, cpu).
- Because
TIF_NEED_FPU_LOAD has not yet been set for prev_p, switch_fpu() calls update_fpu_state_and_flag() -> save_fpregs_to_fpstate() -> os_xsave(fpu->fpstate).
os_xsave() executes u64 mask = fpstate->xfeatures;, triggering an 8-byte concurrent read on the exact location being modified by __fpstate_reset().
- Because
current is still prev_p (task 10978 on CPU 0) at this point in __switch_to(), but the CPU stack pointer %rsp is already pointing to next_p's stack, KCSAN's stack_trace_save() cannot unwind the stack for current and returns 0 frames (0x0), resulting in the header:
BUG: KCSAN: data-race in 0x0 / fpu_flush_thread.
4. Real-World Failure Consequence
Beyond the data race, if the task was previously using dynamic extended features (such as Intel AMX), its fpu->fpstate pointed to a large vzalloc'd buffer. Because fpstate_reset() resets fpu->fpstate = &fpu->__fpstate before invalidating hardware register state or setting TIF_NEED_FPU_LOAD, a preemption or softirq in this window causes switch_fpu() to execute XSAVE for the large live register state into the smaller default &fpu->__fpstate buffer, causing an out-of-bounds write and kernel memory corruption.