| 0/0 |
2026/11/18 00:49 |
flow |
assessment-kcsan |
1h08m
Results: map[Benign:true Explanation:### Race Summary
- **Racing Functions:** `__migrate_enable()` (via `bpf_flow_dissect` /
`bpf_prog_run_pin_on_cpu`) vs. `__migrate_disable()` (via
`bpf_prog_run_clear_cb` / `run_filter` / `packet_rcv` in softirq context).
- **Target Variable:** `current->migration_disabled` (`unsigned short` in
`struct task_struct`).
---
### Synchronization Context
Both accesses occur on the same CPU (CPU 0). Task 5152 is executing in task
context within `bpf_flow_dissect` and invokes `migrate_enable()`, reading
`p->migration_disabled` without local interrupts disabled. A hardware interrupt
arrives on CPU 0, triggering softirq processing (`net_rx_action` -> `packet_rcv`
-> `run_filter`). The softirq invokes `bpf_prog_run_clear_cb` ->
`bpf_prog_run_pin_on_cpu` -> `migrate_disable()`, which modifies
`current->migration_disabled` of the interrupted task.
Mutual exclusion is absent because `__migrate_enable()` only acquires
`guard(preempt)()`, which disables preemption but leaves hardware interrupts
enabled.
---
### Final Classification: **BENIGN**
#### Reasoning
1. **Single-CPU Strict Serialization:** Both the interrupted task and the
interrupt/softirq handler run on the exact same CPU (CPU 0). An interrupt
suspends the task, executes to completion, and returns control to the task.
There is no cross-CPU concurrent access to `p->migration_disabled`.
2. **Strictly Balanced Reentrancy:** In the softirq handler, every invocation of
`migrate_disable()` inside BPF execution helpers (`bpf_prog_run_pin_on_cpu`,
`bpf_prog_run_save_cb`) is strictly paired with a corresponding
`migrate_enable()` before the softirq finishes.
- If the interrupted task had `p->migration_disabled == 1`, softirq
increments the counter to 2 and decrements it back to 1 before exiting.
- If the interrupted task had `p->migration_disabled == 0`, softirq
increments `rq->nr_pinned` and sets `migration_disabled = 1`, then restores
`migration_disabled = 0` and decrements `rq->nr_pinned` before returning.
- In all instruction interleavings, the net change to both
`p->migration_disabled` and `this_rq_pinned()` caused by the softirq
execution is zero.
3. **No Compiler/Ordering Hazards:** The variable is a naturally aligned 16-bit
integer (`unsigned short`), so loads and stores on x86-64 are atomic and cannot
tear. There are no loops or control flow paths that could be broken by compiler
hoisting or reloads.
4. **Historical Context:** Prior to commit `378b7708194f` ("sched: Make
migrate_{en,dis}able() inline"), `migrate_disable()` and `migrate_enable()` were
defined in `kernel/sched/core.c`, which is compiled with `KCSAN_SANITIZE := n`.
Inlining these functions into `include/linux/sched.h` exposed their memory
accesses to KCSAN instrumentation in external callers, triggering this harmless
interrupt reentrancy report.
---
### Recommended Annotations
To suppress the KCSAN report:
1. Mark the access to `p->migration_disabled` in `__migrate_enable()` and
`__migrate_disable()` with `data_race()`:
```c
if (data_race(p->migration_disabled) > 1) {
data_race(p->migration_disabled--);
return;
}
```
2. Alternatively, BPF filter helpers can omit
`migrate_disable()`/`migrate_enable()` when running in interrupt context
(`!in_task()`), since execution on the local CPU cannot migrate while running in
interrupt or bottom-half context. FailureDetectableBy:]
|
| 1/1 |
2026/11/18 00:49 |
action |
kernel-checkouter |
0m
Results: map[KernelSrc:/app/workdir/cache/src/6d0c4d912ac4c28457116fbfe153047fb3052961/data]
|
| 2/1 |
2026/11/18 00:49 |
action |
kernel-builder |
0m
Results: map[KernelObj:/app/workdir/cache/build/4fc9cb093f0faa8b5951a9949e7cc5ff4ce76564/data]
|
| 3/1 |
2026/11/18 00:49 |
action |
codesearch-prepare |
0m
Results: map[Index:codesearch-index]
|
| 4/1 |
2026/11/18 00:49 |
agent |
expert |
1h08m
Model:
Results: map[Benign:true FailureDetectableBy:]
Instruction: # KCSAN Data Race Severity Analysis Protocol
You are an expert Linux kernel concurrency engineer analyzing a Kernel
Concurrency Sanitizer (KCSAN) data race report to determine if it is
**BENIGN** or **HARMFUL**.
## 1. CLASSIFICATIONS
### **BENIGN (Truly Benign)**
The logic is sound and structurally tolerant to compiler optimizations or
stale reads (or benign tearing on diagnostics/stats).
- **Diagnostics/Stats:** Reads or non-critical updates used only for `/proc`,
`/sys`, counters, timestamps, or `pr_info`.
- **Heuristic Hints:** A "hint" flag or threshold where an old value only
causes a slightly delayed update or a sub-optimal but safe fast-path.
- **Single-Writer Flag Updates:** A single writer updating flags where the
concurrent read is a simple bitwise check (e.g., `flags & MASK`). These are
historically tolerated, assuming neither "Fused Accesses" nor "Ordering
Violations" are relevant in this context.
- **Marked Reloads & Lock Re-checks:** A load feeding into a `cmpxchg()` loop,
checked against a later `READ_ONCE()` reload, or re-verified under a lock
(assuming no "Ordering Violations" on the lockless fast-path).
- **Safe Overwrites & Plain Scalar Accesses:** Writing the same value already
present, or aligned word-sized scalar accesses (e.g., `read (marked)` vs
plain `write`) where observing either the old or new value is safe.
### **HARMFUL (Logic Bug or Marking Required)**
The race causes incorrect behavior due to a synchronization failure or
because missing annotations allow the compiler or CPU to break the algorithm.
**Marking Required for Correctness:**
Do **not** classify a race as HARMFUL merely because an access lacks
`READ_ONCE()`/`WRITE_ONCE()`; you must prove one of these concrete hazards:
- **Fused Accesses:** The compiler hoists a plain load out of a tight
barrier-free spin loop causing a livelock (note: non-inlined calls,
`schedule()`, and locks act as compiler barriers), or reloads a pointer/index
between check and use (TOCTOU).
- **Torn Accesses:** An access larger than word size (e.g., 64-bit on 32-bit
arch, or multi-word structs; aligned `<= sizeof(long)` scalars and pointers
do not tear) is split into multiple non-atomic accesses. Note that
`READ_ONCE()` does **not** guarantee atomicity for 64-bit variables on 32-bit
architectures.
- **Ordering Violations:** The race breaks a required "happens-before"
relationship between dependent memory accesses (requires primitives with
implied or explicit memory barriers).
**Logic Bugs:**
A fundamental synchronization failure. Marking accesses will **not** fix it;
the logic itself must change.
- **Pointers/Lifecycle:** The racing variable is a pointer being dereferenced
or a refcount governing object lifecycle (Use-After-Free risk).
- **Control Flow:** The variable guards a critical section, memory allocation,
or hardware command.
- **Bitfields:** Concurrent writes to different bits in the same word.
Compilers often use non-atomic read-modify-write sequences, meaning a
write to `bit_A` can "clobber" a concurrent write to `bit_B`. However,
do not blindly assume all bitfield accesses are harmful; you must prove
that a concurrent write actually clobbers another in a way that breaks
logic.
- **Complex Structures:** Races on shared lists, trees, or hashmaps.
- **Lossy Updates:** Concurrent plain RMW operations (e.g., `var++`) on
non-diagnostic variables where every increment must be preserved.
- **State Machines:** Races allowing a state machine to bypass transitions
or enter an invalid state.
- **Adjacent Unsynchronized Operations:** Consider races happening at the
same time. For example, if both threads execute `struct->has_elements = true;
list_add(node, &struct->list);`, the race on `has_elements` implies an
adjacent race on `list_head`, which is HARMFUL.
## 2. RESEARCH & ANALYSIS WORKFLOW
1. **Locate the Race:** Find the exact variables and functions in the stack
traces. Do not speculate about hypothetical compiler
behaviors or theoretical dangers (e.g., dismissing something as
"fundamentally unsafe") without tracing the actual data flow to a crash.
2. **Contextualize:** Identify held locks, RCU sections, or interrupt
contexts. Explain why the race is possible (e.g., "Thread A holds `lock`,
but Thread B is a lockless reader").
3. **Data Flow:** Follow the racing variable into its subsequent uses. If
the reader sees a stale or torn value, what branch is taken? Does it lead
to a pointer dereference or `BUG_ON()`? **You must prove harm via this
data flow.**
4. **Failure Sequencing (if HARMFUL):** Construct a two-column execution trace
demonstrating the failure.
*Example format:*
```text
CPU0 CPU1
function_a()
// lockless read
ptr = global_ptr
<PREEMPT>
function_b()
lock(&my_lock)
global_ptr = NULL
unlock(&my_lock)
kfree(ptr)
if (ptr)
*ptr = 1; // -> Use-After-Free!
```
## 3. OUTPUT FORMAT
- **Race Summary:** `[Function A]` vs `[Function B]` on `[Variable/Field]`.
- **Synchronization context:** List held locks and explain the lack of mutual
exclusion.
- **Final Classification:** **BENIGN** or **HARMFUL**.
**If BENIGN:**
- **Reasoning:** Briefly explain why the race is structurally tolerant to stale
reads (or benign tearing on diagnostics/stats).
- **Recommended Annotations:** Suggest appropriate annotations (e.g.,
`READ_ONCE()`, `WRITE_ONCE()`, `data_race()`).
**If HARMFUL:**
- **Failure Reasoning:** Explain and provide a two-column interleaving showing
exactly how the race leads to a failure such as corruption or kernel crash.
- **FailureDetectableBy:** One of:
- `"kasan"`: memory safety bug (use-after-free, out-of-bounds, double-free).
- `"kmsan"`: uninitialized memory read/exposure (e.g. missing publish/subscribe ordering).
- `"any"`: kernel crash/hang (`BUG_ON`, `WARN_ON`, NULL deref, GPF, deadlock).
- `"user"`: user-visible ABI/semantic corruption detectable by userspace without a kernel crash.
- `"none"`: purely internal kernel state/accounting bug without crash or user-visible invariant violation.
- **Recommended Fix (ONLY if fix is trivial):** Suggest a structural fix (e.g.,
"Hold `mapping->i_pages` lock", "Convert to `atomic_t`") or required memory
ordering annotations (e.g., "Wrap in `READ_ONCE()`", "Use
`smp_load_acquire()`").
Don't make assumptions about the kernel source code (it may be different from what you assume it is).
Extensively use the provided code access tools (codesearch-*, git-*, grepper, etc)
to examine the actual source code, and confirm any assumptions.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt:
The data race report is:
==================================================================
BUG: KCSAN: data-race in bpf_flow_dissect / bpf_prog_run_clear_cb
write to 0xffff8881046d0478 of 2 bytes by interrupt on cpu 0:
__migrate_disable include/linux/sched.h:2490 [inline]
migrate_disable include/linux/sched.h:2512 [inline]
bpf_prog_run_pin_on_cpu include/linux/filter.h:836 [inline]
bpf_prog_run_clear_cb+0x77/0x290 include/linux/filter.h:1119
run_filter net/packet/af_packet.c:2111 [inline]
packet_rcv+0x26e/0xba0 net/packet/af_packet.c:2184
deliver_skb+0x10b/0x170 net/core/dev.c:2564
deliver_ptype_list_skb net/core/dev.c:2579 [inline]
__netif_receive_skb_core+0x13e0/0x19a0 net/core/dev.c:6391
__netif_receive_skb_list_core+0x112/0x520 net/core/dev.c:6522
__netif_receive_skb_list net/core/dev.c:6589 [inline]
netif_receive_skb_list_internal+0x471/0x5e0 net/core/dev.c:6680
gro_normal_list include/net/gro.h:539 [inline]
gro_flush_normal include/net/gro.h:547 [inline]
napi_complete_done+0x19c/0x410 net/core/dev.c:7048
virtqueue_napi_complete drivers/net/virtio_net.c:736 [inline]
virtnet_poll+0x1a9a/0x1f60 drivers/net/virtio_net.c:3048
__napi_poll+0x73/0x350 net/core/dev.c:7997
napi_poll net/core/dev.c:8060 [inline]
net_rx_action+0x33f/0x850 net/core/dev.c:8217
handle_softirqs+0x109/0x310 kernel/softirq.c:669
__do_softirq kernel/softirq.c:703 [inline]
invoke_softirq kernel/softirq.c:542 [inline]
__irq_exit_rcu+0x40/0xf0 kernel/softirq.c:786
common_interrupt+0x83/0x90 arch/x86/kernel/irq.c:273
asm_common_interrupt+0x26/0x40 arch/x86/include/asm/idtentry.h:665
__preempt_count_add arch/x86/include/asm/preempt.h:98 [inline]
delay_tsc+0x4b/0xa0 arch/x86/lib/delay.c:77
udelay include/asm-generic/delay.h:64 [inline]
delay_access kernel/kcsan/core.c:332 [inline]
kcsan_setup_watchpoint+0x255/0x3f0 kernel/kcsan/core.c:622
__migrate_enable include/linux/sched.h:2457 [inline]
migrate_enable include/linux/sched.h:2517 [inline]
bpf_prog_run_pin_on_cpu include/linux/filter.h:838 [inline]
bpf_flow_dissect+0x162/0x360 net/core/flow_dissector.c:1024
bpf_prog_test_run_flow_dissector+0x25d/0x360 net/bpf/test_run.c:1565
bpf_prog_test_run+0x21b/0x360 kernel/bpf/syscall.c:4860
__sys_bpf+0x914/0xc50 kernel/bpf/syscall.c:6478
__do_sys_bpf kernel/bpf/syscall.c:6582 [inline]
__se_sys_bpf kernel/bpf/syscall.c:6579 [inline]
__x64_sys_bpf+0x69/0x80 kernel/bpf/syscall.c:6579
x64_sys_call+0xf26/0x2550 arch/x86/include/generated/asm/syscalls_64.h:322
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x11e/0x3a0 arch/x86/entry/syscall_64.c:89
entry_SYSCALL_64_after_hwframe+0x77/0x7f
read to 0xffff8881046d0478 of 2 bytes by task 5152 on cpu 0:
__migrate_enable include/linux/sched.h:2457 [inline]
migrate_enable include/linux/sched.h:2517 [inline]
bpf_prog_run_pin_on_cpu include/linux/filter.h:838 [inline]
bpf_flow_dissect+0x162/0x360 net/core/flow_dissector.c:1024
bpf_prog_test_run_flow_dissector+0x25d/0x360 net/bpf/test_run.c:1565
bpf_prog_test_run+0x21b/0x360 kernel/bpf/syscall.c:4860
__sys_bpf+0x914/0xc50 kernel/bpf/syscall.c:6478
__do_sys_bpf kernel/bpf/syscall.c:6582 [inline]
__se_sys_bpf kernel/bpf/syscall.c:6579 [inline]
__x64_sys_bpf+0x69/0x80 kernel/bpf/syscall.c:6579
x64_sys_call+0xf26/0x2550 arch/x86/include/generated/asm/syscalls_64.h:322
do_syscall_x64 arch/x86/entry/syscall_64.c:61 [inline]
do_syscall_64+0x11e/0x3a0 arch/x86/entry/syscall_64.c:89
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Reported by Kernel Concurrency Sanitizer on:
CPU: 0 UID: 0 PID: 5152 Comm: syz.0.776 Not tainted syzkaller #0 PREEMPT(lazy)
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 11/06/2026
==================================================================
Reply: ### Race Summary
- **Racing Functions:** `__migrate_enable()` (via `bpf_flow_dissect` / `bpf_prog_run_pin_on_cpu`) vs. `__migrate_disable()` (via `bpf_prog_run_clear_cb` / `run_filter` / `packet_rcv` in softirq context).
- **Target Variable:** `current->migration_disabled` (`unsigned short` in `struct task_struct`).
---
### Synchronization Context
Both accesses occur on the same CPU (CPU 0). Task 5152 is executing in task context within `bpf_flow_dissect` and invokes `migrate_enable()`, reading `p->migration_disabled` without local interrupts disabled. A hardware interrupt arrives on CPU 0, triggering softirq processing (`net_rx_action` -> `packet_rcv` -> `run_filter`). The softirq invokes `bpf_prog_run_clear_cb` -> `bpf_prog_run_pin_on_cpu` -> `migrate_disable()`, which modifies `current->migration_disabled` of the interrupted task.
Mutual exclusion is absent because `__migrate_enable()` only acquires `guard(preempt)()`, which disables preemption but leaves hardware interrupts enabled.
---
### Final Classification: **BENIGN**
#### Reasoning
1. **Single-CPU Strict Serialization:** Both the interrupted task and the interrupt/softirq handler run on the exact same CPU (CPU 0). An interrupt suspends the task, executes to completion, and returns control to the task. There is no cross-CPU concurrent access to `p->migration_disabled`.
2. **Strictly Balanced Reentrancy:** In the softirq handler, every invocation of `migrate_disable()` inside BPF execution helpers (`bpf_prog_run_pin_on_cpu`, `bpf_prog_run_save_cb`) is strictly paired with a corresponding `migrate_enable()` before the softirq finishes.
- If the interrupted task had `p->migration_disabled == 1`, softirq increments the counter to 2 and decrements it back to 1 before exiting.
- If the interrupted task had `p->migration_disabled == 0`, softirq increments `rq->nr_pinned` and sets `migration_disabled = 1`, then restores `migration_disabled = 0` and decrements `rq->nr_pinned` before returning.
- In all instruction interleavings, the net change to both `p->migration_disabled` and `this_rq_pinned()` caused by the softirq execution is zero.
3. **No Compiler/Ordering Hazards:** The variable is a naturally aligned 16-bit integer (`unsigned short`), so loads and stores on x86-64 are atomic and cannot tear. There are no loops or control flow paths that could be broken by compiler hoisting or reloads.
4. **Historical Context:** Prior to commit `378b7708194f` ("sched: Make migrate_{en,dis}able() inline"), `migrate_disable()` and `migrate_enable()` were defined in `kernel/sched/core.c`, which is compiled with `KCSAN_SANITIZE := n`. Inlining these functions into `include/linux/sched.h` exposed their memory accesses to KCSAN instrumentation in external callers, triggering this harmless interrupt reentrancy report.
---
### Recommended Annotations
To suppress the KCSAN report:
1. Mark the access to `p->migration_disabled` in `__migrate_enable()` and `__migrate_disable()` with `data_race()`:
```c
if (data_race(p->migration_disabled) > 1) {
data_race(p->migration_disabled--);
return;
}
```
2. Alternatively, BPF filter helpers can omit `migrate_disable()`/`migrate_enable()` when running in interrupt context (`!in_task()`), since execution on the local CPU cannot migrate while running in interrupt or bottom-half context.
|
| 5/2 |
2026/11/18 00:49 |
llm |
expert |
0m
Model:
Tokens:
input: 7759
output: 36
thoughts: 990
Thoughts: **Analyzing the KCSAN Data Race Report**
Okay, let's dive into this KCSAN data race report. The core issue seems to be a conflict within `bpf_flow_dissect` and, more specifically, `bpf_prog_run_clear_cb`. The report pinpoints a 2-byte write to memory address `0xffff8881046d0478` and a 2-byte read from the same location.
The critical thing is that both the read and write are happening on CPU 0. I note the write is triggered by an interrupt, while the read is coming from task 5152. This immediately screams "race condition". The trace shows the write path involving `__migrate_disable`, called deep inside an interrupt handler from `packet_rcv`, which in turn comes from the softirq processing. On the other hand the read path involves `__migrate_enable`.
I see `kcsan_setup_watchpoint` involved, so let's backtrack that. The `kcsan` watchpoint was set when task 5152 was running within `__migrate_enable`, then an interrupt struck CPU 0, triggering the `migrate_disable`. This means the interrupt happened *during* the critical section.
The key observation is that the read and write are both accessing the same 2 bytes at `0xffff8881046d0478`. This memory address immediately has me thinking about the `current->migration_disabled` flag! Since `migrate_disable` and `migrate_enable` directly modify that scheduler flag, it's a prime suspect.
I'm going to quickly check `include/linux/sched.h` around lines 2450 to 2520 to confirm the variable's location. This should confirm my suspicion and, if so, quickly point us to the heart of the problem!
|
| 6/2 |
2026/11/18 00:49 |
tool |
read-file |
0m
Args: map[File:include/linux/sched.h FirstLine:2440 LineCount:90]
Results: map[Contents:2440: #define this_rq_raw() PERCPU_PTR(&runqueues)
2441: #endif
2442: #define this_rq_pinned() (*(unsigned int *)((void *)this_rq_raw() + RQ_nr_pinned))
2443:
2444: static inline void __migrate_enable(void)
2445: {
2446: struct task_struct *p = current;
2447:
2448: #ifdef CONFIG_DEBUG_PREEMPT
2449: /*
2450: * Check both overflow from migrate_disable() and superfluous
2451: * migrate_enable().
2452: */
2453: if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
2454: return;
2455: #endif
2456:
2457: if (p->migration_disabled > 1) {
2458: p->migration_disabled--;
2459: return;
2460: }
2461:
2462: /*
2463: * Ensure stop_task runs either before or after this, and that
2464: * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
2465: */
2466: guard(preempt)();
2467: if (unlikely(p->cpus_ptr != &p->cpus_mask))
2468: ___migrate_enable();
2469: /*
2470: * Mustn't clear migration_disabled() until cpus_ptr points back at the
2471: * regular cpus_mask, otherwise things that race (eg.
2472: * select_fallback_rq) get confused.
2473: */
2474: barrier();
2475: p->migration_disabled = 0;
2476: this_rq_pinned()--;
2477: }
2478:
2479: static inline void __migrate_disable(void)
2480: {
2481: struct task_struct *p = current;
2482:
2483: if (p->migration_disabled) {
2484: #ifdef CONFIG_DEBUG_PREEMPT
2485: /*
2486: *Warn about overflow half-way through the range.
2487: */
2488: WARN_ON_ONCE((s16)p->migration_disabled < 0);
2489: #endif
2490: p->migration_disabled++;
2491: return;
2492: }
2493:
2494: guard(preempt)();
2495: this_rq_pinned()++;
2496: p->migration_disabled = 1;
2497: }
2498: #else /* !COMPILE_OFFSETS */
2499: static inline void __migrate_disable(void) { }
2500: static inline void __migrate_enable(void) { }
2501: #endif /* !COMPILE_OFFSETS */
2502:
2503: /*
2504: * So that it is possible to not export the runqueues variable, define and
2505: * export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
2506: * them for the modules. The macro "INSTANTIATE_EXPORTED_MIGRATE_DISABLE" will
2507: * be defined in kernel/sched/core.c.
2508: */
2509: #ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
2510: static __always_inline void migrate_disable(void)
2511: {
2512: __migrate_disable();
2513: }
2514:
2515: static __always_inline void migrate_enable(void)
2516: {
2517: __migrate_enable();
2518: }
2519: #else /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
2520: extern void migrate_disable(void);
2521: extern void migrate_enable(void);
2522: #endif /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
2523:
2524: #else /* MODULE */
2525: extern void migrate_disable(void);
2526: extern void migrate_enable(void);
2527: #endif /* MODULE */
2528:
2529: DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
]
|
| 7/2 |
2026/11/18 00:49 |
llm |
expert |
0m
Model:
Tokens:
input: 10484
output: 35
thoughts: 799
Thoughts: **Investigating a Potential Race Condition in Task Migration Disabling**
Okay, so I've stumbled upon something interesting: `p->migration_disabled`. My initial thought is to check `include/linux/sched.h`, lines 2457 and 2490 specifically, to understand how it's handled. The decrement and increment operations there are straightforward enough, but the alarm bells are starting to ring. I need to know if `migrate_disable()` and `migrate_enable()` are ever called from interrupt or softirq context. This could be where the trouble lies.
Let's dig into the call chain. The stack trace shows the write is happening during an interrupt on CPU 0. We've got `__migrate_disable`, `migrate_disable`, then `bpf_prog_run_pin_on_cpu`, all the way down to `packet_rcv`, which is part of an softirq. Aha! `packet_rcv` is indeed running in softirq context! This is the key piece of the puzzle. Now I need to figure out what task `current` is when this happens.
When a softirq is executed after an interrupt return, `current` is whatever task was interrupted. In this instance, task 5152 was interrupted, and, looking at the traces it was in the middle of executing `__migrate_enable()`! This is getting interesting. So, task 5152 calls `migrate_enable()`. `__migrate_enable()` reads `p->migration_disabled`. Then, while that very task is executing inside `__migrate_enable()`, or KCSAN delayed it, an interrupt occurs on CPU 0. The interrupt executes softirqs, which then call `packet_rcv` -> `run_filter` -> `bpf_prog_run_clear_cb` -> `bpf_prog_run_pin_on_cpu` -> `migrate_disable()`. The critical part is that `migrate_disable()` operates on `current` which is task 5152, and it modifies `p->migration_disabled`!
We've got a potential race. I need to confirm whether it's valid to call `migrate_disable()` or `migrate_enable()` from within a softirq or interrupt context. The issue clearly centers around `bpf_prog_run_pin_on_cpu`. I need to review `include/linux/filter.h` around lines 836 and 1119 to understand the implications of BPF program execution within this context.
|
| 8/2 |
2026/11/18 00:49 |
tool |
read-file |
0m
Args: map[File:include/linux/filter.h FirstLine:820 LineCount:50]
Results: map[Contents: 820: return __bpf_prog_run(prog, ctx, bpf_dispatcher_nop_func);
821: }
822:
823: /*
824: * Use in preemptible and therefore migratable context to make sure that
825: * the execution of the BPF program runs on one CPU.
826: *
827: * This uses migrate_disable/enable() explicitly to document that the
828: * invocation of a BPF program does not require reentrancy protection
829: * against a BPF program which is invoked from a preempting task.
830: */
831: static inline u32 bpf_prog_run_pin_on_cpu(const struct bpf_prog *prog,
832: const void *ctx)
833: {
834: u32 ret;
835:
836: migrate_disable();
837: ret = bpf_prog_run(prog, ctx);
838: migrate_enable();
839: return ret;
840: }
841:
842: static inline bool is_stack_arg_ldx(const struct bpf_insn *insn)
843: {
844: return insn->code == (BPF_LDX | BPF_MEM | BPF_DW) &&
845: insn->src_reg == BPF_REG_PARAMS &&
846: insn->off > 0 && insn->off % 8 == 0;
847: }
848:
849: static inline bool is_stack_arg_st(const struct bpf_insn *insn)
850: {
851: return insn->code == (BPF_ST | BPF_MEM | BPF_DW) &&
852: insn->dst_reg == BPF_REG_PARAMS &&
853: insn->off < 0 && insn->off % 8 == 0;
854: }
855:
856: static inline bool is_stack_arg_stx(const struct bpf_insn *insn)
857: {
858: return insn->code == (BPF_STX | BPF_MEM | BPF_DW) &&
859: insn->dst_reg == BPF_REG_PARAMS &&
860: insn->off < 0 && insn->off % 8 == 0;
861: }
862:
863: #define BPF_SKB_CB_LEN QDISC_CB_PRIV_LEN
864:
865: struct bpf_skb_data_end {
866: struct qdisc_skb_cb qdisc_cb;
867: void *data_meta;
868: void *data_end;
869: };
]
|
| 9/2 |
2026/11/18 00:49 |
llm |
expert |
0m
Model:
Tokens:
input: 12702
output: 36
thoughts: 23
Thoughts: Here's my summary, as if I were thinking these thoughts:
**Diving Deep into the Filter Header**
Okay, so we're dealing with something related to filtering, likely something low-level. I need to make sure I'm getting the full picture, and the best place to start, given the context, is definitely the header file. And, more specifically, I want to confirm something that is defined on line 1119 of `include/linux/filter.h`. It is easy to skim the code and miss a crucial detail. I think I know what to expect, but I'll make sure to verify exactly what's going on there. I'm going to take a look and make a mental note of the critical aspect that I need to confirm. Let's see what we're actually working with.
|
| 10/2 |
2026/11/18 00:49 |
tool |
read-file |
0m
Args: map[File:include/linux/filter.h FirstLine:1100 LineCount:40]
Results: map[Contents:1100: struct sk_buff *skb)
1101: {
1102: u32 res;
1103:
1104: migrate_disable();
1105: res = __bpf_prog_run_save_cb(prog, skb);
1106: migrate_enable();
1107: return res;
1108: }
1109:
1110: static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
1111: struct sk_buff *skb)
1112: {
1113: u8 *cb_data = bpf_skb_cb(skb);
1114: u32 res;
1115:
1116: if (unlikely(prog->cb_access))
1117: memset(cb_data, 0, BPF_SKB_CB_LEN);
1118:
1119: res = bpf_prog_run_pin_on_cpu(prog, skb);
1120: return res;
1121: }
1122:
1123: DECLARE_BPF_DISPATCHER(xdp)
1124:
1125: DECLARE_STATIC_KEY_FALSE(bpf_master_redirect_enabled_key);
1126:
1127: u32 xdp_master_redirect(struct xdp_buff *xdp);
1128:
1129: void bpf_prog_change_xdp(struct bpf_prog *prev_prog, struct bpf_prog *prog);
1130:
1131: static inline u32 bpf_prog_insn_size(const struct bpf_prog *prog)
1132: {
1133: return prog->len * sizeof(struct bpf_insn);
1134: }
1135:
1136: static inline unsigned int bpf_prog_size(unsigned int proglen)
1137: {
1138: return max(sizeof(struct bpf_prog),
1139: offsetof(struct bpf_prog, insns[proglen]));
]
|
| 11/2 |
2026/11/18 00:49 |
llm |
expert |
0m
Model:
Tokens:
input: 13550
output: 38
thoughts: 343
Thoughts: **Analyzing BPF Program Execution and Migration**
Okay, let's break this down. My initial observation is that `bpf_prog_run_pin_on_cpu` is calling `migrate_disable()` and `migrate_enable()`. This immediately raises a flag. Why would we be disabling migration within a function that seemingly executes a BPF program? It's not immediately obvious why we're doing this.
Let's look at the comment above the function. Ah, the comment clarifies. It says: "Use in preemptible and therefore migratable context to make sure that the execution of the BPF program runs on one CPU. This uses `migrate_disable/enable()` explicitly to document that the invocation of a BPF program does not require reentrancy protection against a BPF program which is invoked from a preempting task." So, we're explicitly disabling migration to pin the execution to a specific CPU. The comment is highlighting an interesting point about reentrancy within the BPF context.
Now, I see a call stack - `run_filter` at `net/packet/af_packet.c:2111`, called inline by `packet_rcv` at `net/packet/af_packet.c:2184`. That's interesting, let's look at `net/packet/af_packet.c:2111`, because that's where `run_filter` is used. This call chain might be very relevant to understanding the use-case for these functions.
|
| 12/2 |
2026/11/18 00:50 |
tool |
read-file |
0m
Args: map[File:net/packet/af_packet.c FirstLine:2100 LineCount:40]
Results: map[Contents:2100: }
2101:
2102: static unsigned int run_filter(struct sk_buff *skb,
2103: const struct sock *sk,
2104: unsigned int res)
2105: {
2106: struct sk_filter *filter;
2107:
2108: rcu_read_lock();
2109: filter = rcu_dereference(sk->sk_filter);
2110: if (filter != NULL)
2111: res = bpf_prog_run_clear_cb(filter->prog, skb);
2112: rcu_read_unlock();
2113:
2114: return res;
2115: }
2116:
2117: static int packet_rcv_vnet(struct msghdr *msg, const struct sk_buff *skb,
2118: size_t *len, int vnet_hdr_sz)
2119: {
2120: struct virtio_net_hdr_mrg_rxbuf vnet_hdr = { .num_buffers = 0 };
2121:
2122: if (*len < vnet_hdr_sz)
2123: return -EINVAL;
2124: *len -= vnet_hdr_sz;
2125:
2126: if (virtio_net_hdr_from_skb(skb, (struct virtio_net_hdr *)&vnet_hdr, vio_le(), true, 0))
2127: return -EINVAL;
2128:
2129: return memcpy_to_msg(msg, (void *)&vnet_hdr, vnet_hdr_sz);
2130: }
2131:
2132: /*
2133: * This function makes lazy skb cloning in hope that most of packets
2134: * are discarded by BPF.
2135: *
2136: * Note tricky part: we DO mangle shared skb! skb->data, skb->len
2137: * and skb->cb are mangled. It works because (and until) packets
2138: * falling here are owned by current CPU. Output packets are cloned
2139: * by dev_queue_xmit_nit(), input packets are processed by net_bh
]
|
| 13/2 |
2026/11/18 00:50 |
llm |
expert |
0m
Model:
Tokens:
input: 14902
output: 28
thoughts: 137
Thoughts: **Investigating the Concurrency of `migrate_disable()`**
Okay, hold on a second. I need to re-examine `run_filter` within `af_packet.c`. My mind is racing. I see that it's being invoked within a softirq context, or essentially any context where `packet_rcv` is called. That means we're looking at scenarios like `netif_receive_skb_list`, which leads us to `napi_poll` and the softirq context. This raises a crucial question in my mind: can `migrate_disable()` and `migrate_enable()` safely be called from *within* a softirq or even a hardirq context?
This feels like a concurrency minefield, and I need to be absolutely certain. Before I jump to conclusions, I need to check the kernel's documentation, or better yet, the comments within `migrate_disable`'s definition itself. Let's dig into `kernel/sched/core.c` or wherever it resides. We're looking for comments that definitively address the context in which these functions are intended to operate. Understanding these constraints is essential for maintaining system stability and performance in this area.
|
| 14/2 |
2026/11/18 00:50 |
tool |
grepper |
0m
Args: map[Expression:migrate_disable PathPrefixes:[kernel/sched/]]
Results: map[Output:kernel/sched/core.c=2469=do_set_cpus_allowed(struct task_struct *p, struct affinity_context *ctx);
kernel/sched/core.c-2470-
kernel/sched/core.c:2471:static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
kernel/sched/core.c-2472-{
--
kernel/sched/core.c=2498=EXPORT_SYMBOL_GPL(___migrate_enable);
kernel/sched/core.c-2499-
kernel/sched/core.c:2500:void migrate_disable(void)
kernel/sched/core.c-2501-{
kernel/sched/core.c:2502: __migrate_disable();
kernel/sched/core.c-2503-}
kernel/sched/core.c:2504:EXPORT_SYMBOL_GPL(migrate_disable);
kernel/sched/core.c-2505-
--
kernel/sched/core.c=2539=static inline bool is_cpu_allowed(struct task_struct *p, int cpu)
--
kernel/sched/core.c-2544-
kernel/sched/core.c:2545: /* migrate_disabled() must be allowed to finish. */
kernel/sched/core.c-2546- if (is_migration_disabled(p))
--
kernel/sched/core.c=2924=void release_user_cpus_ptr(struct task_struct *p)
--
kernel/sched/core.c-2943- *
kernel/sched/core.c:2944: * migrate_disable();
kernel/sched/core.c-2945- * <preempted>
--
kernel/sched/core.c-2953- *
kernel/sched/core.c:2954: * migrate_disable();
kernel/sched/core.c-2955- * <preempted>
--
kernel/sched/core.c-2988- * <P0>
kernel/sched/core.c:2989: * migrate_disable();
kernel/sched/core.c-2990- * <preempted>
--
kernel/sched/core.c=7211=static void __sched notrace __schedule(int sched_mode)
--
kernel/sched/core.c-7238- rcu_note_context_switch(preempt);
kernel/sched/core.c:7239: migrate_disable_switch(rq, prev);
kernel/sched/core.c-7240-
--
kernel/sched/core.c=8698=int sched_cpu_deactivate(unsigned int cpu)
--
kernel/sched/core.c-8720- * From this point forward, this CPU will refuse to run any task that
kernel/sched/core.c:8721: * is not: migrate_disable() or KTHREAD_IS_PER_CPU, and will actively
kernel/sched/core.c-8722- * push those tasks away until this gets cleared, see
--
kernel/sched/deadline.c=3057=static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq)
--
kernel/sched/deadline.c-3088- * 1. It is possible the task was scheduled,
kernel/sched/deadline.c:3089: * migrate_disabled was set and then got preempted,
kernel/sched/deadline.c-3090- * so we must check the task migration disable
--
kernel/sched/ext/ext.c=2814=static bool task_can_run_on_remote_rq(struct scx_sched *sch,
--
kernel/sched/ext/ext.c-2841- * If @p has migration disabled, @p->cpus_ptr is updated to contain only
kernel/sched/ext/ext.c:2842: * the pinned CPU in migrate_disable_switch() while @p is being switched
kernel/sched/ext/ext.c-2843- * out. However, put_prev_task_scx() is called before @p->cpus_ptr is
--
kernel/sched/ext/idle.c=923=static bool check_builtin_idle_enabled(struct scx_sched *sch)
--
kernel/sched/ext/idle.c-939- *
kernel/sched/ext/idle.c:940: * Since the BPF prolog calls migrate_disable() only when CONFIG_PREEMPT_RCU
kernel/sched/ext/idle.c-941- * is enabled (via rcu_read_lock_dont_migrate()), migration_disabled == 1 for
kernel/sched/ext/idle.c-942- * the current task is ambiguous only in that case: it could be from the BPF
kernel/sched/ext/idle.c:943: * prolog rather than a real migrate_disable() call.
kernel/sched/ext/idle.c-944- *
kernel/sched/ext/idle.c:945: * Without CONFIG_PREEMPT_RCU, the BPF prolog never calls migrate_disable(),
kernel/sched/ext/idle.c-946- * so migration_disabled == 1 always means the task is truly
--
kernel/sched/rt.c=1900=static struct rq *find_lock_lowest_rq(struct task_struct *task, struct rq *rq)
--
kernel/sched/rt.c-1932- * It is possible the task was scheduled, set
kernel/sched/rt.c:1933: * "migrate_disabled" and then got preempted, so we must
kernel/sched/rt.c-1934- * check the task migration disable flag here too.
]
|
| 15/2 |
2026/11/18 00:50 |
llm |
expert |
0m
Model:
Tokens:
input: 16784
output: 34
thoughts: 73
Thoughts: Here's my thought process, as if I were analyzing this code:
**Finding the Source of `__migrate_enable`**
Alright, let's track down the commit that introduced the inlined functions `__migrate_disable` and `__migrate_enable` within `include/linux/sched.h`. That's the key to understanding the context of this change. My initial approach will be to use `git log` on the `include/linux/sched.h` file itself. I'll need to be precise in my search. I could look for the symbol name directly, `SymbolName="__migrate_enable"`, or, because I suspect these are macros, I'll broaden the net a bit and look for a regular expression match on the code, `CodeRegexp="__migrate_enable"`. Either of those options should pinpoint the exact commit or commits responsible for adding those macros. Once I have the commit hash, I can then dive deeper to understand the rationale and surrounding code modifications. This should be straightforward, but I need to make sure I am looking in the correct git repository.
|
| 16/2 |
2026/11/18 00:50 |
tool |
git-log |
0m
Args: map[CodeRegexp:__migrate_enable PathPrefix:include/linux/sched.h]
Results: map[Output:378b7708194f sched: Make migrate_{en,dis}able() inline
]
|
| 17/2 |
2026/11/18 00:50 |
llm |
expert |
0m
Model:
Tokens:
input: 17160
output: 27
thoughts: 25
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Diving into the Code History**
Alright, time to get granular. Commit `378b7708194f` is the target; I'm pulling it up using `git show`. I need to understand the exact modifications made at that specific point in the project's history. No quick `git log` overview here, I want to see the diff, the author, the commit message β the whole enchilada. I'm looking for the changes that this specific commit introduced. I'm prepared to dissect the code and understand any potential impact of these changes. I'll be scrutinizing the context and its impact in the history of the codebase. My goal is to extract the details that will help me understand why this commit was necessary or, alternatively, how it might have caused a regression or introduced a bug.
|
| 18/2 |
2026/11/18 00:50 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 378b7708194fff77c9020392067329931c3fcc04
Author: Menglong Dong <menglong8.dong@gmail.com>
Date: Wed Sep 17 14:09:15 2025 +0800
sched: Make migrate_{en,dis}able() inline
For now, migrate_enable and migrate_disable are global, which makes them
become hotspots in some case. Take BPF for example, the function calling
to migrate_enable and migrate_disable in BPF trampoline can introduce
significant overhead, and following is the 'perf top' of FENTRY's
benchmark (./tools/testing/selftests/bpf/bench trig-fentry):
54.63% bpf_prog_2dcccf652aac1793_bench_trigger_fentry [k]
bpf_prog_2dcccf652aac1793_bench_trigger_fentry
10.43% [kernel] [k] migrate_enable
10.07% bpf_trampoline_6442517037 [k] bpf_trampoline_6442517037
8.06% [kernel] [k] __bpf_prog_exit_recur
4.11% libc.so.6 [.] syscall
2.15% [kernel] [k] entry_SYSCALL_64
1.48% [kernel] [k] memchr_inv
1.32% [kernel] [k] fput
1.16% [kernel] [k] _copy_to_user
0.73% [kernel] [k] bpf_prog_test_run_raw_tp
So in this commit, we make migrate_enable/migrate_disable inline to obtain
better performance. The struct rq is defined internally in
kernel/sched/sched.h, and the field "nr_pinned" is accessed in
migrate_enable/migrate_disable, which makes it hard to make them inline.
Alexei Starovoitov suggests to generate the offset of "nr_pinned" in [1],
so we can define the migrate_enable/migrate_disable in
include/linux/sched.h and access "this_rq()->nr_pinned" with
"(void *)this_rq() + RQ_nr_pinned".
The offset of "nr_pinned" is generated in include/generated/rq-offsets.h
by kernel/sched/rq-offsets.c.
Generally speaking, we move the definition of migrate_enable and
migrate_disable to include/linux/sched.h from kernel/sched/core.c. The
calling to __set_cpus_allowed_ptr() is leaved in ___migrate_enable().
The "struct rq" is not available in include/linux/sched.h, so we can't
access the "runqueues" with this_cpu_ptr(), as the compilation will fail
in this_cpu_ptr() -> raw_cpu_ptr() -> __verify_pcpu_ptr():
typeof((ptr) + 0)
So we introduce the this_rq_raw() and access the runqueues with
arch_raw_cpu_ptr/PERCPU_PTR directly.
The variable "runqueues" is not visible in the kernel modules, and export
it is not a good idea. As Peter Zijlstra advised in [2], we define and
export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
them for the modules.
Before this patch, the performance of BPF FENTRY is:
fentry : 113.030 Β± 0.149M/s
fentry : 112.501 Β± 0.187M/s
fentry : 112.828 Β± 0.267M/s
fentry : 115.287 Β± 0.241M/s
After this patch, the performance of BPF FENTRY increases to:
fentry : 143.644 Β± 0.670M/s
fentry : 149.764 Β± 0.362M/s
fentry : 149.642 Β± 0.156M/s
fentry : 145.263 Β± 0.221M/s
Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/bpf/CAADnVQ+5sEDKHdsJY5ZsfGDO_1SEhhQWHrt2SMBG5SYyQ+jt7w@mail.gmail.com/ [1]
Link: https://lore.kernel.org/all/20250819123214.GH4067720@noisy.programming.kicks-ass.net/ [2]
diff --git a/Kbuild b/Kbuild
index f327ca86990c..13324b4bbe23 100644
--- a/Kbuild
+++ b/Kbuild
@@ -34,13 +34,24 @@ arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file)
$(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s FORCE
$(call filechk,offsets,__ASM_OFFSETS_H__)
+# Generate rq-offsets.h
+
+rq-offsets-file := include/generated/rq-offsets.h
+
+targets += kernel/sched/rq-offsets.s
+
+kernel/sched/rq-offsets.s: $(offsets-file)
+
+$(rq-offsets-file): kernel/sched/rq-offsets.s FORCE
+ $(call filechk,offsets,__RQ_OFFSETS_H__)
+
# Check for missing system calls
quiet_cmd_syscalls = CALL $<
cmd_syscalls = $(CONFIG_SHELL) $< $(CC) $(c_flags) $(missing_syscalls_flags)
PHONY += missing-syscalls
-missing-syscalls: scripts/checksyscalls.sh $(offsets-file)
+missing-syscalls: scripts/checksyscalls.sh $(rq-offsets-file)
$(call cmd,syscalls)
# Check the manual modification of atomic headers
diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index 1fad1c8a4c76..92237c319035 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -424,8 +424,6 @@ static inline void preempt_notifier_init(struct preempt_notifier *notifier,
* work-conserving schedulers.
*
*/
-extern void migrate_disable(void);
-extern void migrate_enable(void);
/**
* preempt_disable_nested - Disable preemption inside a normally preempt disabled section
@@ -471,7 +469,6 @@ static __always_inline void preempt_enable_nested(void)
DEFINE_LOCK_GUARD_0(preempt, preempt_disable(), preempt_enable())
DEFINE_LOCK_GUARD_0(preempt_notrace, preempt_disable_notrace(), preempt_enable_notrace())
-DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
#ifdef CONFIG_PREEMPT_DYNAMIC
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 644a01bdae70..d60ecaccdffc 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -49,6 +49,9 @@
#include <linux/tracepoint-defs.h>
#include <linux/unwind_deferred_types.h>
#include <asm/kmap_size.h>
+#ifndef COMPILE_OFFSETS
+#include <generated/rq-offsets.h>
+#endif
/* task_struct member predeclarations (sorted alphabetically): */
struct audit_context;
@@ -2317,4 +2320,114 @@ static __always_inline void alloc_tag_restore(struct alloc_tag *tag, struct allo
#define alloc_tag_restore(_tag, _old) do {} while (0)
#endif
+#ifndef MODULE
+#ifndef COMPILE_OFFSETS
+
+extern void ___migrate_enable(void);
+
+struct rq;
+DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
+
+/*
+ * The "struct rq" is not available here, so we can't access the
+ * "runqueues" with this_cpu_ptr(), as the compilation will fail in
+ * this_cpu_ptr() -> raw_cpu_ptr() -> __verify_pcpu_ptr():
+ * typeof((ptr) + 0)
+ *
+ * So use arch_raw_cpu_ptr()/PERCPU_PTR() directly here.
+ */
+#ifdef CONFIG_SMP
+#define this_rq_raw() arch_raw_cpu_ptr(&runqueues)
+#else
+#define this_rq_raw() PERCPU_PTR(&runqueues)
+#endif
+#define this_rq_pinned() (*(unsigned int *)((void *)this_rq_raw() + RQ_nr_pinned))
+
+static inline void __migrate_enable(void)
+{
+ struct task_struct *p = current;
+
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ * Check both overflow from migrate_disable() and superfluous
+ * migrate_enable().
+ */
+ if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
+ return;
+#endif
+
+ if (p->migration_disabled > 1) {
+ p->migration_disabled--;
+ return;
+ }
+
+ /*
+ * Ensure stop_task runs either before or after this, and that
+ * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
+ */
+ guard(preempt)();
+ if (unlikely(p->cpus_ptr != &p->cpus_mask))
+ ___migrate_enable();
+ /*
+ * Mustn't clear migration_disabled() until cpus_ptr points back at the
+ * regular cpus_mask, otherwise things that race (eg.
+ * select_fallback_rq) get confused.
+ */
+ barrier();
+ p->migration_disabled = 0;
+ this_rq_pinned()--;
+}
+
+static inline void __migrate_disable(void)
+{
+ struct task_struct *p = current;
+
+ if (p->migration_disabled) {
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ *Warn about overflow half-way through the range.
+ */
+ WARN_ON_ONCE((s16)p->migration_disabled < 0);
+#endif
+ p->migration_disabled++;
+ return;
+ }
+
+ guard(preempt)();
+ this_rq_pinned()++;
+ p->migration_disabled = 1;
+}
+#else /* !COMPILE_OFFSETS */
+static inline void __migrate_disable(void) { }
+static inline void __migrate_enable(void) { }
+#endif /* !COMPILE_OFFSETS */
+
+/*
+ * So that it is possible to not export the runqueues variable, define and
+ * export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
+ * them for the modules. The macro "INSTANTIATE_EXPORTED_MIGRATE_DISABLE" will
+ * be defined in kernel/sched/core.c.
+ */
+#ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
+static inline void migrate_disable(void)
+{
+ __migrate_disable();
+}
+
+static inline void migrate_enable(void)
+{
+ __migrate_enable();
+}
+#else /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
+extern void migrate_disable(void);
+extern void migrate_enable(void);
+#endif /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
+
+#else /* MODULE */
+extern void migrate_disable(void);
+extern void migrate_enable(void);
+#endif /* MODULE */
+
+DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
+
#endif
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c4f69a9e9af6..de9078a9df3a 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -23855,6 +23855,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
BTF_SET_START(btf_id_deny)
BTF_ID_UNUSED
#ifdef CONFIG_SMP
+BTF_ID(func, ___migrate_enable)
BTF_ID(func, migrate_disable)
BTF_ID(func, migrate_enable)
#endif
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index feb750aae71b..7f1e5cb94c53 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7,6 +7,8 @@
* Copyright (C) 1991-2002 Linus Torvalds
* Copyright (C) 1998-2024 Ingo Molnar, Red Hat
*/
+#define INSTANTIATE_EXPORTED_MIGRATE_DISABLE
+#include <linux/sched.h>
#include <linux/highmem.h>
#include <linux/hrtimer_api.h>
#include <linux/ktime_api.h>
@@ -2381,28 +2383,7 @@ static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
__do_set_cpus_allowed(p, &ac);
}
-void migrate_disable(void)
-{
- struct task_struct *p = current;
-
- if (p->migration_disabled) {
-#ifdef CONFIG_DEBUG_PREEMPT
- /*
- *Warn about overflow half-way through the range.
- */
- WARN_ON_ONCE((s16)p->migration_disabled < 0);
-#endif
- p->migration_disabled++;
- return;
- }
-
- guard(preempt)();
- this_rq()->nr_pinned++;
- p->migration_disabled = 1;
-}
-EXPORT_SYMBOL_GPL(migrate_disable);
-
-void migrate_enable(void)
+void ___migrate_enable(void)
{
struct task_struct *p = current;
struct affinity_context ac = {
@@ -2410,35 +2391,19 @@ void migrate_enable(void)
.flags = SCA_MIGRATE_ENABLE,
};
-#ifdef CONFIG_DEBUG_PREEMPT
- /*
- * Check both overflow from migrate_disable() and superfluous
- * migrate_enable().
- */
- if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
- return;
-#endif
+ __set_cpus_allowed_ptr(p, &ac);
+}
+EXPORT_SYMBOL_GPL(___migrate_enable);
- if (p->migration_disabled > 1) {
- p->migration_disabled--;
- return;
- }
+void migrate_disable(void)
+{
+ __migrate_disable();
+}
+EXPORT_SYMBOL_GPL(migrate_disable);
- /*
- * Ensure stop_task runs either before or after this, and that
- * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
- */
- guard(preempt)();
- if (p->cpus_ptr != &p->cpus_mask)
- __set_cpus_allowed_ptr(p, &ac);
- /*
- * Mustn't clear migration_disabled() until cpus_ptr points back at the
- * regular cpus_mask, otherwise things that race (eg.
- * select_fallback_rq) get confused.
- */
- barrier();
- p->migration_disabled = 0;
- this_rq()->nr_pinned--;
+void migrate_enable(void)
+{
+ __migrate_enable();
}
EXPORT_SYMBOL_GPL(migrate_enable);
diff --git a/kernel/sched/rq-offsets.c b/kernel/sched/rq-offsets.c
new file mode 100644
index 000000000000..a23747bbe25b
--- /dev/null
+++ b/kernel/sched/rq-offsets.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+#define COMPILE_OFFSETS
+#include <linux/kbuild.h>
+#include <linux/types.h>
+#include "sched.h"
+
+int main(void)
+{
+ DEFINE(RQ_nr_pinned, offsetof(struct rq, nr_pinned));
+
+ return 0;
+}
]
|
| 19/2 |
2026/11/18 00:50 |
llm |
expert |
0m
Model:
Tokens:
input: 21901
output: 34
thoughts: 3712
Thoughts: **Analyzing a Data Race in Migrate Disable/Enable**
Okay, so it seems we've got a data race report, and I need to figure out how this happened. Commit 378b7708194f moved `migrate_disable()` and `migrate_enable()` to inline functions, and that's likely the key. I'm seeing two stack traces: one indicating a write and the other a read to the same memory location, `p->migration_disabled`. The write is triggered by KCSAN while detecting a data race, which is itself triggered during a read by a different process.
The write happens during an interrupt, specifically within `__migrate_disable()`, which is called by the `bpf_prog_run_pin_on_cpu` function. This write is happening *during* an `udelay()` call performed by KCSAN to detect a race. This interrupt is triggered by some code in the network stack, involving `virtio_net`.
The read is in the main path, within `__migrate_enable()`, again called by `bpf_prog_run_pin_on_cpu`, while the program is using `bpf_flow_dissect` and then `bpf_prog_test_run`. KCSAN has set up a watchpoint on the read of `p->migration_disabled` at line 2457 in `include/linux/sched.h`, and it is reading the value 1.
The critical issue is this: can `migrate_disable()` and `migrate_enable()` be called within an interrupt context? If so, we have a problem.
Let's break down the logic: `__migrate_disable()` increments `p->migration_disabled`, while `__migrate_enable()` decrements it, but only *if* `p->migration_disabled` is greater than 1. The problem arises when an interrupt interrupts a task inside `__migrate_enable()` or `__migrate_disable()`. It seems that even though there's a guard to prevent preemption, it's not enough to prevent interrupts.
If an interrupt occurs *after* the read of `p->migration_disabled` within `__migrate_enable()` but before the guard to disable preemption, the interrupt could increment `p->migration_disabled` via `migrate_disable()`. The original task would then, after the interrupt returns, decrement `p->migration_disabled` to zero and decrement the `this_rq_pinned`. If the original `p->migration_disabled` value was `0`, the interrupt would increase this to `1`. But this looks intended:
* If the value is zero, then we increment the value.
* If the value is larger than zero, then we decrement the value.
The thing is, it looks like `migrate_disable` and `migrate_enable` were *never* intended to be called in atomic/interrupt/softirq context! The problem is `bpf_prog_run_pin_on_cpu` calls `migrate_disable()`, and that means an interrupt can easily occur at the wrong time and trigger a data race. Now I just need to find the commit that introduced this call!
|
| 20/2 |
2026/11/18 00:51 |
tool |
git-log |
10m
Error:
Args: map[CodeRegexp:bpf_prog_run_pin_on_cpu Count:10]
Results:
|
| 21/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 26369
output: 42
thoughts: 12
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience:
**Adding `PathPrefix` to `git-log` - Initial Considerations**
Alright, so the idea pops into my head: let's integrate `PathPrefix` into `git-log`. My initial thought process is pretty straightforward. I'm immediately considering the implications from a functionality perspective, along with the performance and potential impact on existing workflows.
I'm thinking about how this would change the existing command structure. What's the best way to implement it? As an optional argument alongside the existing path arguments, or a dedicated, distinct option? From a usability standpoint, I have a clear vision of how it should behave: it should let me specify a prefix to filter *within* the specified path or file. I'm already imagining the possibilities for navigating complex repositories and drilling down into specific sub-directories with ease. I can see the value immediately.
I'm also considering the edge cases right away. How would this interact with other filtering options, like `--author`, `--grep`, or `--since`? Are there potential conflicts or ambiguities to resolve? I am quickly thinking about the potential pitfalls here, but I think a carefully thought-out implementation will provide an excellent solution. My instinct says we need to think this through very carefully, before we even start coding.
|
| 22/2 |
2026/11/18 01:01 |
tool |
git-log |
0m
Args: map[CodeRegexp:bpf_prog_run_pin_on_cpu PathPrefix:include/linux/filter.h]
Results: map[Output:3d9f773cf287 bpf: Use bpf_prog_run_pin_on_cpu() at simple call sites.
3c58482a382b bpf: Provide bpf_prog_run_pin_on_cpu() helper
]
|
| 23/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 26783
output: 27
thoughts: 34
Thoughts: Okay, here's my take on those commits, framed as my internal thought process, assuming I'm an expert in the field:
**Diving into the Code Changes**
Alright, let's get down to brass tacks. Time to dig into these commits: `3c58482a382b` and `3d9f773cf287`. Immediately, I need to get a clear picture of what's been touched. Are we talking feature implementation, a crucial bug fix, refactoring, or just some minor tweaks? Context is key here, so I'll need to remember the surrounding commits. Based on the naming conventions in the project and the associated issue tracker, I will formulate my expectations before starting.
|
| 24/2 |
2026/11/18 01:01 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 3c58482a382bae89410439247152eb342e9872f7
Author: Thomas Gleixner <tglx@kernel.org>
Date: Mon Feb 24 15:01:41 2020 +0100
bpf: Provide bpf_prog_run_pin_on_cpu() helper
BPF programs require to run on one CPU to completion as they use per CPU
storage, but according to Alexei they don't need reentrancy protection as
obviously BPF programs running in thread context can always be 'preempted'
by hard and soft interrupts and instrumentation and the same program can
run concurrently on a different CPU.
The currently used mechanism to ensure CPUness is to wrap the invocation
into a preempt_disable/enable() pair. Disabling preemption is also
disabling migration for a task.
preempt_disable/enable() is used because there is no explicit way to
reliably disable only migration.
Provide a separate macro to invoke a BPF program which can be used in
migrateable task context.
It wraps BPF_PROG_RUN() in a migrate_disable/enable() pair which maps on
non RT enabled kernels to preempt_disable/enable(). On RT enabled kernels
this merely disables migration. Both methods ensure that the invoked BPF
program runs on one CPU to completion.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200224145643.474592620@linutronix.de
diff --git a/include/linux/filter.h b/include/linux/filter.h
index f349e2c0884c..38f60188bb26 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -576,8 +576,30 @@ DECLARE_STATIC_KEY_FALSE(bpf_stats_enabled_key);
} \
ret; })
-#define BPF_PROG_RUN(prog, ctx) __BPF_PROG_RUN(prog, ctx, \
- bpf_dispatcher_nopfunc)
+#define BPF_PROG_RUN(prog, ctx) \
+ __BPF_PROG_RUN(prog, ctx, bpf_dispatcher_nopfunc)
+
+/*
+ * Use in preemptible and therefore migratable context to make sure that
+ * the execution of the BPF program runs on one CPU.
+ *
+ * This uses migrate_disable/enable() explicitly to document that the
+ * invocation of a BPF program does not require reentrancy protection
+ * against a BPF program which is invoked from a preempting task.
+ *
+ * For non RT enabled kernels migrate_disable/enable() maps to
+ * preempt_disable/enable(), i.e. it disables also preemption.
+ */
+static inline u32 bpf_prog_run_pin_on_cpu(const struct bpf_prog *prog,
+ const void *ctx)
+{
+ u32 ret;
+
+ migrate_disable();
+ ret = __BPF_PROG_RUN(prog, ctx, bpf_dispatcher_nopfunc);
+ migrate_enable();
+ return ret;
+}
#define BPF_SKB_CB_LEN QDISC_CB_PRIV_LEN
]
|
| 25/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 27919
output: 34
thoughts: 250
Thoughts: **Analyzing a Recent Kernel Commit**
Okay, so I'm looking at this commit message: "It wraps BPF_PROG_RUN() in a `migrate_disable`/`enable()` pair which maps on non RT enabled kernels to `preempt_disable`/`enable()`. On RT enabled kernels this merely disables migration. Both methods ensure that the invoked BPF program runs on one CPU to completion." And I see the comment in the code says: "Use in preemptible and therefore migratable context to make sure that the execution of the BPF program runs on one CPU." and "For non RT enabled kernels `migrate_disable`/`enable()` maps to `preempt_disable`/`enable()`, i.e. it disables also preemption."
Wait a second... something is bugging me. I remember seeing this mapping of `migrate_disable`/`enable()` to `preempt_disable`/`enable()` for non-RT kernels mentioned... but then I recalled that `migrate_disable()` was implemented for non-RT kernels in mainline later! I need to see what's actually happening here. I need to dig back further than what I'm seeing now, especially in this context, because I'm not seeing *all* of the code.
I need to see what the state of things were in 2020. I *need* to understand when general `migrate_disable()` was implemented in mainline, and the rules around its use. It's time to fire up git and do a `git-log` on `migrate_disable` within `kernel/sched/core.c`. Let's get to the bottom of this!
|
| 26/2 |
2026/11/18 01:01 |
tool |
git-log |
0m
Args: map[CodeRegexp:void migrate_disable PathPrefix:kernel/sched/core.c]
Results: map[Output:378b7708194f sched: Make migrate_{en,dis}able() inline
cac5cefbade9 sched/smp: Make SMP unconditional
af449901b84c sched: Add migrate_disable()
]
|
| 27/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 28626
output: 26
thoughts: 19
Thoughts: Okay, here's my interpretation of that prompt, and my response:
**Reviewing Commit af449901b84c**
Alright, let's dive into commit `af449901b84c`. It's time to thoroughly examine the changes introduced in this specific commit. I need to understand its impact within the broader context of the project. I'll need to look at its purpose, the changes it introduced, and whether it aligns with the overall architectural goals. I'll be focused on its potential effects. I'll be looking for any potential regressions, performance impacts, or unexpected interactions with existing code. Given my expertise, I should also be able to infer possible refactoring opportunities. It's time to see what this commit is all about.
|
| 28/2 |
2026/11/18 01:01 |
tool |
git-show |
0m
Args:
Results: map[Output:commit af449901b84c98cbd84a0113223ba3bcfcb12a26
Author: Peter Zijlstra <peterz@infradead.org>
Date: Thu Sep 17 10:38:30 2020 +0200
sched: Add migrate_disable()
Add the base migrate_disable() support (under protest).
While migrate_disable() is (currently) required for PREEMPT_RT, it is
also one of the biggest flaws in the system.
Notably this is just the base implementation, it is broken vs
sched_setaffinity() and hotplug, both solved in additional patches for
ease of review.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Link: https://lkml.kernel.org/r/20201023102346.818170844@infradead.org
diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index 7d9c1c0e149c..97ba7c920653 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -322,6 +322,69 @@ static inline void preempt_notifier_init(struct preempt_notifier *notifier,
#endif
+#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+
+/*
+ * Migrate-Disable and why it is (strongly) undesired.
+ *
+ * The premise of the Real-Time schedulers we have on Linux
+ * (SCHED_FIFO/SCHED_DEADLINE) is that M CPUs can/will run M tasks
+ * concurrently, provided there are sufficient runnable tasks, also known as
+ * work-conserving. For instance SCHED_DEADLINE tries to schedule the M
+ * earliest deadline threads, and SCHED_FIFO the M highest priority threads.
+ *
+ * The correctness of various scheduling models depends on this, but is it
+ * broken by migrate_disable() that doesn't imply preempt_disable(). Where
+ * preempt_disable() implies an immediate priority ceiling, preemptible
+ * migrate_disable() allows nesting.
+ *
+ * The worst case is that all tasks preempt one another in a migrate_disable()
+ * region and stack on a single CPU. This then reduces the available bandwidth
+ * to a single CPU. And since Real-Time schedulability theory considers the
+ * Worst-Case only, all Real-Time analysis shall revert to single-CPU
+ * (instantly solving the SMP analysis problem).
+ *
+ *
+ * The reason we have it anyway.
+ *
+ * PREEMPT_RT breaks a number of assumptions traditionally held. By forcing a
+ * number of primitives into becoming preemptible, they would also allow
+ * migration. This turns out to break a bunch of per-cpu usage. To this end,
+ * all these primitives employ migirate_disable() to restore this implicit
+ * assumption.
+ *
+ * This is a 'temporary' work-around at best. The correct solution is getting
+ * rid of the above assumptions and reworking the code to employ explicit
+ * per-cpu locking or short preempt-disable regions.
+ *
+ * The end goal must be to get rid of migrate_disable(), alternatively we need
+ * a schedulability theory that does not depend on abritrary migration.
+ *
+ *
+ * Notes on the implementation.
+ *
+ * The implementation is particularly tricky since existing code patterns
+ * dictate neither migrate_disable() nor migrate_enable() is allowed to block.
+ * This means that it cannot use cpus_read_lock() to serialize against hotplug,
+ * nor can it easily migrate itself into a pending affinity mask change on
+ * migrate_enable().
+ *
+ *
+ * Note: even non-work-conserving schedulers like semi-partitioned depends on
+ * migration, so migrate_disable() is not only a problem for
+ * work-conserving schedulers.
+ *
+ */
+extern void migrate_disable(void);
+extern void migrate_enable(void);
+
+#elif defined(CONFIG_PREEMPT_RT)
+
+static inline void migrate_disable(void) { }
+static inline void migrate_enable(void) { }
+
+#else /* !CONFIG_PREEMPT_RT */
+
/**
* migrate_disable - Prevent migration of the current task
*
@@ -352,4 +415,6 @@ static __always_inline void migrate_enable(void)
preempt_enable();
}
+#endif /* CONFIG_SMP && CONFIG_PREEMPT_RT */
+
#endif /* __LINUX_PREEMPT_H */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 063cd120b459..0732356c0eca 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -714,6 +714,9 @@ struct task_struct {
int nr_cpus_allowed;
const cpumask_t *cpus_ptr;
cpumask_t cpus_mask;
+#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+ int migration_disabled;
+#endif
#ifdef CONFIG_PREEMPT_RCU
int rcu_read_lock_nesting;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 396accb1d69c..6a3f1c2e185b 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1696,6 +1696,61 @@ void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags)
#ifdef CONFIG_SMP
+#ifdef CONFIG_PREEMPT_RT
+
+static void
+__do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask, u32 flags);
+
+static int __set_cpus_allowed_ptr(struct task_struct *p,
+ const struct cpumask *new_mask,
+ u32 flags);
+
+static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
+{
+ if (likely(!p->migration_disabled))
+ return;
+
+ if (p->cpus_ptr != &p->cpus_mask)
+ return;
+
+ /*
+ * Violates locking rules! see comment in __do_set_cpus_allowed().
+ */
+ __do_set_cpus_allowed(p, cpumask_of(rq->cpu), SCA_MIGRATE_DISABLE);
+}
+
+void migrate_disable(void)
+{
+ if (current->migration_disabled++)
+ return;
+
+ barrier();
+}
+EXPORT_SYMBOL_GPL(migrate_disable);
+
+void migrate_enable(void)
+{
+ struct task_struct *p = current;
+
+ if (--p->migration_disabled)
+ return;
+
+ barrier();
+
+ if (p->cpus_ptr == &p->cpus_mask)
+ return;
+
+ __set_cpus_allowed_ptr(p, &p->cpus_mask, SCA_MIGRATE_ENABLE);
+}
+EXPORT_SYMBOL_GPL(migrate_enable);
+
+static inline bool is_migration_disabled(struct task_struct *p)
+{
+ return p->migration_disabled;
+}
+
+#endif
+
/*
* Per-CPU kthreads are allowed to run on !active && online CPUs, see
* __set_cpus_allowed_ptr() and select_fallback_rq().
@@ -1705,7 +1760,7 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu)
if (!cpumask_test_cpu(cpu, p->cpus_ptr))
return false;
- if (is_per_cpu_kthread(p))
+ if (is_per_cpu_kthread(p) || is_migration_disabled(p))
return cpu_online(cpu);
return cpu_active(cpu);
@@ -1826,6 +1881,11 @@ static int migration_cpu_stop(void *data)
*/
void set_cpus_allowed_common(struct task_struct *p, const struct cpumask *new_mask, u32 flags)
{
+ if (flags & (SCA_MIGRATE_ENABLE | SCA_MIGRATE_DISABLE)) {
+ p->cpus_ptr = new_mask;
+ return;
+ }
+
cpumask_copy(&p->cpus_mask, new_mask);
p->nr_cpus_allowed = cpumask_weight(new_mask);
}
@@ -1836,7 +1896,22 @@ __do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask, u32
struct rq *rq = task_rq(p);
bool queued, running;
- lockdep_assert_held(&p->pi_lock);
+ /*
+ * This here violates the locking rules for affinity, since we're only
+ * supposed to change these variables while holding both rq->lock and
+ * p->pi_lock.
+ *
+ * HOWEVER, it magically works, because ttwu() is the only code that
+ * accesses these variables under p->pi_lock and only does so after
+ * smp_cond_load_acquire(&p->on_cpu, !VAL), and we're in __schedule()
+ * before finish_task().
+ *
+ * XXX do further audits, this smells like something putrid.
+ */
+ if (flags & SCA_MIGRATE_DISABLE)
+ SCHED_WARN_ON(!p->on_cpu);
+ else
+ lockdep_assert_held(&p->pi_lock);
queued = task_on_rq_queued(p);
running = task_current(rq, p);
@@ -1887,9 +1962,14 @@ static int __set_cpus_allowed_ptr(struct task_struct *p,
rq = task_rq_lock(p, &rf);
update_rq_clock(rq);
- if (p->flags & PF_KTHREAD) {
+ if (p->flags & PF_KTHREAD || is_migration_disabled(p)) {
/*
- * Kernel threads are allowed on online && !active CPUs
+ * Kernel threads are allowed on online && !active CPUs.
+ *
+ * Specifically, migration_disabled() tasks must not fail the
+ * cpumask_any_and_distribute() pick below, esp. so on
+ * SCA_MIGRATE_ENABLE, otherwise we'll not call
+ * set_cpus_allowed_common() and actually reset p->cpus_ptr.
*/
cpu_valid_mask = cpu_online_mask;
}
@@ -1903,7 +1983,7 @@ static int __set_cpus_allowed_ptr(struct task_struct *p,
goto out;
}
- if (cpumask_equal(&p->cpus_mask, new_mask))
+ if (!(flags & SCA_MIGRATE_ENABLE) && cpumask_equal(&p->cpus_mask, new_mask))
goto out;
/*
@@ -1995,6 +2075,8 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
* Clearly, migrating tasks to offline CPUs is a fairly daft thing.
*/
WARN_ON_ONCE(!cpu_online(new_cpu));
+
+ WARN_ON_ONCE(is_migration_disabled(p));
#endif
trace_sched_migrate_task(p, new_cpu);
@@ -2325,6 +2407,12 @@ static int select_fallback_rq(int cpu, struct task_struct *p)
}
fallthrough;
case possible:
+ /*
+ * XXX When called from select_task_rq() we only
+ * hold p->pi_lock and again violate locking order.
+ *
+ * More yuck to audit.
+ */
do_set_cpus_allowed(p, cpu_possible_mask);
state = fail;
break;
@@ -2359,7 +2447,7 @@ int select_task_rq(struct task_struct *p, int cpu, int sd_flags, int wake_flags)
{
lockdep_assert_held(&p->pi_lock);
- if (p->nr_cpus_allowed > 1)
+ if (p->nr_cpus_allowed > 1 && !is_migration_disabled(p))
cpu = p->sched_class->select_task_rq(p, cpu, sd_flags, wake_flags);
else
cpu = cpumask_any(p->cpus_ptr);
@@ -2421,6 +2509,17 @@ static inline int __set_cpus_allowed_ptr(struct task_struct *p,
#endif /* CONFIG_SMP */
+#if !defined(CONFIG_SMP) || !defined(CONFIG_PREEMPT_RT)
+
+static inline void migrate_disable_switch(struct rq *rq, struct task_struct *p) { }
+
+static inline bool is_migration_disabled(struct task_struct *p)
+{
+ return false;
+}
+
+#endif
+
static void
ttwu_stat(struct task_struct *p, int cpu, int wake_flags)
{
@@ -4570,6 +4669,7 @@ static void __sched notrace __schedule(bool preempt)
*/
++*switch_count;
+ migrate_disable_switch(rq, prev);
psi_sched_switch(prev, next, !task_on_rq_queued(prev));
trace_sched_switch(preempt, prev, next);
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 0420d80fb250..72d8e47cf0bb 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1902,14 +1902,16 @@ static inline bool sched_fair_runnable(struct rq *rq)
extern struct task_struct *pick_next_task_fair(struct rq *rq, struct task_struct *prev, struct rq_flags *rf);
extern struct task_struct *pick_next_task_idle(struct rq *rq);
+#define SCA_CHECK 0x01
+#define SCA_MIGRATE_DISABLE 0x02
+#define SCA_MIGRATE_ENABLE 0x04
+
#ifdef CONFIG_SMP
extern void update_group_capacity(struct sched_domain *sd, int cpu);
extern void trigger_load_balance(struct rq *rq);
-#define SCA_CHECK 0x01
-
extern void set_cpus_allowed_common(struct task_struct *p, const struct cpumask *new_mask, u32 flags);
#endif
diff --git a/lib/smp_processor_id.c b/lib/smp_processor_id.c
index 525222e4f409..faaa927ac2c8 100644
--- a/lib/smp_processor_id.c
+++ b/lib/smp_processor_id.c
@@ -26,6 +26,11 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)
if (current->nr_cpus_allowed == 1)
goto out;
+#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+ if (current->migration_disabled)
+ goto out;
+#endif
+
/*
* It is valid to assume CPU-locality during early bootup:
*/
]
|
| 29/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 33045
output: 37
thoughts: 73
Thoughts: Here's my summary, as I sift through this code:
**Analyzing the Evolution of `migrate_disable()`**
Okay, so I'm trying to understand the history of `migrate_disable()`. I seem to recall that way back in commit `af449901b84c`, it was exclusively tied to `CONFIG_PREEMPT_RT`. Right, that's what sticks in my mind.
Now, the question is: when did this function become more generally accessible and enabled outside of that real-time context? I need to get a clear timeline here, to see what changes were made that broadened its scope.
My next move is obvious: I'll fire up `git log` and zero in on `kernel/sched/core.c`, specifically looking for occurrences of `migrate_disable()`. That should give me the commit history of the function and help me to see when the change occurred. That will be the key to understanding this.
|
| 30/2 |
2026/11/18 01:01 |
tool |
git-log |
0m
Args: map[CodeRegexp:migration_disabled Count:5 PathPrefix:kernel/sched/core.c]
Results: map[Output:74699f56ebcf sched/core: Push current task from non preferred CPU
fe3c73d7bc76 sched/core: Avoid false migration warning for proxy donors
378b7708194f sched: Make migrate_{en,dis}able() inline
f207dc2dcdcf sched/core: Add ENQUEUE_RQ_SELECTED to indicate whether ->select_task_rq() was called
0ec8d5aed4d1 sched/core: Add WARN_ON_ONCE() to check overflow for migrate_disable()
]
|
| 31/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 33507
output: 34
thoughts: 22
Thoughts: Okay, here's my attempt at summarizing that thought process, tailored for an expert audience, and written in the first person:
**Investigating the `CONFIG_PREEMPT_RT` Removal**
Alright, so I'm thinking about a specific scenario. I need to track down the exact commit or series of commits where the check for `CONFIG_PREEMPT_RT` was removed, specifically in relation to `migration_disabled`. My intuition tells me that knowing when that removal happened will provide valuable context and help me understand the rationale behind the change. I need to understand if this was a refactoring effort, a bug fix, or a strategic decision related to real-time performance or synchronization constraints. I will start by searching the kernel's Git history around the `migration_disabled` variable. Looking for the removal of the specific check against `CONFIG_PREEMPT_RT` will be my primary focus for this initial query. That's the most effective way to start drilling down.
|
| 32/2 |
2026/11/18 01:01 |
tool |
git-log |
0m
Args: map[CodeRegexp:migration_disabled; PathPrefix:include/linux/sched.h]
Results: map[Output:a7c81556ec4d sched: Fix migrate_disable() vs rt/dl balancing
af449901b84c sched: Add migrate_disable()
]
|
| 33/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 33823
output: 26
thoughts: 18
Thoughts: Okay, here's my interpretation of what's going through my mind when I approach that commit, rendered in the first person:
**Diving into Commit a7c81556ec4d**
Alright, let's pull up that commit: `a7c81556ec4d`. Immediately, I need to get a sense of the scope. What was the motivation behind this particular change? What problem were they trying to solve, or what feature were they trying to add? Is there a Jira ticket or pull request associated that I can look at? That's always my first step β contextualize the work within the broader project goals.
Next, I'll probably scan the diff. I'm looking for the *style* of the change. Is the code clean and well-structured, or is it a bit messy? Is it consistent with the existing codebase? I'm paying attention to things like indentation, naming conventions, and overall readability. Even before I understand *what* the code does, I can make some judgements about its maintainability and quality. That includes the number of changed lines - sometimes a low number of changed lines can suggest a focused solution.
Now I'm diving deeper into the specifics. I'll focus on the primary functions or classes modified. What's the logic here? Are there any obvious bugs or vulnerabilities? Are there any potential performance bottlenecks? I'll probably try to mentally execute the code, walking through the key paths and thinking about edge cases. I'm keeping an eye out for anything that might break existing functionality or introduce unexpected side effects. Testing is key, so if there's no testing in the commit, I'll ask about it.
I'm also looking for opportunities to improve the code. Even if the commit does what it's supposed to do, could it be more efficient? More elegant? More maintainable? I'm not afraid to make suggestions β the goal is always to create the best possible code, and a critical eye is essential.
|
| 34/2 |
2026/11/18 01:01 |
tool |
git-show |
0m
Args:
Results: map[Output:commit a7c81556ec4d341dfdbf2cc478ead89d73e474a7
Author: Peter Zijlstra <peterz@infradead.org>
Date: Mon Sep 28 17:06:07 2020 +0200
sched: Fix migrate_disable() vs rt/dl balancing
In order to minimize the interference of migrate_disable() on lower
priority tasks, which can be deprived of runtime due to being stuck
below a higher priority task. Teach the RT/DL balancers to push away
these higher priority tasks when a lower priority task gets selected
to run on a freshly demoted CPU (pull).
This adds migration interference to the higher priority task, but
restores bandwidth to system that would otherwise be irrevocably lost.
Without this it would be possible to have all tasks on the system
stuck on a single CPU, each task preempted in a migrate_disable()
section with a single high priority task running.
This way we can still approximate running the M highest priority tasks
on the system.
Migrating the top task away is (ofcourse) still subject to
migrate_disable() too, which means the lower task is subject to an
interference equivalent to the worst case migrate_disable() section.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Link: https://lkml.kernel.org/r/20201023102347.499155098@infradead.org
diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index 97ba7c920653..8b43922e65df 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -325,24 +325,28 @@ static inline void preempt_notifier_init(struct preempt_notifier *notifier,
#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
/*
- * Migrate-Disable and why it is (strongly) undesired.
- *
- * The premise of the Real-Time schedulers we have on Linux
- * (SCHED_FIFO/SCHED_DEADLINE) is that M CPUs can/will run M tasks
- * concurrently, provided there are sufficient runnable tasks, also known as
- * work-conserving. For instance SCHED_DEADLINE tries to schedule the M
- * earliest deadline threads, and SCHED_FIFO the M highest priority threads.
- *
- * The correctness of various scheduling models depends on this, but is it
- * broken by migrate_disable() that doesn't imply preempt_disable(). Where
- * preempt_disable() implies an immediate priority ceiling, preemptible
- * migrate_disable() allows nesting.
- *
- * The worst case is that all tasks preempt one another in a migrate_disable()
- * region and stack on a single CPU. This then reduces the available bandwidth
- * to a single CPU. And since Real-Time schedulability theory considers the
- * Worst-Case only, all Real-Time analysis shall revert to single-CPU
- * (instantly solving the SMP analysis problem).
+ * Migrate-Disable and why it is undesired.
+ *
+ * When a preempted task becomes elegible to run under the ideal model (IOW it
+ * becomes one of the M highest priority tasks), it might still have to wait
+ * for the preemptee's migrate_disable() section to complete. Thereby suffering
+ * a reduction in bandwidth in the exact duration of the migrate_disable()
+ * section.
+ *
+ * Per this argument, the change from preempt_disable() to migrate_disable()
+ * gets us:
+ *
+ * - a higher priority tasks gains reduced wake-up latency; with preempt_disable()
+ * it would have had to wait for the lower priority task.
+ *
+ * - a lower priority tasks; which under preempt_disable() could've instantly
+ * migrated away when another CPU becomes available, is now constrained
+ * by the ability to push the higher priority task away, which might itself be
+ * in a migrate_disable() section, reducing it's available bandwidth.
+ *
+ * IOW it trades latency / moves the interference term, but it stays in the
+ * system, and as long as it remains unbounded, the system is not fully
+ * deterministic.
*
*
* The reason we have it anyway.
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 90a0c92741d7..3af9d52fe093 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -716,8 +716,9 @@ struct task_struct {
cpumask_t cpus_mask;
void *migration_pending;
#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
- int migration_disabled;
+ unsigned short migration_disabled;
#endif
+ unsigned short migration_flags;
#ifdef CONFIG_PREEMPT_RCU
int rcu_read_lock_nesting;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 9ce2fc7d3d51..e92d7853057c 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1763,11 +1763,6 @@ void migrate_enable(void)
}
EXPORT_SYMBOL_GPL(migrate_enable);
-static inline bool is_migration_disabled(struct task_struct *p)
-{
- return p->migration_disabled;
-}
-
static inline bool rq_has_pinned_tasks(struct rq *rq)
{
return rq->nr_pinned;
@@ -1972,6 +1967,49 @@ static int migration_cpu_stop(void *data)
return 0;
}
+int push_cpu_stop(void *arg)
+{
+ struct rq *lowest_rq = NULL, *rq = this_rq();
+ struct task_struct *p = arg;
+
+ raw_spin_lock_irq(&p->pi_lock);
+ raw_spin_lock(&rq->lock);
+
+ if (task_rq(p) != rq)
+ goto out_unlock;
+
+ if (is_migration_disabled(p)) {
+ p->migration_flags |= MDF_PUSH;
+ goto out_unlock;
+ }
+
+ p->migration_flags &= ~MDF_PUSH;
+
+ if (p->sched_class->find_lock_rq)
+ lowest_rq = p->sched_class->find_lock_rq(p, rq);
+
+ if (!lowest_rq)
+ goto out_unlock;
+
+ // XXX validate p is still the highest prio task
+ if (task_rq(p) == rq) {
+ deactivate_task(rq, p, 0);
+ set_task_cpu(p, lowest_rq->cpu);
+ activate_task(lowest_rq, p, 0);
+ resched_curr(lowest_rq);
+ }
+
+ double_unlock_balance(rq, lowest_rq);
+
+out_unlock:
+ rq->push_busy = false;
+ raw_spin_unlock(&rq->lock);
+ raw_spin_unlock_irq(&p->pi_lock);
+
+ put_task_struct(p);
+ return 0;
+}
+
/*
* sched_class::set_cpus_allowed must do the below, but is not required to
* actually call this function.
@@ -2052,6 +2090,14 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag
/* Can the task run on the task's current CPU? If so, we're done */
if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) {
+ struct task_struct *push_task = NULL;
+
+ if ((flags & SCA_MIGRATE_ENABLE) &&
+ (p->migration_flags & MDF_PUSH) && !rq->push_busy) {
+ rq->push_busy = true;
+ push_task = get_task_struct(p);
+ }
+
pending = p->migration_pending;
if (pending) {
refcount_inc(&pending->refs);
@@ -2060,6 +2106,11 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag
}
task_rq_unlock(rq, p, rf);
+ if (push_task) {
+ stop_one_cpu_nowait(rq->cpu, push_cpu_stop,
+ p, &rq->push_work);
+ }
+
if (complete)
goto do_complete;
@@ -2098,6 +2149,7 @@ static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flag
if (flags & SCA_MIGRATE_ENABLE) {
refcount_inc(&pending->refs); /* pending->{arg,stop_work} */
+ p->migration_flags &= ~MDF_PUSH;
task_rq_unlock(rq, p, rf);
pending->arg = (struct migration_arg) {
@@ -2716,11 +2768,6 @@ static inline int __set_cpus_allowed_ptr(struct task_struct *p,
static inline void migrate_disable_switch(struct rq *rq, struct task_struct *p) { }
-static inline bool is_migration_disabled(struct task_struct *p)
-{
- return false;
-}
-
static inline bool rq_has_pinned_tasks(struct rq *rq)
{
return false;
diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c
index 3d3fd8370447..eed2e449b313 100644
--- a/kernel/sched/deadline.c
+++ b/kernel/sched/deadline.c
@@ -2129,6 +2129,9 @@ static int push_dl_task(struct rq *rq)
return 0;
retry:
+ if (is_migration_disabled(next_task))
+ return 0;
+
if (WARN_ON(next_task == rq->curr))
return 0;
@@ -2206,7 +2209,7 @@ static void push_dl_tasks(struct rq *rq)
static void pull_dl_task(struct rq *this_rq)
{
int this_cpu = this_rq->cpu, cpu;
- struct task_struct *p;
+ struct task_struct *p, *push_task;
bool resched = false;
struct rq *src_rq;
u64 dmin = LONG_MAX;
@@ -2236,6 +2239,7 @@ static void pull_dl_task(struct rq *this_rq)
continue;
/* Might drop this_rq->lock */
+ push_task = NULL;
double_lock_balance(this_rq, src_rq);
/*
@@ -2267,17 +2271,27 @@ static void pull_dl_task(struct rq *this_rq)
src_rq->curr->dl.deadline))
goto skip;
- resched = true;
-
- deactivate_task(src_rq, p, 0);
- set_task_cpu(p, this_cpu);
- activate_task(this_rq, p, 0);
- dmin = p->dl.deadline;
+ if (is_migration_disabled(p)) {
+ push_task = get_push_task(src_rq);
+ } else {
+ deactivate_task(src_rq, p, 0);
+ set_task_cpu(p, this_cpu);
+ activate_task(this_rq, p, 0);
+ dmin = p->dl.deadline;
+ resched = true;
+ }
/* Is there any other task even earlier? */
}
skip:
double_unlock_balance(this_rq, src_rq);
+
+ if (push_task) {
+ raw_spin_unlock(&this_rq->lock);
+ stop_one_cpu_nowait(src_rq->cpu, push_cpu_stop,
+ push_task, &src_rq->push_work);
+ raw_spin_lock(&this_rq->lock);
+ }
}
if (resched)
@@ -2524,6 +2538,7 @@ const struct sched_class dl_sched_class
.rq_online = rq_online_dl,
.rq_offline = rq_offline_dl,
.task_woken = task_woken_dl,
+ .find_lock_rq = find_lock_later_rq,
#endif
.task_tick = task_tick_dl,
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index cf63346a07e4..c592e47cafed 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -1859,7 +1859,7 @@ static struct task_struct *pick_next_pushable_task(struct rq *rq)
* running task can migrate over to a CPU that is running a task
* of lesser priority.
*/
-static int push_rt_task(struct rq *rq)
+static int push_rt_task(struct rq *rq, bool pull)
{
struct task_struct *next_task;
struct rq *lowest_rq;
@@ -1873,6 +1873,34 @@ static int push_rt_task(struct rq *rq)
return 0;
retry:
+ if (is_migration_disabled(next_task)) {
+ struct task_struct *push_task = NULL;
+ int cpu;
+
+ if (!pull || rq->push_busy)
+ return 0;
+
+ cpu = find_lowest_rq(rq->curr);
+ if (cpu == -1 || cpu == rq->cpu)
+ return 0;
+
+ /*
+ * Given we found a CPU with lower priority than @next_task,
+ * therefore it should be running. However we cannot migrate it
+ * to this other CPU, instead attempt to push the current
+ * running task on this CPU away.
+ */
+ push_task = get_push_task(rq);
+ if (push_task) {
+ raw_spin_unlock(&rq->lock);
+ stop_one_cpu_nowait(rq->cpu, push_cpu_stop,
+ push_task, &rq->push_work);
+ raw_spin_lock(&rq->lock);
+ }
+
+ return 0;
+ }
+
if (WARN_ON(next_task == rq->curr))
return 0;
@@ -1927,12 +1955,10 @@ static int push_rt_task(struct rq *rq)
deactivate_task(rq, next_task, 0);
set_task_cpu(next_task, lowest_rq->cpu);
activate_task(lowest_rq, next_task, 0);
- ret = 1;
-
resched_curr(lowest_rq);
+ ret = 1;
double_unlock_balance(rq, lowest_rq);
-
out:
put_task_struct(next_task);
@@ -1942,7 +1968,7 @@ static int push_rt_task(struct rq *rq)
static void push_rt_tasks(struct rq *rq)
{
/* push_rt_task will return true if it moved an RT */
- while (push_rt_task(rq))
+ while (push_rt_task(rq, false))
;
}
@@ -2095,7 +2121,8 @@ void rto_push_irq_work_func(struct irq_work *work)
*/
if (has_pushable_tasks(rq)) {
raw_spin_lock(&rq->lock);
- push_rt_tasks(rq);
+ while (push_rt_task(rq, true))
+ ;
raw_spin_unlock(&rq->lock);
}
@@ -2120,7 +2147,7 @@ static void pull_rt_task(struct rq *this_rq)
{
int this_cpu = this_rq->cpu, cpu;
bool resched = false;
- struct task_struct *p;
+ struct task_struct *p, *push_task;
struct rq *src_rq;
int rt_overload_count = rt_overloaded(this_rq);
@@ -2167,6 +2194,7 @@ static void pull_rt_task(struct rq *this_rq)
* double_lock_balance, and another CPU could
* alter this_rq
*/
+ push_task = NULL;
double_lock_balance(this_rq, src_rq);
/*
@@ -2194,11 +2222,14 @@ static void pull_rt_task(struct rq *this_rq)
if (p->prio < src_rq->curr->prio)
goto skip;
- resched = true;
-
- deactivate_task(src_rq, p, 0);
- set_task_cpu(p, this_cpu);
- activate_task(this_rq, p, 0);
+ if (is_migration_disabled(p)) {
+ push_task = get_push_task(src_rq);
+ } else {
+ deactivate_task(src_rq, p, 0);
+ set_task_cpu(p, this_cpu);
+ activate_task(this_rq, p, 0);
+ resched = true;
+ }
/*
* We continue with the search, just in
* case there's an even higher prio task
@@ -2208,6 +2239,13 @@ static void pull_rt_task(struct rq *this_rq)
}
skip:
double_unlock_balance(this_rq, src_rq);
+
+ if (push_task) {
+ raw_spin_unlock(&this_rq->lock);
+ stop_one_cpu_nowait(src_rq->cpu, push_cpu_stop,
+ push_task, &src_rq->push_work);
+ raw_spin_lock(&this_rq->lock);
+ }
}
if (resched)
@@ -2449,6 +2487,7 @@ const struct sched_class rt_sched_class
.rq_offline = rq_offline_rt,
.task_woken = task_woken_rt,
.switched_from = switched_from_rt,
+ .find_lock_rq = find_lock_lowest_rq,
#endif
.task_tick = task_tick_rt,
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 42de1406c0dc..56992aaca48e 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1057,6 +1057,8 @@ struct rq {
#if defined(CONFIG_PREEMPT_RT) && defined(CONFIG_SMP)
unsigned int nr_pinned;
#endif
+ unsigned int push_busy;
+ struct cpu_stop_work push_work;
};
#ifdef CONFIG_FAIR_GROUP_SCHED
@@ -1084,6 +1086,16 @@ static inline int cpu_of(struct rq *rq)
#endif
}
+#define MDF_PUSH 0x01
+
+static inline bool is_migration_disabled(struct task_struct *p)
+{
+#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+ return p->migration_disabled;
+#else
+ return false;
+#endif
+}
#ifdef CONFIG_SCHED_SMT
extern void __update_idle_core(struct rq *rq);
@@ -1823,6 +1835,8 @@ struct sched_class {
void (*rq_online)(struct rq *rq);
void (*rq_offline)(struct rq *rq);
+
+ struct rq *(*find_lock_rq)(struct task_struct *p, struct rq *rq);
#endif
void (*task_tick)(struct rq *rq, struct task_struct *p, int queued);
@@ -1918,6 +1932,24 @@ extern void trigger_load_balance(struct rq *rq);
extern void set_cpus_allowed_common(struct task_struct *p, const struct cpumask *new_mask, u32 flags);
+static inline struct task_struct *get_push_task(struct rq *rq)
+{
+ struct task_struct *p = rq->curr;
+
+ lockdep_assert_held(&rq->lock);
+
+ if (rq->push_busy)
+ return NULL;
+
+ if (p->nr_cpus_allowed == 1)
+ return NULL;
+
+ rq->push_busy = true;
+ return get_task_struct(p);
+}
+
+extern int push_cpu_stop(void *arg);
+
#endif
#ifdef CONFIG_CPU_IDLE
]
|
| 35/2 |
2026/11/18 01:01 |
llm |
expert |
0m
Model:
Tokens:
input: 40009
output: 33
thoughts: 47
Thoughts: Okay, let's break this down.
**Investigating the `migration_disabled` and `CONFIG_PREEMPT_RT` Interaction**
Alright, I need to understand why `CONFIG_PREEMPT_RT` might have been dropped in relation to `migration_disabled`. My instinct is to trace the history, specifically to see the commit messages and any related code changes. I'll need to use the `git log` command to search the commit history. The `-S` flag is perfect for searching for the string "migration_disabled". But I need to narrow the search down to the relevant header file. I know that `migration_disabled` is likely defined or used in the scheduler code, so I'll target `include/linux/sched.h`. This should give me a focused history of changes where "migration_disabled" appears, which will hopefully illuminate the relationship with `CONFIG_PREEMPT_RT`. Let's see what that yields.
|
| 36/2 |
2026/11/18 01:02 |
tool |
git-log |
0m
Args: map[CodeRegexp:migration_disabled PathPrefix:include/linux/sched.h]
Results: map[Output:378b7708194f sched: Make migrate_{en,dis}able() inline
a7c81556ec4d sched: Fix migrate_disable() vs rt/dl balancing
af449901b84c sched: Add migrate_disable()
]
|
| 37/2 |
2026/11/18 01:02 |
llm |
expert |
0m
Model:
Tokens:
input: 40370
output: 31
thoughts: 52
Thoughts: Here's my thought process, as requested:
**Checking the Migration Disable Flag in `sched.h`**
Okay, so I need to quickly refresh my memory on how `migration_disabled` is actually defined within the `task_struct`. Time to look it up in `include/linux/sched.h`. Specifically, I need to pinpoint the exact line numbers to ensure I'm up-to-date with any potential changes or recent modifications in the kernel source. Let's get those line numbers and see what we have.
|
| 38/2 |
2026/11/18 01:02 |
tool |
grepper |
0m
Args: map[Expression:migration_disabled PathPrefixes:[include/linux/sched.h]]
Results: map[Output:include/linux/sched.h=836=struct task_struct {
--
include/linux/sched.h-947- void *migration_pending;
include/linux/sched.h:948: unsigned short migration_disabled;
include/linux/sched.h-949- unsigned short migration_flags;
--
include/linux/sched.h=2444=static inline void __migrate_enable(void)
--
include/linux/sched.h-2452- */
include/linux/sched.h:2453: if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
include/linux/sched.h-2454- return;
--
include/linux/sched.h-2456-
include/linux/sched.h:2457: if (p->migration_disabled > 1) {
include/linux/sched.h:2458: p->migration_disabled--;
include/linux/sched.h-2459- return;
--
include/linux/sched.h-2469- /*
include/linux/sched.h:2470: * Mustn't clear migration_disabled() until cpus_ptr points back at the
include/linux/sched.h-2471- * regular cpus_mask, otherwise things that race (eg.
--
include/linux/sched.h-2474- barrier();
include/linux/sched.h:2475: p->migration_disabled = 0;
include/linux/sched.h-2476- this_rq_pinned()--;
--
include/linux/sched.h=2479=static inline void __migrate_disable(void)
--
include/linux/sched.h-2482-
include/linux/sched.h:2483: if (p->migration_disabled) {
include/linux/sched.h-2484-#ifdef CONFIG_DEBUG_PREEMPT
--
include/linux/sched.h-2487- */
include/linux/sched.h:2488: WARN_ON_ONCE((s16)p->migration_disabled < 0);
include/linux/sched.h-2489-#endif
include/linux/sched.h:2490: p->migration_disabled++;
include/linux/sched.h-2491- return;
--
include/linux/sched.h-2495- this_rq_pinned()++;
include/linux/sched.h:2496: p->migration_disabled = 1;
include/linux/sched.h-2497-}
]
|
| 39/2 |
2026/11/18 01:02 |
llm |
expert |
0m
Model:
Tokens:
input: 41241
output: 39
thoughts: 781
Thoughts: **Analyzing a Race Condition in the Kernel's Migration Disable/Enable Logic**
Alright, let's dive into this. First, I see that `migration_disabled` is now a core part of the `task_struct`, which is expected. I also see the functions `__migrate_disable` and `__migrate_enable` handling its manipulation. My initial question is whether `migrate_disable()` and `migrate_enable()` can be called from interrupt context.
Then, there's that KCSAN report: a write to `p->migration_disabled` by an interrupt on CPU 0. Immediately, my mental alarms go off. Why is this happening? I know KCSAN uses `other_info` to track if we're in an interrupt or not. Here it's indicating a softirq (`handle_softirqs`). The next question is, why was a softirq running on CPU 0 in this scenario?
Ah, it's task 5152 running `bpf_flow_dissect` on CPU 0 that kicked off `migrate_enable()`. This is very interesting. Let's look at `__migrate_enable` itself:
The immediate question: Is preemption disabled *before* the check on line 2457? No. Absolutely not! Preemption *isn't* disabled and neither are interrupts.
This is where the problem lies. The preemption is guarded *after* the initial `p->migration_disabled > 1` check. Therefore, Task 5152, which called `migrate_enable()` can get preempted and an interrupt could happen and modify `migration_disabled`!
Let me see how we got here:
bpf_prog_run_pin_on_cpu include/linux/filter.h:838 [inline]
bpf_flow_dissect+0x162/0x360 net/core/flow_dissector.c:1024
bpf_prog_test_run_flow_dissector+0x25d/0x360 net/bpf/test_run.c:1565
bpf_prog_test_run+0x21b/0x360 kernel/bpf/syscall.c:4860
And if I check `net/core/flow_dissector.c`, the call to `__migrate_enable` in `bpf_flow_dissect` isn't protected by a preemption or interrupt disablement.
So, the root cause is preemption can occur between decrementing the ref count and the final preemption guard.
This race can be exploited, and needs to be addressed.
|
| 40/2 |
2026/11/18 01:02 |
tool |
read-file |
0m
Args: map[File:net/core/flow_dissector.c FirstLine:1010 LineCount:30]
Results: map[Contents:1010: /* Pass parameters to the BPF program */
1011: memset(flow_keys, 0, sizeof(*flow_keys));
1012: flow_keys->n_proto = proto;
1013: flow_keys->nhoff = nhoff;
1014: flow_keys->thoff = flow_keys->nhoff;
1015:
1016: BUILD_BUG_ON((int)BPF_FLOW_DISSECTOR_F_PARSE_1ST_FRAG !=
1017: (int)FLOW_DISSECTOR_F_PARSE_1ST_FRAG);
1018: BUILD_BUG_ON((int)BPF_FLOW_DISSECTOR_F_STOP_AT_FLOW_LABEL !=
1019: (int)FLOW_DISSECTOR_F_STOP_AT_FLOW_LABEL);
1020: BUILD_BUG_ON((int)BPF_FLOW_DISSECTOR_F_STOP_AT_ENCAP !=
1021: (int)FLOW_DISSECTOR_F_STOP_AT_ENCAP);
1022: flow_keys->flags = flags;
1023:
1024: result = bpf_prog_run_pin_on_cpu(prog, ctx);
1025:
1026: /* bpf_flow_keys offsets are u16: do not let @hlen be truncated. */
1027: hlen = min_t(int, hlen, U16_MAX);
1028: flow_keys->nhoff = clamp_t(u16, flow_keys->nhoff, nhoff, hlen);
1029: flow_keys->thoff = clamp_t(u16, flow_keys->thoff,
1030: flow_keys->nhoff, hlen);
1031:
1032: return result;
1033: }
1034:
1035: static bool is_pppoe_ses_hdr_valid(const struct pppoe_hdr *hdr)
1036: {
1037: return hdr->ver == 1 && hdr->type == 1 && hdr->code == 0;
1038: }
1039:
]
|
| 41/2 |
2026/11/18 01:02 |
llm |
expert |
0m
Model:
Tokens:
input: 43249
output: 35
thoughts: 824
Thoughts: **Investigating Potential Context Issues in BPF Packet Filtering**
Okay, so I'm digging into this `bpf_prog_run_pin_on_cpu` function and something's not quite right. It seems to be using `migrate_disable()` and `migrate_enable()`, which is a red flag. Let's see, it's called from `bpf_prog_run_clear_cb`, which is called within `run_filter` in `af_packet.c`. This `run_filter` function is *critical*, right? It's where the BPF filter actually gets executed on incoming packets.
And, of course, `run_filter` is called in `packet_rcv`, which happens in a softirq context! Specifically, in the network receive path, after `net_rx_action` is called in the NAPI poll loop, so `invoke_softirq()` is called in `irq_exit()`. Softirqs have `local_bh_disable()` but *not* full preemption or interrupt disable. They can run concurrently, which is important. The question is, is `migrate_disable()` even valid in this softirq/interrupt context? That's the crux of it.
I need to quickly search the git log, or maybe lore or even commit messages, for any usage or discussion of `migrate_disable()` and interrupts. This feels like it could be a major issue.
Oh, wait, let me look at `bpf_prog_run_pin_on_cpu` again. There's a comment in there! "*Use in preemptible and therefore migratable context to make sure that the execution of the BPF program runs on one CPU.*" The comment explicitly mentions that it should be in a preemptible context!
The problem is, packet receive *isn't* preemptible! It's invoked from softirq. That means `migrate_disable` and `migrate_enable` should not be used in this context. This is a very serious issue, potentially a concurrency bug that could lead to undefined behavior or even data corruption. The packet receive path is *not* "preemptible" in the way they intended.
Now I need to investigate the history of `bpf_prog_run_clear_cb` itself and see when this functionality was added and if this was understood then.
|
| 42/2 |
2026/11/18 01:03 |
tool |
git-log |
1m
Args: map[CodeRegexp:bpf_prog_run_clear_cb PathPrefix:net/]
Results: map[Output:ff936a04e5f2 bpf: fix cb access in socket filter programs
]
|
| 43/2 |
2026/11/18 01:04 |
llm |
expert |
0m
Model:
Tokens:
input: 44650
output: 26
thoughts: 19
Thoughts: Okay, here's my attempt at summarizing the thought process, tailored for an expert user and in the first person:
**Diving into Revision ff936a04e5f2**
Alright, time to dig in. The task is clear: I need to take a look at commit `ff936a04e5f2`. This likely represents a specific point in the project's history that requires my attention, perhaps for debugging, code review, understanding a feature introduction, or assessing an implementation's impact. The hex code alone tells me it's a specific, identifiable checkpoint. My mental checklist kicks in: I'll immediately consider the context of this commit. What branch is it on? What are the surrounding commits in the history? What was the general direction of the work at that time? Understanding the "why" behind this specific revision will greatly inform my analysis. I'll be looking for signs that would be an expected occurrence for this change. I'm ready to start looking at the details.
|
| 44/2 |
2026/11/18 01:04 |
tool |
git-show |
0m
Args:
Results: map[Output:commit ff936a04e5f28b7e0455be0e7fa91334f89e4b44
Author: Alexei Starovoitov <ast@kernel.org>
Date: Wed Oct 7 10:55:41 2015 -0700
bpf: fix cb access in socket filter programs
eBPF socket filter programs may see junk in 'u32 cb[5]' area,
since it could have been used by protocol layers earlier.
For socket filter programs used in af_packet we need to clean
20 bytes of skb->cb area if it could be used by the program.
For programs attached to TCP/UDP sockets we need to save/restore
these 20 bytes, since it's used by protocol layers.
Remove SK_RUN_FILTER macro, since it's no longer used.
Long term we may move this bpf cb area to per-cpu scratch, but that
requires addition of new 'per-cpu load/store' instructions,
so not suitable as a short term fix.
Fixes: d691f9e8d440 ("bpf: allow programs to write to certain skb fields")
Reported-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Alexei Starovoitov <ast@plumgrid.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 3697ad563899..b4fdee6cb686 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -100,6 +100,8 @@ enum bpf_access_type {
BPF_WRITE = 2
};
+struct bpf_prog;
+
struct bpf_verifier_ops {
/* return eBPF function prototype for verification */
const struct bpf_func_proto *(*get_func_proto)(enum bpf_func_id func_id);
@@ -111,7 +113,7 @@ struct bpf_verifier_ops {
u32 (*convert_ctx_access)(enum bpf_access_type type, int dst_reg,
int src_reg, int ctx_off,
- struct bpf_insn *insn);
+ struct bpf_insn *insn, struct bpf_prog *prog);
};
struct bpf_prog_type_list {
@@ -120,8 +122,6 @@ struct bpf_prog_type_list {
enum bpf_prog_type type;
};
-struct bpf_prog;
-
struct bpf_prog_aux {
atomic_t refcnt;
u32 used_map_cnt;
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 1bbce14bcf17..4165e9ac9e36 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -13,6 +13,7 @@
#include <linux/printk.h>
#include <linux/workqueue.h>
#include <linux/sched.h>
+#include <net/sch_generic.h>
#include <asm/cacheflush.h>
@@ -302,10 +303,6 @@ struct bpf_prog_aux;
bpf_size; \
})
-/* Macro to invoke filter function. */
-#define SK_RUN_FILTER(filter, ctx) \
- (*filter->prog->bpf_func)(ctx, filter->prog->insnsi)
-
#ifdef CONFIG_COMPAT
/* A struct sock_filter is architecture independent. */
struct compat_sock_fprog {
@@ -329,6 +326,7 @@ struct bpf_prog {
kmemcheck_bitfield_begin(meta);
u16 jited:1, /* Is our filter JIT'ed? */
gpl_compatible:1, /* Is filter GPL compatible? */
+ cb_access:1, /* Is control block accessed? */
dst_needed:1; /* Do we need dst entry? */
kmemcheck_bitfield_end(meta);
u32 len; /* Number of filter blocks */
@@ -352,6 +350,39 @@ struct sk_filter {
#define BPF_PROG_RUN(filter, ctx) (*filter->bpf_func)(ctx, filter->insnsi)
+static inline u32 bpf_prog_run_save_cb(const struct bpf_prog *prog,
+ struct sk_buff *skb)
+{
+ u8 *cb_data = qdisc_skb_cb(skb)->data;
+ u8 saved_cb[QDISC_CB_PRIV_LEN];
+ u32 res;
+
+ BUILD_BUG_ON(FIELD_SIZEOF(struct __sk_buff, cb) !=
+ QDISC_CB_PRIV_LEN);
+
+ if (unlikely(prog->cb_access)) {
+ memcpy(saved_cb, cb_data, sizeof(saved_cb));
+ memset(cb_data, 0, sizeof(saved_cb));
+ }
+
+ res = BPF_PROG_RUN(prog, skb);
+
+ if (unlikely(prog->cb_access))
+ memcpy(cb_data, saved_cb, sizeof(saved_cb));
+
+ return res;
+}
+
+static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
+ struct sk_buff *skb)
+{
+ u8 *cb_data = qdisc_skb_cb(skb)->data;
+
+ if (unlikely(prog->cb_access))
+ memset(cb_data, 0, QDISC_CB_PRIV_LEN);
+ return BPF_PROG_RUN(prog, skb);
+}
+
static inline unsigned int bpf_prog_size(unsigned int proglen)
{
return max(sizeof(struct bpf_prog),
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index b074b23000d6..f8da034c2258 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -2024,7 +2024,7 @@ static int convert_ctx_accesses(struct verifier_env *env)
cnt = env->prog->aux->ops->
convert_ctx_access(type, insn->dst_reg, insn->src_reg,
- insn->off, insn_buf);
+ insn->off, insn_buf, env->prog);
if (cnt == 0 || cnt >= ARRAY_SIZE(insn_buf)) {
verbose("bpf verifier is misconfigured\n");
return -EINVAL;
diff --git a/net/core/filter.c b/net/core/filter.c
index 342e6c8fc415..5f4cf1cffed3 100644
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -56,10 +56,10 @@
* @sk: sock associated with &sk_buff
* @skb: buffer to filter
*
- * Run the filter code and then cut skb->data to correct size returned by
- * SK_RUN_FILTER. If pkt_len is 0 we toss packet. If skb->len is smaller
+ * Run the eBPF program and then cut skb->data to correct size returned by
+ * the program. If pkt_len is 0 we toss packet. If skb->len is smaller
* than pkt_len we keep whole skb->data. This is the socket level
- * wrapper to SK_RUN_FILTER. It returns 0 if the packet should
+ * wrapper to BPF_PROG_RUN. It returns 0 if the packet should
* be accepted or -EPERM if the packet should be tossed.
*
*/
@@ -83,7 +83,7 @@ int sk_filter(struct sock *sk, struct sk_buff *skb)
rcu_read_lock();
filter = rcu_dereference(sk->sk_filter);
if (filter) {
- unsigned int pkt_len = SK_RUN_FILTER(filter, skb);
+ unsigned int pkt_len = bpf_prog_run_save_cb(filter->prog, skb);
err = pkt_len ? pskb_trim(skb, pkt_len) : -EPERM;
}
@@ -1736,7 +1736,8 @@ static bool tc_cls_act_is_valid_access(int off, int size,
static u32 bpf_net_convert_ctx_access(enum bpf_access_type type, int dst_reg,
int src_reg, int ctx_off,
- struct bpf_insn *insn_buf)
+ struct bpf_insn *insn_buf,
+ struct bpf_prog *prog)
{
struct bpf_insn *insn = insn_buf;
@@ -1827,6 +1828,7 @@ static u32 bpf_net_convert_ctx_access(enum bpf_access_type type, int dst_reg,
offsetof(struct __sk_buff, cb[4]):
BUILD_BUG_ON(FIELD_SIZEOF(struct qdisc_skb_cb, data) < 20);
+ prog->cb_access = 1;
ctx_off -= offsetof(struct __sk_buff, cb[0]);
ctx_off += offsetof(struct sk_buff, cb);
ctx_off += offsetof(struct qdisc_skb_cb, data);
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index 81c900fbc4a4..104910f7d1fb 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -1423,7 +1423,7 @@ static unsigned int fanout_demux_bpf(struct packet_fanout *f,
rcu_read_lock();
prog = rcu_dereference(f->bpf_prog);
if (prog)
- ret = BPF_PROG_RUN(prog, skb) % num;
+ ret = bpf_prog_run_clear_cb(prog, skb) % num;
rcu_read_unlock();
return ret;
@@ -1939,16 +1939,16 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg,
return err;
}
-static unsigned int run_filter(const struct sk_buff *skb,
- const struct sock *sk,
- unsigned int res)
+static unsigned int run_filter(struct sk_buff *skb,
+ const struct sock *sk,
+ unsigned int res)
{
struct sk_filter *filter;
rcu_read_lock();
filter = rcu_dereference(sk->sk_filter);
if (filter != NULL)
- res = SK_RUN_FILTER(filter, skb);
+ res = bpf_prog_run_clear_cb(filter->prog, skb);
rcu_read_unlock();
return res;
]
|
| 45/2 |
2026/11/18 01:04 |
llm |
expert |
0m
Model:
Tokens:
input: 47968
output: 26
thoughts: 66
Thoughts: Okay, let's dive into this change. My immediate thought is, "Aha! We need to understand the evolution of `bpf_prog_run_clear_cb` and its transition from `BPF_PROG_RUN` to `bpf_prog_run_pin_on_cpu`." I'm mentally tracing this through.
**Investigating the BPF Callback Transition**
The critical piece here is tracking down commit `3d9f773cf287`. That commit is the Rosetta Stone, it'll illuminate the 'why' and 'how' behind this function's modification. It's the key to unpacking the logic and the reasoning behind this shift in how the callback is handled. I need to get into the details of the patch, the motivations of the original authors, and the potential implications of this functional change to understand this better. Let's see what that commit holds.
|
| 46/2 |
2026/11/18 01:04 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 3d9f773cf2876c01a505b9fe27270901d464e90a
Author: David Miller <davem@davemloft.net>
Date: Mon Feb 24 15:01:43 2020 +0100
bpf: Use bpf_prog_run_pin_on_cpu() at simple call sites.
All of these cases are strictly of the form:
preempt_disable();
BPF_PROG_RUN(...);
preempt_enable();
Replace this with bpf_prog_run_pin_on_cpu() which wraps BPF_PROG_RUN()
with:
migrate_disable();
BPF_PROG_RUN(...);
migrate_enable();
On non RT enabled kernels this maps to preempt_disable/enable() and on RT
enabled kernels this solely prevents migration, which is sufficient as
there is no requirement to prevent reentrancy to any BPF program from a
preempting task. The only requirement is that the program stays on the same
CPU.
Therefore, this is a trivially correct transformation.
The seccomp loop does not need protection over the loop. It only needs
protection per BPF filter program
[ tglx: Converted to bpf_prog_run_pin_on_cpu() ]
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200224145643.691493094@linutronix.de
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 1982a52eb4c9..9270de2a0df8 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -717,9 +717,7 @@ static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
if (unlikely(prog->cb_access))
memset(cb_data, 0, BPF_SKB_CB_LEN);
- preempt_disable();
- res = BPF_PROG_RUN(prog, skb);
- preempt_enable();
+ res = bpf_prog_run_pin_on_cpu(prog, skb);
return res;
}
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index b6ea3dcb57bf..787041eb011b 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -268,16 +268,14 @@ static u32 seccomp_run_filters(const struct seccomp_data *sd,
* All filters in the list are evaluated and the lowest BPF return
* value always takes priority (ignoring the DATA).
*/
- preempt_disable();
for (; f; f = f->prev) {
- u32 cur_ret = BPF_PROG_RUN(f->prog, sd);
+ u32 cur_ret = bpf_prog_run_pin_on_cpu(f->prog, sd);
if (ACTION_ONLY(cur_ret) < ACTION_ONLY(ret)) {
ret = cur_ret;
*match = f;
}
}
- preempt_enable();
return ret;
}
#endif /* CONFIG_SECCOMP_FILTER */
diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c
index a1670dff0629..3eff84824c8b 100644
--- a/net/core/flow_dissector.c
+++ b/net/core/flow_dissector.c
@@ -920,9 +920,7 @@ bool bpf_flow_dissect(struct bpf_prog *prog, struct bpf_flow_dissector *ctx,
(int)FLOW_DISSECTOR_F_STOP_AT_ENCAP);
flow_keys->flags = flags;
- preempt_disable();
- result = BPF_PROG_RUN(prog, ctx);
- preempt_enable();
+ result = bpf_prog_run_pin_on_cpu(prog, ctx);
flow_keys->nhoff = clamp_t(u16, flow_keys->nhoff, nhoff, hlen);
flow_keys->thoff = clamp_t(u16, flow_keys->thoff,
diff --git a/net/core/skmsg.c b/net/core/skmsg.c
index eeb28cb85664..c479372f2cd2 100644
--- a/net/core/skmsg.c
+++ b/net/core/skmsg.c
@@ -628,7 +628,6 @@ int sk_psock_msg_verdict(struct sock *sk, struct sk_psock *psock,
struct bpf_prog *prog;
int ret;
- preempt_disable();
rcu_read_lock();
prog = READ_ONCE(psock->progs.msg_parser);
if (unlikely(!prog)) {
@@ -638,7 +637,7 @@ int sk_psock_msg_verdict(struct sock *sk, struct sk_psock *psock,
sk_msg_compute_data_pointers(msg);
msg->sk = sk;
- ret = BPF_PROG_RUN(prog, msg);
+ ret = bpf_prog_run_pin_on_cpu(prog, msg);
ret = sk_psock_map_verd(ret, msg->sk_redir);
psock->apply_bytes = msg->apply_bytes;
if (ret == __SK_REDIRECT) {
@@ -653,7 +652,6 @@ int sk_psock_msg_verdict(struct sock *sk, struct sk_psock *psock,
}
out:
rcu_read_unlock();
- preempt_enable();
return ret;
}
EXPORT_SYMBOL_GPL(sk_psock_msg_verdict);
@@ -665,9 +663,7 @@ static int sk_psock_bpf_run(struct sk_psock *psock, struct bpf_prog *prog,
skb->sk = psock->sk;
bpf_compute_data_end_sk_skb(skb);
- preempt_disable();
- ret = BPF_PROG_RUN(prog, skb);
- preempt_enable();
+ ret = bpf_prog_run_pin_on_cpu(prog, skb);
/* strparser clones the skb before handing it to a upper layer,
* meaning skb_orphan has been called. We NULL sk on the way out
* to ensure we don't trigger a BUG_ON() in skb/sk operations
diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c
index ea9e73428ed9..56fac24a627a 100644
--- a/net/kcm/kcmsock.c
+++ b/net/kcm/kcmsock.c
@@ -380,9 +380,7 @@ static int kcm_parse_func_strparser(struct strparser *strp, struct sk_buff *skb)
struct bpf_prog *prog = psock->bpf_prog;
int res;
- preempt_disable();
- res = BPF_PROG_RUN(prog, skb);
- preempt_enable();
+ res = bpf_prog_run_pin_on_cpu(prog, skb);
return res;
}
]
|
| 47/2 |
2026/11/18 01:04 |
llm |
expert |
0m
Model:
Tokens:
input: 50386
output: 25
thoughts: 336
Thoughts: **Analyzing the Kernel Commit and its Implications**
Okay, let's break this down. That commit... on non-real-time kernels, it's essentially a `preempt_disable`/`preempt_enable` pairing. But on RT kernels, it's just about preventing task migration, which is enough because we don't need to worry about reentrancy for BPF programs being preempted. The crucial thing is that the program stays on the same CPU. So, the transformation seems trivially correct.
Now, let's rewind to 2020. I remember this! On non-RT kernels, `migrate_disable()` was *literally* defined as `preempt_disable()`! And `migrate_enable()` was `preempt_enable()`. So, if `migrate_disable()` is, in fact, `preempt_disable()`, then calling `preempt_disable()` *does* disable preemption! `preempt_disable()` itself just increments the preempt count β it works fine in interrupt context, softirq, process context, anywhere. And yes, interrupts *can* still happen while preemption is disabled, but even when a softirq runs, that `preempt_disable()` still functions as intended.
But wait a minute... now, *today*, `migrate_disable()` and `migrate_enable()` are **no longer** equivalent to `preempt_disable()` and `preempt_enable()` on non-RT kernels. Hold on! When did this change happen? When did `migrate_disable()` stop being the same as `preempt_disable()` on non-RT systems? I need to find the specific commit where this definition changed. Let's dig into the git history. I need to search for instances where `CONFIG_PREEMPT_RT` was dropped, or where the definition of `migrate_disable` was modified. That'll pinpoint the change.
|
| 48/2 |
2026/11/18 01:04 |
tool |
git-log |
10m
Error:
Args: map[CodeRegexp:define migrate_disable Count:5]
Results:
|
| 49/2 |
2026/11/18 01:14 |
llm |
expert |
0m
Model:
Tokens:
input: 51186
output: 34
thoughts: 15
Thoughts: Okay, here's my attempt at summarizing the thought process, tailored for an expert audience, and using a first-person perspective:
**Diving into Preemption Mechanisms**
Alright, I'm thinking I need to understand how preemption is managed within the Linux kernel, so I'll start with the standard location for this kind of core functionality. That would be `include/linux/preempt.h`. It's almost always the best place to start when digging into the low-level preemption mechanics. I expect to find the fundamental definitions, macros, and probably some of the core data structures that govern interrupt handling and preemption enabling/disabling at various priority levels. I anticipate seeing the definitions of the preemption counter (`preempt_count`), and likely the core mechanisms for determining whether or not a task can be preempted. It should also have any architecture specific definitions relating to preemption.
|
| 50/2 |
2026/11/18 01:14 |
tool |
git-log |
0m
Args: map[CodeRegexp:migrate_disable PathPrefix:include/linux/preempt.h]
Results: map[Output:45b7f780739a sched: Fix some typos in include/linux/preempt.h
378b7708194f sched: Make migrate_{en,dis}able() inline
cac5cefbade9 sched/smp: Make SMP unconditional
54da6a092431 locking: Introduce __cleanup() based infrastructure
74d862b682f5 sched: Make migrate_disable/enable() independent of RT
a7c81556ec4d sched: Fix migrate_disable() vs rt/dl balancing
af449901b84c sched: Add migrate_disable()
66630058e56b sched/rt: Provide migrate_disable/enable() inlines
]
|
| 51/2 |
2026/11/18 01:14 |
llm |
expert |
0m
Model:
Tokens:
input: 51617
output: 27
thoughts: 39
Thoughts: Okay, here's my interpretation of that text, presented as a personal thought process, as requested:
**Diving into Commit 74d862b682f5**
Alright, let's take a look at this commit, specifically commit `74d862b682f5`. The title immediately grabs my attention: "sched: Make migrate_disable/enable() independent of RT". Hmm, interesting. They're decoupling the migration disabling and enabling functions from the Real-Time (RT) aspects of the scheduler. I'm keen to see the details here. What were the dependencies they were trying to break? Were there performance implications, or perhaps complexity concerns, that prompted this change? I'll need to dig into the code and associated discussions to fully understand the rationale and the impact on the existing scheduler architecture. Gotta see how this affects preemption models, latency guarantees, and the overall interaction between RT and non-RT tasks. This looks like it could be a significant refactoring, so let's start digging.
|
| 52/2 |
2026/11/18 01:15 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 74d862b682f51e45d25b95b1ecf212428a4967b0
Author: Thomas Gleixner <tglx@kernel.org>
Date: Wed Nov 18 20:48:42 2020 +0100
sched: Make migrate_disable/enable() independent of RT
Now that the scheduler can deal with migrate disable properly, there is no
real compelling reason to make it only available for RT.
There are quite some code pathes which needlessly disable preemption in
order to prevent migration and some constructs like kmap_atomic() enforce
it implicitly.
Making it available independent of RT allows to provide a preemptible
variant of kmap_atomic() and makes the code more consistent in general.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Grudgingly-Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20201118204007.269943012@linutronix.de
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index 2f05e9128201..665837f9a831 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -204,6 +204,7 @@ extern int _cond_resched(void);
extern void ___might_sleep(const char *file, int line, int preempt_offset);
extern void __might_sleep(const char *file, int line, int preempt_offset);
extern void __cant_sleep(const char *file, int line, int preempt_offset);
+extern void __cant_migrate(const char *file, int line);
/**
* might_sleep - annotation for functions that can sleep
@@ -227,6 +228,18 @@ extern void __cant_sleep(const char *file, int line, int preempt_offset);
# define cant_sleep() \
do { __cant_sleep(__FILE__, __LINE__, 0); } while (0)
# define sched_annotate_sleep() (current->task_state_change = 0)
+
+/**
+ * cant_migrate - annotation for functions that cannot migrate
+ *
+ * Will print a stack trace if executed in code which is migratable
+ */
+# define cant_migrate() \
+ do { \
+ if (IS_ENABLED(CONFIG_SMP)) \
+ __cant_migrate(__FILE__, __LINE__); \
+ } while (0)
+
/**
* non_block_start - annotate the start of section where sleeping is prohibited
*
@@ -251,6 +264,7 @@ extern void __cant_sleep(const char *file, int line, int preempt_offset);
int preempt_offset) { }
# define might_sleep() do { might_resched(); } while (0)
# define cant_sleep() do { } while (0)
+# define cant_migrate() do { } while (0)
# define sched_annotate_sleep() do { } while (0)
# define non_block_start() do { } while (0)
# define non_block_end() do { } while (0)
@@ -258,13 +272,6 @@ extern void __cant_sleep(const char *file, int line, int preempt_offset);
#define might_sleep_if(cond) do { if (cond) might_sleep(); } while (0)
-#ifndef CONFIG_PREEMPT_RT
-# define cant_migrate() cant_sleep()
-#else
- /* Placeholder for now */
-# define cant_migrate() do { } while (0)
-#endif
-
/**
* abs - return absolute value of an argument
* @x: the value. If it is unsigned type, it is converted to signed type first.
diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index 8b43922e65df..6df63cbe8bb0 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -322,7 +322,7 @@ static inline void preempt_notifier_init(struct preempt_notifier *notifier,
#endif
-#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+#ifdef CONFIG_SMP
/*
* Migrate-Disable and why it is undesired.
@@ -382,43 +382,11 @@ static inline void preempt_notifier_init(struct preempt_notifier *notifier,
extern void migrate_disable(void);
extern void migrate_enable(void);
-#elif defined(CONFIG_PREEMPT_RT)
+#else
static inline void migrate_disable(void) { }
static inline void migrate_enable(void) { }
-#else /* !CONFIG_PREEMPT_RT */
-
-/**
- * migrate_disable - Prevent migration of the current task
- *
- * Maps to preempt_disable() which also disables preemption. Use
- * migrate_disable() to annotate that the intent is to prevent migration,
- * but not necessarily preemption.
- *
- * Can be invoked nested like preempt_disable() and needs the corresponding
- * number of migrate_enable() invocations.
- */
-static __always_inline void migrate_disable(void)
-{
- preempt_disable();
-}
-
-/**
- * migrate_enable - Allow migration of the current task
- *
- * Counterpart to migrate_disable().
- *
- * As migrate_disable() can be invoked nested, only the outermost invocation
- * reenables migration.
- *
- * Currently mapped to preempt_enable().
- */
-static __always_inline void migrate_enable(void)
-{
- preempt_enable();
-}
-
-#endif /* CONFIG_SMP && CONFIG_PREEMPT_RT */
+#endif /* CONFIG_SMP */
#endif /* __LINUX_PREEMPT_H */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 3af9d52fe093..a33f35f68060 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -715,7 +715,7 @@ struct task_struct {
const cpumask_t *cpus_ptr;
cpumask_t cpus_mask;
void *migration_pending;
-#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+#ifdef CONFIG_SMP
unsigned short migration_disabled;
#endif
unsigned short migration_flags;
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index e6473ecaab3c..c962922784d1 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1728,8 +1728,6 @@ void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags)
#ifdef CONFIG_SMP
-#ifdef CONFIG_PREEMPT_RT
-
static void
__do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask, u32 flags);
@@ -1800,8 +1798,6 @@ static inline bool rq_has_pinned_tasks(struct rq *rq)
return rq->nr_pinned;
}
-#endif
-
/*
* Per-CPU kthreads are allowed to run on !active && online CPUs, see
* __set_cpus_allowed_ptr() and select_fallback_rq().
@@ -2882,7 +2878,7 @@ void sched_set_stop_task(int cpu, struct task_struct *stop)
}
}
-#else
+#else /* CONFIG_SMP */
static inline int __set_cpus_allowed_ptr(struct task_struct *p,
const struct cpumask *new_mask,
@@ -2891,10 +2887,6 @@ static inline int __set_cpus_allowed_ptr(struct task_struct *p,
return set_cpus_allowed_ptr(p, new_mask);
}
-#endif /* CONFIG_SMP */
-
-#if !defined(CONFIG_SMP) || !defined(CONFIG_PREEMPT_RT)
-
static inline void migrate_disable_switch(struct rq *rq, struct task_struct *p) { }
static inline bool rq_has_pinned_tasks(struct rq *rq)
@@ -2902,7 +2894,7 @@ static inline bool rq_has_pinned_tasks(struct rq *rq)
return false;
}
-#endif
+#endif /* !CONFIG_SMP */
static void
ttwu_stat(struct task_struct *p, int cpu, int wake_flags)
@@ -7924,6 +7916,39 @@ void __cant_sleep(const char *file, int line, int preempt_offset)
add_taint(TAINT_WARN, LOCKDEP_STILL_OK);
}
EXPORT_SYMBOL_GPL(__cant_sleep);
+
+#ifdef CONFIG_SMP
+void __cant_migrate(const char *file, int line)
+{
+ static unsigned long prev_jiffy;
+
+ if (irqs_disabled())
+ return;
+
+ if (is_migration_disabled(current))
+ return;
+
+ if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
+ return;
+
+ if (preempt_count() > 0)
+ return;
+
+ if (time_before(jiffies, prev_jiffy + HZ) && prev_jiffy)
+ return;
+ prev_jiffy = jiffies;
+
+ pr_err("BUG: assuming non migratable context at %s:%d\n", file, line);
+ pr_err("in_atomic(): %d, irqs_disabled(): %d, migration_disabled() %u pid: %d, name: %s\n",
+ in_atomic(), irqs_disabled(), is_migration_disabled(current),
+ current->pid, current->comm);
+
+ debug_show_held_locks(current);
+ dump_stack();
+ add_taint(TAINT_WARN, LOCKDEP_STILL_OK);
+}
+EXPORT_SYMBOL_GPL(__cant_migrate);
+#endif
#endif
#ifdef CONFIG_MAGIC_SYSRQ
diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index 590e6f27068c..f5acb6c5ce49 100644
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1056,7 +1056,7 @@ struct rq {
struct cpuidle_state *idle_state;
#endif
-#if defined(CONFIG_PREEMPT_RT) && defined(CONFIG_SMP)
+#ifdef CONFIG_SMP
unsigned int nr_pinned;
#endif
unsigned int push_busy;
@@ -1092,7 +1092,7 @@ static inline int cpu_of(struct rq *rq)
static inline bool is_migration_disabled(struct task_struct *p)
{
-#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+#ifdef CONFIG_SMP
return p->migration_disabled;
#else
return false;
diff --git a/lib/smp_processor_id.c b/lib/smp_processor_id.c
index faaa927ac2c8..1c1dbd300325 100644
--- a/lib/smp_processor_id.c
+++ b/lib/smp_processor_id.c
@@ -26,7 +26,7 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)
if (current->nr_cpus_allowed == 1)
goto out;
-#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
+#ifdef CONFIG_SMP
if (current->migration_disabled)
goto out;
#endif
]
|
| 53/2 |
2026/11/18 01:15 |
llm |
expert |
0m
Model:
Tokens:
input: 55084
output: 35
thoughts: 1048
Thoughts: **Diving Deep into `migrate_disable()` and Interrupt Context**
Okay, so I'm looking at this code, and a few things are immediately jumping out at me. It seems that back in November 2020, `migrate_disable()` was decoupled from the Real-Time (RT) preemption code. Now, the functions `migrate_disable()` and `migrate_enable()` directly manipulate `current->migration_disabled`. That's the starting point.
My main concern is, what happens if `migrate_disable()` gets called from an interrupt or softirq context? I'm trying to mentally trace the execution path. Can that even *happen*? Is this allowed? Is it a bug, or potentially a race condition? I need to know.
Let's break down `__migrate_enable()` and `__migrate_disable()` now to be sure of the state.
I'm seeing that `__migrate_enable()` contains a `guard(preempt)()` call. Crucially, this is disabling *preemption*, not interrupts. And, before the `guard()` there's a check, `if (p->migration_disabled > 1)`. The point is, neither preemption nor interrupts are disabled when the check is true.
What happens when an interrupt or softirq occurs while a task is running? The softirq code runs on the *same* CPU as the interrupted task, and `current` still refers to that interrupted task. If code within the softirq - specifically, it appears to be `run_filter` calling `bpf_prog_run_clear_cb`, which then calls `bpf_prog_run_pin_on_cpu` which calls `migrate_disable()` - executes, then `migrate_disable()` *does* access `current->migration_disabled`! That means it's touching the *interrupted* task's state.
Wait, why are we even getting to the point where `run_filter` is calling `bpf_prog_run_clear_cb`, which calls `bpf_prog_run_pin_on_cpu`? How does this happen? Let's trace it back. Okay, the source code in `af_packet.c` shows the call chain: `run_filter` calls `bpf_prog_run_clear_cb` conditionally via the `sk->sk_filter` member. And, given this, I need to know where else these callbacks, `bpf_prog_run_clear_cb` and `bpf_prog_run_save_cb`, are being called in the kernel. Let's do a search for those functions. That should help me figure out what's really going on here.
|
| 54/2 |
2026/11/18 01:15 |
tool |
grepper |
0m
Args: map[Expression:bpf_prog_run_clear_cb|bpf_prog_run_save_cb]
Results: map[Output:drivers/net/tun.c=483=static u16 tun_ebpf_select_queue(struct tun_struct *tun, struct sk_buff *skb)
--
drivers/net/tun.c-494- if (prog)
drivers/net/tun.c:495: ret = bpf_prog_run_clear_cb(prog->prog, skb);
drivers/net/tun.c-496-
--
drivers/net/tun.c=1054=static unsigned int run_ebpf_filter(struct tun_struct *tun,
--
drivers/net/tun.c-1060- if (prog)
drivers/net/tun.c:1061: len = bpf_prog_run_clear_cb(prog->prog, skb);
drivers/net/tun.c-1062-
--
include/linux/filter.h=1058=static inline u8 *bpf_skb_cb(const struct sk_buff *skb)
--
include/linux/filter.h-1077-/* Must be invoked with migration disabled */
include/linux/filter.h:1078:static inline u32 __bpf_prog_run_save_cb(const struct bpf_prog *prog,
include/linux/filter.h-1079- const void *ctx)
--
include/linux/filter.h-1098-
include/linux/filter.h:1099:static inline u32 bpf_prog_run_save_cb(const struct bpf_prog *prog,
include/linux/filter.h-1100- struct sk_buff *skb)
--
include/linux/filter.h-1104- migrate_disable();
include/linux/filter.h:1105: res = __bpf_prog_run_save_cb(prog, skb);
include/linux/filter.h-1106- migrate_enable();
--
include/linux/filter.h-1109-
include/linux/filter.h:1110:static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
include/linux/filter.h-1111- struct sk_buff *skb)
--
kernel/bpf/cgroup.c=1789=int __cgroup_bpf_run_filter_skb(struct sock *sk,
--
kernel/bpf/cgroup.c-1822- ret = bpf_prog_run_array_cg(&cgrp->bpf, atype, skb,
kernel/bpf/cgroup.c:1823: __bpf_prog_run_save_cb, 0, &flags);
kernel/bpf/cgroup.c-1824-
--
kernel/bpf/cgroup.c-1848- ret = bpf_prog_run_array_cg(&cgrp->bpf, atype,
kernel/bpf/cgroup.c:1849: skb, __bpf_prog_run_save_cb, 0,
kernel/bpf/cgroup.c-1850- NULL);
--
net/core/filter.c=137=sk_filter_trim_cap(struct sock *sk, struct sk_buff *skb, unsigned int cap)
--
net/core/filter.c-167- skb->sk = sk;
net/core/filter.c:168: pkt_len = bpf_prog_run_save_cb(filter->prog, skb);
net/core/filter.c-169- skb->sk = save_sk;
--
net/core/lwt_bpf.c=39=static int run_lwt_bpf(struct sk_buff *skb, struct bpf_lwt_prog *lwt,
--
net/core/lwt_bpf.c-50- bpf_compute_data_pointers(skb);
net/core/lwt_bpf.c:51: ret = bpf_prog_run_save_cb(lwt->prog, skb);
net/core/lwt_bpf.c-52-
--
net/core/lwt_bpf.c-92- * stays: behind a VRF skb->skb_iif is the L3 master and not the port.
net/core/lwt_bpf.c:93: * bpf_lwt_push_encap() cannot do this, because bpf_prog_run_save_cb() can
net/core/lwt_bpf.c-94- * restore skb->cb after the prog.
--
net/core/sock_reuseport.c=499=static struct sock *run_bpf_filter(struct sock_reuseport *reuse, u16 socks,
--
net/core/sock_reuseport.c-517- }
net/core/sock_reuseport.c:518: index = bpf_prog_run_save_cb(prog, skb);
net/core/sock_reuseport.c-519- __skb_push(skb, hdr_len);
--
net/ipv6/seg6_local.c=1597=static int input_action_end_bpf(struct sk_buff *skb,
--
net/ipv6/seg6_local.c-1619- * statement. It is also accessed by the bpf_lwt_seg6_* helpers via
net/ipv6/seg6_local.c:1620: * bpf_prog_run_save_cb().
net/ipv6/seg6_local.c-1621- */
--
net/ipv6/seg6_local.c-1629- bpf_compute_data_pointers(skb);
net/ipv6/seg6_local.c:1630: ret = bpf_prog_run_save_cb(slwt->bpf.prog, skb);
net/ipv6/seg6_local.c-1631- rcu_read_unlock();
--
net/netfilter/xt_bpf.c=96=static bool bpf_mt_v1(const struct sk_buff *skb, struct xt_action_param *par)
--
net/netfilter/xt_bpf.c-99-
net/netfilter/xt_bpf.c:100: return !!bpf_prog_run_save_cb(info->filter, (struct sk_buff *) skb);
net/netfilter/xt_bpf.c-101-}
--
net/packet/af_packet.c=1458=static unsigned int fanout_demux_bpf(struct packet_fanout *f,
--
net/packet/af_packet.c-1467- if (prog)
net/packet/af_packet.c:1468: ret = bpf_prog_run_clear_cb(prog, skb) % num;
net/packet/af_packet.c-1469- rcu_read_unlock();
--
net/packet/af_packet.c=2102=static unsigned int run_filter(struct sk_buff *skb,
--
net/packet/af_packet.c-2110- if (filter != NULL)
net/packet/af_packet.c:2111: res = bpf_prog_run_clear_cb(filter->prog, skb);
net/packet/af_packet.c-2112- rcu_read_unlock();
]
|
| 55/2 |
2026/11/18 01:15 |
llm |
expert |
0m
Model:
Tokens:
input: 58656
output: 36
thoughts: 2971
Thoughts: **BPF Context Migration Concerns**
Okay, here's what I'm seeing: The `sk_filter_trim_cap` function, which gets hit on every TCP/UDP/etc. packet, calls `bpf_prog_run_save_cb`. This in turn uses `migrate_disable()` and `migrate_enable()`. These functions are also used in other places like `af_packet` packet reception and fanout demuxing. The core concern is whether these `migrate_disable()` and `migrate_enable()` calls can interact badly with softirqs. Specifically, can they cause issues if they're called within the softirq context. The `in_interrupt()` flag will be true in a softirq context.
`migrate_disable()` seems relatively straightforward. It disables migration (not preemption) with a guard scope that re-enables preemption at the end. It also increments a counter (`this_rq_pinned`) and sets `migration_disabled` to 1. The key point is that it *doesn't* actually keep preemption disabled across the critical section, it's just about preventing migration.
Now, let's consider what happens concurrently, on the *same* CPU, when a task is interrupted. A task (e.g., task 5152) is running on CPU 0 and then a softirq triggers on the same CPU. When the interrupt happens, it kicks off the interrupt handler, and if softirqs are pending, `__do_softirq()` runs. *Crucially*, `current` in the softirq context points to task 5152.
Let's look at the possible scenarios. First, task 5152 might be in the middle of a `migrate_enable()` call. When the interrupt hits *before* `guard(preempt)()` (or inside it), the softirq then calls `migrate_disable()`. If `p->migration_disabled` is initially 1, it's incremented to 2 in the softirq. Then, during the softirq's `migrate_enable()` call, `p->migration_disabled` gets decremented back to 1. It gets back to where it was.
But wait. If task 5152 had `p->migration_disabled` as 2, and the interrupt arrives in the read-modify-write operation (specifically, the decrement), we can have problems. The softirq increments it to 3 and then decrements it to 2. The original task decrements it back to 1. This would be fine if the original task had `migration_disabled` as 0 or 1.
The softirq could also run with the current task's `migration_disabled` as 0. The softirq calls `migrate_disable()`, incrementing the task's `migration_disabled` to 1. Then `migrate_enable()` runs in the softirq. Here's where it gets interesting: If a task's `p->cpus_ptr` is not its normal `cpus_mask` pointer, then `___migrate_enable` is called. It looks like it is modifying the current task with a `set_cpus_allowed_ptr`, which updates a task's allowed CPUs mask. This can happen if the task had migration disabled or when switching out tasks with `migrate_disable_switch`. Now, what if, because of all the concurrent accesses, we see a corrupted `p->cpus_ptr`? We need to look deeper into `___migrate_enable` and `__set_cpus_allowed_ptr` to see what race conditions might exist.
|
| 56/2 |
2026/11/18 01:16 |
tool |
read-file |
0m
Args: map[File:kernel/sched/core.c FirstLine:2780 LineCount:60]
Results: map[Contents:2780: goto out_unlock;
2781:
2782: if (is_migration_disabled(p)) {
2783: p->migration_flags |= MDF_PUSH;
2784: goto out_unlock;
2785: }
2786:
2787: p->migration_flags &= ~MDF_PUSH;
2788:
2789: if (p->sched_class->find_lock_rq)
2790: lowest_rq = p->sched_class->find_lock_rq(p, rq);
2791:
2792: if (!lowest_rq)
2793: goto out_unlock;
2794:
2795: lockdep_assert_rq_held(lowest_rq);
2796:
2797: // XXX validate p is still the highest prio task
2798: if (task_rq(p) == rq) {
2799: move_queued_task_locked(rq, lowest_rq, p);
2800: resched_curr(lowest_rq);
2801: }
2802:
2803: double_unlock_balance(rq, lowest_rq);
2804:
2805: out_unlock:
2806: rq->push_busy = false;
2807: raw_spin_rq_unlock(rq);
2808: raw_spin_unlock_irq(&p->pi_lock);
2809:
2810: put_task_struct(p);
2811: return 0;
2812: }
2813:
2814: static inline void mm_update_cpus_allowed(struct mm_struct *mm, const cpumask_t *affmask);
2815:
2816: /*
2817: * sched_class::set_cpus_allowed must do the below, but is not required to
2818: * actually call this function.
2819: */
2820: void set_cpus_allowed_common(struct task_struct *p, struct affinity_context *ctx)
2821: {
2822: if (ctx->flags & (SCA_MIGRATE_ENABLE | SCA_MIGRATE_DISABLE)) {
2823: p->cpus_ptr = ctx->new_mask;
2824: return;
2825: }
2826:
2827: cpumask_copy(&p->cpus_mask, ctx->new_mask);
2828: p->nr_cpus_allowed = cpumask_weight(ctx->new_mask);
2829: mm_update_cpus_allowed(p->mm, ctx->new_mask);
2830:
2831: /*
2832: * Swap in a new user_cpus_ptr if SCA_USER flag set
2833: */
2834: if (ctx->flags & SCA_USER)
2835: swap(p->user_cpus_ptr, ctx->user_mask);
2836: }
2837:
2838: static void
2839: do_set_cpus_allowed(struct task_struct *p, struct affinity_context *ctx)
]
|
| 57/2 |
2026/11/18 01:16 |
llm |
expert |
0m
Model:
Tokens:
input: 63360
output: 36
thoughts: 17
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Hunting Down `__set_cpus_allowed_ptr`**
Right, let's track down the definition of `__set_cpus_allowed_ptr`. I need to get to the heart of how CPU affinity is managed, likely at a very low level. This suggests Iβm dealing with some fundamental scheduling or resource allocation concern. Knowing the specific architecture or kernel version would obviously help narrow the search immediately, but assuming I don't have that context readily available, I'll start with the standard approaches.
I'll probably begin with a grep across the kernel source, or, depending on the toolchain at my disposal, perhaps use cscope or another more sophisticated code navigation tool if the project setup supports it. I'll likely look for the declaration, then the implementation. I'll also be mindful of any potential macros or inline functions that might wrap this. I'll be looking for any hints about where it's used; are there any calls from within the scheduler or process management code? Does it interact directly with task structures or CPU masks? I expect it to be a kernel-space function and that itβll likely reside in the task or scheduling sections of the kernel. The specific naming convention, with the double underscores, suggests it's likely intended for internal kernel use, not something exposed directly to user space. Therefore, I'll be focused on core kernel header files and source files, skipping user-space libraries. The "ptr" suffix strongly implies that we're dealing with a pointer to a function or data structure, something that would need a dedicated pointer to be able to modify. I can expect to find related code with "cpu_mask" related variables.
|
| 58/2 |
2026/11/18 01:16 |
tool |
read-file |
0m
Args: map[File:kernel/sched/core.c FirstLine:2950 LineCount:60]
Results: map[Contents:2950: * This means we need the following scheme:
2951: *
2952: * P0@CPU0 P1
2953: *
2954: * migrate_disable();
2955: * <preempted>
2956: * set_cpus_allowed_ptr(P0, [1]);
2957: * <blocks>
2958: * <resumes>
2959: * migrate_enable();
2960: * __set_cpus_allowed_ptr();
2961: * <wakes local stopper>
2962: * `--> <woken on migration completion>
2963: *
2964: * Now the fun stuff: there may be several P1-like tasks, i.e. multiple
2965: * concurrent set_cpus_allowed_ptr(P0, [*]) calls. CPU affinity changes of any
2966: * task p are serialized by p->pi_lock, which we can leverage: the one that
2967: * should come into effect at the end of the Migrate-Disable region is the last
2968: * one. This means we only need to track a single cpumask (i.e. p->cpus_mask),
2969: * but we still need to properly signal those waiting tasks at the appropriate
2970: * moment.
2971: *
2972: * This is implemented using struct set_affinity_pending. The first
2973: * __set_cpus_allowed_ptr() caller within a given Migrate-Disable region will
2974: * setup an instance of that struct and install it on the targeted task_struct.
2975: * Any and all further callers will reuse that instance. Those then wait for
2976: * a completion signaled at the tail of the CPU stopper callback (1), triggered
2977: * on the end of the Migrate-Disable region (i.e. outermost migrate_enable()).
2978: *
2979: *
2980: * (1) In the cases covered above. There is one more where the completion is
2981: * signaled within affine_move_task() itself: when a subsequent affinity request
2982: * occurs after the stopper bailed out due to the targeted task still being
2983: * Migrate-Disable. Consider:
2984: *
2985: * Initial conditions: P0->cpus_mask = [0, 1]
2986: *
2987: * CPU0 P1 P2
2988: * <P0>
2989: * migrate_disable();
2990: * <preempted>
2991: * set_cpus_allowed_ptr(P0, [1]);
2992: * <blocks>
2993: * <migration/0>
2994: * migration_cpu_stop()
2995: * is_migration_disabled()
2996: * <bails>
2997: * set_cpus_allowed_ptr(P0, [0, 1]);
2998: * <signal completion>
2999: * <awakes>
3000: *
3001: * Note that the above is safe vs a concurrent migrate_enable(), as any
3002: * pending affinity completion is preceded by an uninstallation of
3003: * p->migration_pending done with p->pi_lock held.
3004: */
3005: static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf,
3006: int dest_cpu, unsigned int flags)
3007: __releases(__rq_lockp(rq), &p->pi_lock)
3008: {
3009: struct set_affinity_pending my_pending = { }, *pending = NULL;
]
|
| 59/2 |
2026/11/18 01:16 |
llm |
expert |
0m
Model:
Tokens:
input: 64857
output: 35
thoughts: 1721
Thoughts: **Analyzing a Potential Preemption Vulnerability in the Migration Mechanism**
Okay, so I'm looking at this code and I've spotted something interesting, a possible preemption issue within the migration disable/enable sequence. It started with this sequence: `migrate_disable()`, then some preemption, then `set_cpus_allowed_ptr()`, and finally `migrate_enable()`. I immediately noticed the `__migrate_enable()` function, and I see a potential issue involving softirqs.
Specifically, I'm thinking about a scenario where a task, let's call it T1, is already running with `migration_disabled == 0`. Now an interrupt hits, triggering a softirq. The softirq calls `migrate_disable()`. In this softirq context, `migrate_disable()` increments `this_rq_pinned()` and sets `p->migration_disabled = 1`. Then, `migrate_enable()` gets called, and depending on what happens with `cpus_ptr` and `cpus_mask`, we have a potential race condition.
I then started thinking about a case where a task has already called `migrate_disable()`. Suppose task T1 calls `migrate_disable()` and is preempted when `T1->migration_disabled == 1`. Then, `set_cpus_allowed_ptr()` is called on T1, thus `T1->cpus_ptr != &T1->cpus_mask`. Then, T1 is rescheduled. An interrupt fires, softirq runs on the same CPU, and executes `migrate_disable()`, incrementing `T1->migration_disabled` to 2. Then, `migrate_enable()` is called, decrementing migration_disabled back to 1. No race.
But now, what if the softirq interrupts T1 *inside* `__migrate_enable()`? Wait, this is the exact case reported by KCSAN! It shows Task 5152 inside `__migrate_enable()`, specifically the comparison `p->migration_disabled > 1`. Suppose Task 5152 was running some BPF code that in turn executed `__migrate_enable()`. The report is saying that `p->migration_disabled` was 1, and so the test `p->migration_disabled > 1` is false, and it will proceed.
What I'm worried about is the following: The interrupt could occur after we compare `p->migration_disabled` but BEFORE `p->migration_disabled = 0` or worse, after `p->migration_disabled = 0` but before `this_rq_pinned()--`. What's crucial here is that `guard(preempt)` disables preemption, but interrupts are still enabled. So softirqs can run, even with preemption disabled. They are run from interrupt return! Specifically, `irq_exit_rcu` -> `invoke_softirq`. My question is, does `irq_exit()` even run softirqs when preemption is disabled? I need to check `__irq_exit_rcu()` and lines 542 and 786 in `kernel/softirq.c` to be sure.
|
| 60/2 |
2026/11/18 01:16 |
tool |
read-file |
0m
Args: map[File:kernel/softirq.c FirstLine:535 LineCount:25]
Results: map[Contents: 535: if (!force_irqthreads() || !__this_cpu_read(ksoftirqd)) {
536: #ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK
537: /*
538: * We can safely execute softirq on the current stack if
539: * it is the irq stack, because it should be near empty
540: * at this stage.
541: */
542: __do_softirq();
543: #else
544: /*
545: * Otherwise, irq_exit() is called on the task stack that can
546: * be potentially deep already. So call softirq in its own stack
547: * to prevent from any overrun.
548: */
549: do_softirq_own_stack();
550: #endif
551: } else {
552: wakeup_softirqd();
553: }
554: }
555:
556: asmlinkage __visible void do_softirq(void)
557: {
558: __u32 pending;
559: unsigned long flags;
]
|
| 61/2 |
2026/11/18 01:16 |
llm |
expert |
0m
Model:
Tokens:
input: 67662
output: 35
thoughts: 19
Thoughts: Here's my summary, as if I were thinking these thoughts:
**Diving into the SoftIRQ Implementation**
Alright, time to troubleshoot this performance bottleneck. Based on the symptoms, I'm thinking the issue might be related to SoftIRQs. My first instinct is to dive into the kernel code directly. Specifically, let's take a look at the `kernel/softirq.c` file. I'm going to jump to line 786. That's a reasonable place to start, as I suspect this is a common area for related issues. Hopefully, this should show me the relevant section.
|
| 62/2 |
2026/11/18 01:16 |
tool |
read-file |
0m
Args: map[File:kernel/softirq.c FirstLine:775 LineCount:25]
Results: map[Contents: 775:
776: /*
777: * HARDIRQ_OFFSET is still set. Only the outermost interrupt handles
778: * softirqs, and only if it did not hit a softirq or BH disabled section.
779: */
780: if (irq_count() == HARDIRQ_OFFSET && local_softirq_pending()) {
781: /*
782: * If we left hrtimers unarmed, make sure to arm them now,
783: * before enabling interrupts to run softirq.
784: */
785: hrtimer_rearm_deferred();
786: invoke_softirq();
787: }
788:
789: /*
790: * Wake the timer thread even if the interrupt hit a softirq or a
791: * section with BHs disabled. Only the outermost interrupt can wake it.
792: */
793: if (IS_ENABLED(CONFIG_IRQ_FORCED_THREADING) && force_irqthreads() &&
794: local_timers_pending_force_th() &&
795: hardirq_count() == HARDIRQ_OFFSET)
796: wake_timersd();
797:
798: tick_irq_exit();
799: preempt_count_sub(HARDIRQ_OFFSET);
]
|
| 63/2 |
2026/11/18 01:16 |
llm |
expert |
0m
Model:
Tokens:
input: 68247
output: 28
thoughts: 4918
Thoughts: **Deep Dive into `migrate_disable`/`migrate_enable` Interactions and Potential Race Conditions**
Okay, here's what I've been thinking through regarding those functions. I started by looking at line 780, and the `irq_count()` check, which sparked a train of thought about how preemption and softirqs interact with migration disabling. The initial thought was that if preemption is disabled but softirqs aren't, then `irq_count()` might incorrectly signal `HARDIRQ_OFFSET`. Then I dove into the code and looked at `__migrate_enable` and `__migrate_disable` in `sched.h`.
Specifically, I focused on the following scenario: What happens if a task calls `migrate_disable()`, then, before it calls `migrate_enable()`, a softirq (triggered, say, by a BPF filter via `af_packet.c`) fires. The softirq's execution path goes: `packet_rcv` -> `run_filter` -> `bpf_prog_run_clear_cb` -> `bpf_prog_run_pin_on_cpu` -> `migrate_disable()` and then `migrate_enable()`.
I realized that if the softirq enters `__migrate_disable()` *after* the task had set `p->migration_disabled` to 1, the softirq will increment `p->migration_disabled` to 2. Then, the softirq calls `migrate_enable()`, which decrements `p->migration_disabled` back to 1. When the original task *finally* calls `migrate_enable()`, it sees that `migration_disabled` is still 1, and so, the migration disable counter is off.
I started analyzing all possible interrupt points within the execution sequence of `migrate_disable` and `migrate_enable`, in an effort to see if there were scenarios where `this_rq_pinned()` could be incorrectly decremented or incremented. I went through all the possible cases and thought I understood how it could work, until I realized how an interrupt could arrive between the write to p->migration_disabled = 0 and the decrementing of this_rq_pinned().
I also considered potential race conditions around the increment/decrement of `p->migration_disabled` itself, specifically if `p->migration_disabled++` is not atomic. I'm starting to think this is a serious problem. The KCSAN report confirms a data race β a read/write conflict on `p->migration_disabled`! I looked into the call stack to see why 5152 would call `migrate_enable()`. I realized `bpf_flow_dissect` called `migrate_disable()` and then `migrate_enable()`.
The more I thought about it, the more I was troubled by `migration_disabled` being modified by an interrupt, because the interrupt context is not a task. What happens when an interrupt fires while a task is in `migrate_disable()` or `migrate_enable()`? The interrupt could hit the idle task, a user task, or even another kernel thread.
This is a very tricky situation. I must consider how `migration_disabled` is used in the scheduler, so I'll grep for `migration_disabled` in `kernel/sched/`. The whole interaction between interrupt context and task context needs to be carefully scrutinized.
|
| 64/2 |
2026/11/18 01:17 |
tool |
grepper |
0m
Args: map[Expression:migration_disabled PathPrefixes:[kernel/sched/]]
Results: map[Output:
Full output is too long, showing 200 out of 258 lines.
Use more precise expression or PathPrefixes if possible.
Number of matching lines per file (9 files in total):
kernel/sched/core.c:18
kernel/sched/deadline.c:4
kernel/sched/ext/ext.c:5
kernel/sched/ext/idle.c:13
kernel/sched/ext/internal.h:1
kernel/sched/ext/sub.c:2
kernel/sched/ext/sub.h:1
kernel/sched/rt.c:4
kernel/sched/sched.h:3
kernel/sched/core.c=2471=static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
--
kernel/sched/core.c-2477-
kernel/sched/core.c:2478: if (likely(!p->migration_disabled))
kernel/sched/core.c-2479- return;
--
kernel/sched/core.c=2539=static inline bool is_cpu_allowed(struct task_struct *p, int cpu)
--
kernel/sched/core.c-2545- /* migrate_disabled() must be allowed to finish. */
kernel/sched/core.c:2546: if (is_migration_disabled(p))
kernel/sched/core.c-2547- return cpu_online(cpu);
--
kernel/sched/core.c=2657=static int migration_cpu_stop(void *data)
--
kernel/sched/core.c-2698- if (task_rq(p) == rq) {
kernel/sched/core.c:2699: if (is_migration_disabled(p))
kernel/sched/core.c-2700- goto out;
--
kernel/sched/core.c-2746- * When migrate_enable() hits a rq mis-match we can't reliably
kernel/sched/core.c:2747: * determine is_migration_disabled() and so have to chase after
kernel/sched/core.c-2748- * it.
--
kernel/sched/core.c=2771=int push_cpu_stop(void *arg)
--
kernel/sched/core.c-2781-
kernel/sched/core.c:2782: if (is_migration_disabled(p)) {
kernel/sched/core.c-2783- p->migration_flags |= MDF_PUSH;
--
kernel/sched/core.c=2924=void release_user_cpus_ptr(struct task_struct *p)
--
kernel/sched/core.c-2994- * migration_cpu_stop()
kernel/sched/core.c:2995: * is_migration_disabled()
kernel/sched/core.c-2996- * <bails>
--
kernel/sched/core.c=3005=static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf,
--
kernel/sched/core.c-3100- * MIGRATE_ENABLE gets here because 'p == current', but for
kernel/sched/core.c:3101: * anything else we cannot do is_migration_disabled(), punt
kernel/sched/core.c-3102- * and have the stopper function handle it all race-free.
--
kernel/sched/core.c-3122-
kernel/sched/core.c:3123: if (!is_migration_disabled(p)) {
kernel/sched/core.c-3124- if (task_on_rq_queued(p))
--
kernel/sched/core.c=3158=static int __set_cpus_allowed_ptr_locked(struct task_struct *p,
--
kernel/sched/core.c-3169-
kernel/sched/core.c:3170: if (kthread || is_migration_disabled(p)) {
kernel/sched/core.c-3171- /*
--
kernel/sched/core.c-3175- *
kernel/sched/core.c:3176: * Specifically, migration_disabled() tasks must not fail the
kernel/sched/core.c-3177- * cpumask_any_and_distribute() pick below, esp. so on
--
kernel/sched/core.c-3205- if (WARN_ON_ONCE(p == current &&
kernel/sched/core.c:3206: is_migration_disabled(p) &&
kernel/sched/core.c-3207- !cpumask_test_cpu(task_cpu(p), ctx->new_mask))) {
--
kernel/sched/core.c=3387=void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
--
kernel/sched/core.c-3431- */
kernel/sched/core.c:3432: WARN_ON_ONCE(is_migration_disabled(p) && !proxy_migrated);
kernel/sched/core.c-3433-
--
kernel/sched/core.c=3667=int select_task_rq(struct task_struct *p, int cpu, int *wake_flags)
--
kernel/sched/core.c-3670-
kernel/sched/core.c:3671: if (p->nr_cpus_allowed > 1 && !is_migration_disabled(p)) {
kernel/sched/core.c-3672- cpu = p->sched_class->select_task_rq(p, cpu, *wake_flags);
--
kernel/sched/core.c=8435=static void balance_push(struct rq *rq)
--
kernel/sched/core.c-8458- if (kthread_is_per_cpu(push_task) ||
kernel/sched/core.c:8459: is_migration_disabled(push_task)) {
kernel/sched/core.c-8460-
--
kernel/sched/core.c=9233=void __cant_migrate(const char *file, int line)
--
kernel/sched/core.c-9239-
kernel/sched/core.c:9240: if (is_migration_disabled(current))
kernel/sched/core.c-9241- return;
--
kernel/sched/core.c-9253- pr_err("BUG: assuming non migratable context at %s:%d\n", file, line);
kernel/sched/core.c:9254: pr_err("in_atomic(): %d, irqs_disabled(): %d, migration_disabled() %u pid: %d, name: %s\n",
kernel/sched/core.c:9255: in_atomic(), irqs_disabled(), is_migration_disabled(current),
kernel/sched/core.c-9256- current->pid, current->comm);
--
kernel/sched/core.c=11345=void sched_push_current_non_preferred_cpu(struct rq *rq)
--
kernel/sched/core.c-11357-
kernel/sched/core.c:11358: if (is_migration_disabled(push_task))
kernel/sched/core.c-11359- return;
--
kernel/sched/deadline.c=3023=static struct task_struct *pick_next_pushable_dl_task(struct rq *rq)
--
kernel/sched/deadline.c-3034- /* skip tasks that cannot be migrated */
kernel/sched/deadline.c:3035: if (!task_on_cpu(rq, i) && !is_migration_disabled(i)) {
kernel/sched/deadline.c-3036- p = i;
--
kernel/sched/deadline.c=3057=static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq)
--
kernel/sched/deadline.c-3103- */
kernel/sched/deadline.c:3104: if (unlikely(is_migration_disabled(task) ||
kernel/sched/deadline.c-3105- !cpumask_test_cpu(later_rq->cpu, &task->cpus_mask) ||
--
kernel/sched/deadline.c=3141=static int push_dl_task(struct rq *rq)
--
kernel/sched/deadline.c-3163-
kernel/sched/deadline.c:3164: if (is_migration_disabled(next_task))
kernel/sched/deadline.c-3165- return 0;
--
kernel/sched/deadline.c=3221=static void pull_dl_task(struct rq *this_rq)
--
kernel/sched/deadline.c-3283-
kernel/sched/deadline.c:3284: if (is_migration_disabled(p)) {
kernel/sched/deadline.c-3285- push_task = get_push_task(src_rq);
--
kernel/sched/ext/ext.c=391=static struct scx_dispatch_q *bypass_enq_target_dsq(struct scx_sched *sch, struct rq *rq,
--
kernel/sched/ext/ext.c-421- if (unlikely(scx_missing_caps(host, cpu_of(rq), SCX_CAP_BASE)) &&
kernel/sched/ext/ext.c:422: (scx_rescue_enabled() || is_migration_disabled(p) || p->migration_pending)) {
kernel/sched/ext/ext.c-423- *enq_flags |= SCX_ENQ_RESCUE;
--
kernel/sched/ext/ext.c=2247=void scx_do_enqueue_task(struct rq *rq, struct task_struct *p, u64 enq_flags,
--
kernel/sched/ext/ext.c-2333- if (!(sch->ops.flags & SCX_OPS_ENQ_MIGRATION_DISABLED) &&
kernel/sched/ext/ext.c:2334: is_migration_disabled(p)) {
kernel/sched/ext/ext.c-2335- __scx_add_event(sch, SCX_EV_ENQ_SKIP_MIGRATION_DISABLED, 1);
--
kernel/sched/ext/ext.c=2814=static bool task_can_run_on_remote_rq(struct scx_sched *sch,
--
kernel/sched/ext/ext.c-2851- */
kernel/sched/ext/ext.c:2852: if (unlikely(is_migration_disabled(p))) {
kernel/sched/ext/ext.c-2853- if (enforce)
--
kernel/sched/ext/ext.c=2913=static bool task_proxy_unsafe_to_move(struct task_struct *p)
--
kernel/sched/ext/ext.c-2917-
kernel/sched/ext/ext.c:2918: return task_proxy_running_or_donating(p) || is_migration_disabled(p);
kernel/sched/ext/ext.c-2919-}
--
kernel/sched/ext/ext.c=7941=static void scx_dump_task(struct scx_sched *sch, struct seq_buf *s, struct scx_dump_ctx *dctx,
--
kernel/sched/ext/ext.c-7976- scx_dump_line(s, " cpus=%*pb no_mig=%u", cpumask_pr_args(p->cpus_ptr),
kernel/sched/ext/ext.c:7977: p->migration_disabled);
kernel/sched/ext/ext.c-7978-
--
kernel/sched/ext/idle.c=923=static bool check_builtin_idle_enabled(struct scx_sched *sch)
--
kernel/sched/ext/idle.c-935- *
kernel/sched/ext/idle.c:936: * We can't simply check whether @p->migration_disabled is set in a
kernel/sched/ext/idle.c-937- * sched_ext callback, because the BPF prolog (__bpf_prog_enter) may disable
--
kernel/sched/ext/idle.c-940- * Since the BPF prolog calls migrate_disable() only when CONFIG_PREEMPT_RCU
kernel/sched/ext/idle.c:941: * is enabled (via rcu_read_lock_dont_migrate()), migration_disabled == 1 for
kernel/sched/ext/idle.c-942- * the current task is ambiguous only in that case: it could be from the BPF
--
kernel/sched/ext/idle.c-945- * Without CONFIG_PREEMPT_RCU, the BPF prolog never calls migrate_disable(),
kernel/sched/ext/idle.c:946: * so migration_disabled == 1 always means the task is truly
kernel/sched/ext/idle.c-947- * migration-disabled.
kernel/sched/ext/idle.c-948- *
kernel/sched/ext/idle.c:949: * Therefore, when migration_disabled == 1 and CONFIG_PREEMPT_RCU is enabled,
kernel/sched/ext/idle.c-950- * check whether @p is the current task or not: if it is, then migration was
--
kernel/sched/ext/idle.c-954- */
kernel/sched/ext/idle.c:955:static bool is_bpf_migration_disabled(const struct task_struct *p)
kernel/sched/ext/idle.c-956-{
kernel/sched/ext/idle.c:957: if (p->migration_disabled == 1) {
kernel/sched/ext/idle.c-958- if (IS_ENABLED(CONFIG_PREEMPT_RCU))
--
kernel/sched/ext/idle.c-961- }
kernel/sched/ext/idle.c:962: return p->migration_disabled;
kernel/sched/ext/idle.c-963-}
--
kernel/sched/ext/idle.c=965=static s32 select_cpu_restricted(struct task_struct *p, s32 prev_cpu,
kernel/sched/ext/idle.c-966- const struct cpumask *allowed, u64 flags,
kernel/sched/ext/idle.c:967: bool migration_disabled)
kernel/sched/ext/idle.c-968-{
--
kernel/sched/ext/idle.c-970-
kernel/sched/ext/idle.c:971: if (migration_disabled)
kernel/sched/ext/idle.c-972- cpu = task_cpu(p);
--
kernel/sched/ext/idle.c=995=static s32 select_cpu_from_kfunc(struct scx_sched *sch, struct task_struct *p,
--
kernel/sched/ext/idle.c-1000- bool we_locked = false;
kernel/sched/ext/idle.c:1001: bool migration_disabled;
kernel/sched/ext/idle.c-1002- s32 cpu;
--
kernel/sched/ext/idle.c-1040- */
kernel/sched/ext/idle.c:1041: migration_disabled = is_bpf_migration_disabled(p);
kernel/sched/ext/idle.c-1042-
kernel/sched/ext/idle.c:1043: if (p->nr_cpus_allowed == 1 || migration_disabled)
kernel/sched/ext/idle.c-1044- cpu = select_cpu_restricted(p, prev_cpu, allowed, flags,
kernel/sched/ext/idle.c:1045: migration_disabled);
kernel/sched/ext/idle.c-1046- else
--
kernel/sched/ext/internal.h=2073=static const char *scx_enable_state_str[] = {
--
kernel/sched/ext/internal.h-2171- * 7. Otherwise, verify under src_rq lock that the task can be moved to dst_rq
]
|
| 65/2 |
2026/11/18 01:17 |
llm |
expert |
0m
Model:
Tokens:
input: 78061
output: 30
thoughts: 4874
Thoughts: **Analyzing a Potential Race Condition in Kernel Migration Disabling**
Okay, here's what I've been thinking through. KCSAN flagged a potential data race between a task reading `p->migration_disabled` and an interrupt writing to it, both on the same CPU. The key is understanding how `migration_disabled` is used, especially within the BPF prolog and interrupt context.
I started by reviewing the code around `is_bpf_migration_disabled` and noticed the ambiguity introduced when `CONFIG_PREEMPT_RCU` is enabled. It's possible for `migration_disabled` to be set not just by a direct call to `migrate_disable()` but also from the BPF prolog. This led me to carefully trace where the writes and reads occur.
The report clearly shows that both the read (by a task in `__migrate_enable`) and the write (by an interrupt, likely within `packet_rcv` via `migrate_disable`) happen on the *same* CPU. This is crucial: an interrupt *can* interrupt a task. I know that interrupts are disabled in very few places, and it's not the case during `__migrate_enable()` or `__migrate_disable()`. I need to understand if the compiler's optimizations introduce additional reads.
So, the question isn't whether an interrupt *can* occur, but *what happens* when it *does* occur at a very specific spot. I need to consider several scenarios:
* **Scenario A:** Task T executing `__migrate_enable()` with initial `migration_disabled == 1`. An interrupt fires *before* the decrement. If the compiler were to re-read and read `p->migration_disabled` TWICE, it may result in an incorrect value. I need to ensure that the interrupt handling and subsequent calls to `migrate_disable()` and `migrate_enable()` don't corrupt the value when the task T later resumes.
* **Scenario B:** An interrupt that happens *after* `p->migration_disabled = 0`. This scenario is related to `this_rq_pinned()`.
* **Scenario C:** Now, consider the case where `p->migration_disabled` is *initially* 0. An interrupt can, in this case, corrupt the expected value and make them mismatched.
* **Scenario D:** I have to remember that code reordering and re-reading of `p->migration_disabled` is possible with compiler optimizations.
Importantly, the BPF calls *always* pair `migrate_disable()` and `migrate_enable()`, making the race more subtle. However, the use of `guard(preempt)` and the affinity masks introduce more complex scenarios to cover. And what happens when an affinity change request gets in the middle of these.
Specifically, what's concerning is the case where a softirq (caused by an interrupt) can run between the checks and updates within `__migrate_enable()`, particularly the potential for `p->cpus_ptr != &p->cpus_mask` during the execution. I really need to drill down on `__set_cpus_allowed_ptr` and how it interacts. This potentially corrupts `p->migration_disabled` and `this_rq_pinned()` in an unpredictable way! It feels like a real bug, or it should be documented. I need to ensure that the code handles the reordering safely.
|
| 66/2 |
2026/11/18 01:18 |
tool |
grepper |
0m
Args: map[Expression:cpus_ptr = PathPrefixes:[kernel/sched/]]
Results: map[Output:kernel/sched/core.c=2820=void set_cpus_allowed_common(struct task_struct *p, struct affinity_context *ctx)
--
kernel/sched/core.c-2822- if (ctx->flags & (SCA_MIGRATE_ENABLE | SCA_MIGRATE_DISABLE)) {
kernel/sched/core.c:2823: p->cpus_ptr = ctx->new_mask;
kernel/sched/core.c-2824- return;
--
kernel/sched/core.c=2872=int dup_user_cpus_ptr(struct task_struct *dst, struct task_struct *src,
--
kernel/sched/core.c-2881- */
kernel/sched/core.c:2882: dst->user_cpus_ptr = NULL;
kernel/sched/core.c-2883-
]
|
| 67/2 |
2026/11/18 01:18 |
llm |
expert |
0m
Model:
Tokens:
input: 83929
output: 25
thoughts: 1260
Thoughts: **Analyzing the `migrate_disable_switch` and `migrate_enable` Interactions**
Okay, let's break this down. I'm starting by tracing where `migrate_disable_switch` is called. It looks like it's in `kernel/sched/core.c`, and it's invoked when a task, `p`, is being switched OUT, and crucially, `p->migration_disabled` is set. At this point, if migration is disabled, `p->cpus_ptr` gets set to `cpumask_of(rq->cpu)`. So, when the task is switched back in, this `cpus_ptr` *still* points to the original CPU. The task then runs on that CPU until it explicitly calls `migrate_enable()`.
Now, focusing on `__migrate_enable()`. The key here is the check: `if (p->cpus_ptr != &p->cpus_mask)`. Initially, after the switch back in, this condition *is* true, because of what we just saw. So, we end up in `___migrate_enable()`.
My immediate concern is: what if an interrupt happens *during* that `migrate_enable()` sequence, specifically *before* `___migrate_enable()` is executed? Remember that the `guard(preempt)()` call only disables *preemption*, not interrupts. Interrupts are still possible, and importantly, softirqs can run during an interrupt.
So, let's explore that scenario: An interrupt occurs. A softirq runs. What does the softirq do? The softirq calls `migrate_disable()`, which in turn calls `__migrate_disable()`. And in `__migrate_disable()`, if `p->migration_disabled` is not zero, the counter is *incremented*. In our scenario, if the task was preempted, `p->migration_disabled` would likely be 1 at this point. So the softirq would increment it to 2.
The softirq may then call `migrate_enable()`. In `__migrate_enable()`, it sees that `p->migration_disabled` is now 2, meaning the task is still migration-disabled from a previous call to `migrate_disable()`. So the softirq decrements it to 1, and crucially, *doesn't* call `___migrate_enable()`. It then returns.
When the original task resumes, it will then call `___migrate_enable()`, and everything seems okay, as far as that scenario goes. The task eventually restores `cpus_ptr`, and sets `p->migration_disabled` back to 0.
Wait, though! What if the task was running with `p->cpus_ptr != &p->cpus_mask` and `p->migration_disabled == 0`? No, no, that shouldn't happen. The counter has to be 1 or greater if the pointer is set in the way that it is.
The real question I have is, does `___migrate_enable()` or any of the functions it calls sleep? I'm worried about a scheduler call. Line 2463 talks about this:
> `/* * Ensure stop_task runs either before or after this, and that * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule(). */ guard(preempt)();`
That `guard(preempt)()` means we're safe, as `__set_cpus_allowed_ptr` (called by `___migrate_enable()`) shouldn't be able to schedule.
Okay, my main worry now is: Could *another* CPU be modifying `p->migration_disabled` concurrently? Can another CPU write to `p->migration_disabled`? No, it looks like `p->migration_disabled` is only ever modified by the current task! Okay, that's a relief. Let's make sure. Let's check for all writes to `migration_disabled` to make sure there are no other interactions.
|
| 68/2 |
2026/11/18 01:18 |
tool |
grepper |
0m
Args: map[Expression:migration_disabled(\+\+|--|\s*=)]
Results: map[Output:include/linux/sched.h=2444=static inline void __migrate_enable(void)
--
include/linux/sched.h-2457- if (p->migration_disabled > 1) {
include/linux/sched.h:2458: p->migration_disabled--;
include/linux/sched.h-2459- return;
--
include/linux/sched.h-2474- barrier();
include/linux/sched.h:2475: p->migration_disabled = 0;
include/linux/sched.h-2476- this_rq_pinned()--;
--
include/linux/sched.h=2479=static inline void __migrate_disable(void)
--
include/linux/sched.h-2489-#endif
include/linux/sched.h:2490: p->migration_disabled++;
include/linux/sched.h-2491- return;
--
include/linux/sched.h-2495- this_rq_pinned()++;
include/linux/sched.h:2496: p->migration_disabled = 1;
include/linux/sched.h-2497-}
--
kernel/sched/ext/idle.c=923=static bool check_builtin_idle_enabled(struct scx_sched *sch)
--
kernel/sched/ext/idle.c-940- * Since the BPF prolog calls migrate_disable() only when CONFIG_PREEMPT_RCU
kernel/sched/ext/idle.c:941: * is enabled (via rcu_read_lock_dont_migrate()), migration_disabled == 1 for
kernel/sched/ext/idle.c-942- * the current task is ambiguous only in that case: it could be from the BPF
--
kernel/sched/ext/idle.c-945- * Without CONFIG_PREEMPT_RCU, the BPF prolog never calls migrate_disable(),
kernel/sched/ext/idle.c:946: * so migration_disabled == 1 always means the task is truly
kernel/sched/ext/idle.c-947- * migration-disabled.
kernel/sched/ext/idle.c-948- *
kernel/sched/ext/idle.c:949: * Therefore, when migration_disabled == 1 and CONFIG_PREEMPT_RCU is enabled,
kernel/sched/ext/idle.c-950- * check whether @p is the current task or not: if it is, then migration was
--
kernel/sched/ext/idle.c=955=static bool is_bpf_migration_disabled(const struct task_struct *p)
kernel/sched/ext/idle.c-956-{
kernel/sched/ext/idle.c:957: if (p->migration_disabled == 1) {
kernel/sched/ext/idle.c-958- if (IS_ENABLED(CONFIG_PREEMPT_RCU))
--
kernel/sched/ext/idle.c=995=static s32 select_cpu_from_kfunc(struct scx_sched *sch, struct task_struct *p,
--
kernel/sched/ext/idle.c-1040- */
kernel/sched/ext/idle.c:1041: migration_disabled = is_bpf_migration_disabled(p);
kernel/sched/ext/idle.c-1042-
--
tools/sched_ext/include/scx/common.bpf.h=572=int scx_lib_init_probe(void *ctx)
--
tools/sched_ext/include/scx/common.bpf.h-578- if (md > 1)
tools/sched_ext/include/scx/common.bpf.h:579: bpf_printk("scx_lib_init_probe: unexpected migration_disabled=%u "
tools/sched_ext/include/scx/common.bpf.h-580- "upstream of BPF prolog; probe result unreliable",
--
tools/sched_ext/include/scx/common.bpf.h=599=static inline bool is_migration_disabled(const struct task_struct *p)
--
tools/sched_ext/include/scx/common.bpf.h-603- * (__bpf_prog_enter) may call migrate_disable() for the current task,
tools/sched_ext/include/scx/common.bpf.h:604: * making migration_disabled == 1 even for tasks that are not truly
tools/sched_ext/include/scx/common.bpf.h-605- * migration-disabled.
--
tools/sched_ext/include/scx/common.bpf.h-611- * 1) CONFIG_PREEMPT_RCU: prolog always calls migrate_disable(), so
tools/sched_ext/include/scx/common.bpf.h:612: * migration_disabled == 1 for the current task is ambiguous.
tools/sched_ext/include/scx/common.bpf.h-613- * Disambiguate by checking p == current.
--
tools/sched_ext/include/scx/common.bpf.h-615- * 2) v6.18+ without CONFIG_PREEMPT_RCU: prolog never calls
tools/sched_ext/include/scx/common.bpf.h:616: * migrate_disable(), so migration_disabled == 1 is unambiguously
tools/sched_ext/include/scx/common.bpf.h-617- * a real migrate_disable() call.
--
tools/sched_ext/include/scx/common.bpf.h-625- if (bpf_core_field_exists(p->migration_disabled)) {
tools/sched_ext/include/scx/common.bpf.h:626: if (p->migration_disabled == 1) {
tools/sched_ext/include/scx/common.bpf.h-627- /* Fast path: prolog always disables migration */
]
|
| 69/2 |
2026/11/18 01:18 |
llm |
expert |
0m
Model:
Tokens:
input: 87614
output: 36
thoughts: 8391
Thoughts: **Analysis of Potential Data Race in `migration_disabled` within BPF Context**
Okay, so the `p->migration_disabled` variable is only ever touched by the currently running task on a CPU, specifically through `__migrate_disable()` and `__migrate_enable()` in the scheduler. Other CPUs don't get to modify this value directly. I initially thought, "aha, data race!" But I've got to look closer. The KCSAN report flagged a potential data race on the *same* CPU: an interrupt handler writing to `current->migration_disabled` while a task (5152) is reading it. Crucially, both are happening on CPU 0.
So, the read occurs during a `bpf_flow_dissect` call from the task 5152 and the write comes from a softirq within the interrupt. Both calls are calling `bpf_prog_run_pin_on_cpu` which calls both `migrate_disable()` and `migrate_enable()`. KCSAN's complaining because they are accessing the same memory location (`p->migration_disabled`) without proper synchronization.
The compiler optimizations matter! What if the compiler splits the `p->migration_disabled > 1` check into a load/compare/jump sequence? A softirq could intervene between the compare and the decrement, leading to an incorrect value being written back. This also applies when setting `migration_disabled` to 1 or 0 and an interrupt occurs. The interrupt itself is the trigger.
Here's a breakdown of the scenarios:
1. If the task is running and is preempted right before `migrate_disable()` and an interrupt occurs, the interrupt will increase the value of `migration_disabled`. Once the interrupt finishes the value will return back to its original state.
2. If the task is running and is interrupted right before `migrate_enable()` and an interrupt occurs, the interrupt will increase the value of `migration_disabled`. Once the interrupt finishes the value will be incorrect and it will need to be checked.
Itβs about the interleaving and the fact that an interrupt handler can also call `migrate_disable()` and `migrate_enable()`, especially when BPF is involved via something like `bpf_prog_run_pin_on_cpu`. This is happening because the BPF program can be triggered via a network packet.
Now, *is* this a real bug? KCSAN is correct in identifying it as a data race under the C memory model. Interrupt handlers are, in effect, asynchronous signals. The question is, does this lead to *corruption* or a problem in practice?
Let me investigate `this_rq_pinned()`, `guard(preempt)` and other locks that are involved, and where and when preemption is enabled. If I can prove that in some specific, common interleaving, the logic of the scheduler *fails* due to the race, then yes, it's a critical bug. Otherwise, it might be a benign data race that's reported by KCSAN but doesn't cause functional problems. I need to dig deeper into the scheduler logic and how `migrate_disable`/`enable` interacts with task affinity changes, and whether a potential race affects `this_rq_pinned()` values, or `p->cpus_ptr` which leads to the task's affinity. I should search for existing discussions or patches relating to this specific interaction.
|
| 70/2 |
2026/11/18 01:19 |
tool |
git-log |
5m
Args: map[CodeRegexp:bpf_prog_run_pin_on_cpu Since:4 years]
Results: map[Output:79511603a65b tls: remove dead sockmap (psock) handling from the SW path
785d4625d3e0 io_uring/bpf_filter: use bpf_prog_run_pin_on_cpu() to prevent migration
17ce3e5949bc bpf: Disable migration in nf_hook_run_bpf().
]
|
| 71/2 |
2026/11/18 01:25 |
llm |
expert |
0m
Model:
Tokens:
input: 96882
output: 27
thoughts: 13
Thoughts: Okay, here's my summarized thought process, assuming I'm an expert and this is a typical task:
**Analyzing Commit Messages for `migration_disabled`**
Alright, let's see... I need to find instances where `migration_disabled` is mentioned within commit messages. This likely indicates some form of system or application state related to database migrations being temporarily or permanently halted. This is pretty straightforward, I'll need to use my usual tools to dig through the commit history. I'm thinking I'll use `git log` with a `grep` or similar to quickly identify relevant commits. I'm looking for a pattern, a string match, likely with some context around it to understand *why* the migration was disabled. Was it a temporary issue during an upgrade? A specific bug that needed to be addressed? Or a long-term decision? Understanding the context will be key here. I'll need to parse the results carefully to get a good overview. No doubt the results will be noisy, so I'll need to keep an eye out for false positives. Let's see what we can find!
|
| 72/2 |
2026/11/18 01:25 |
tool |
git-log |
0m
Args: map[MessageRegexps:[migration_disabled] Since:4 years]
Results: map[Output:7a9df7667963 sched_ext: Fix stall when a task enqueued with SCX_ENQ_LAST lands back on its CPU's local DSQ
e0bb53501771 sched_ext: Sync common and compat headers from the scx repo
dae5c0292080 sched/rt,dl: Skip migrate-disabled tasks when picking a push candidate
9e8581a090c0 sched_ext: Sync common and compat headers from the scx repo
5c94a3ab6ed9 sched_ext: Check remote rq eligibility under task's rq lock
d8c897b20bf4 lockdep/selftests: Restore migrate_disable() state on PREEMPT_RT
7c7bb206e87d tools/sched_ext: Handle migration-disabled tasks in scx_central
0c4a59df370b sched_ext: Fix is_bpf_migration_disabled() false negative on non-PREEMPT_RCU
55ed11b181c4 sched_ext: idle: Handle migration-disabled tasks in BPF code
de68c05189cc tools/sched_ext: Receive updates from SCX repo
]
|
| 73/2 |
2026/11/18 01:25 |
llm |
expert |
0m
Model:
Tokens:
input: 97460
output: 35
thoughts: 22
Thoughts: Okay, here's my interpretation of those thoughts, tailored for an expert audience, and delivered as a first-person summary:
**Digging into Interrupts and Migration Disabling**
Alright, I'm thinking we need to delve into the commit history. The goal here is to identify any potential interactions or changes related to interrupt handling and processor migration. Specifically, I'm going to search for commits that contain the terms `migrate_disable`, `interrupt`, or `softirq`. These keywords likely point to areas where we might find relevant code modifications or bug fixes. My intuition is that these terms together could unveil areas where interrupt behavior and processor affinity might interact in unexpected ways. This could be crucial for performance troubleshooting, understanding specific architectural behavior, or identifying potential optimization opportunities within our system. This is where I'd start my investigation.
|
| 74/2 |
2026/11/18 01:25 |
tool |
git-log |
0m
Args: map[MessageRegexps:[migrate_disable softirq|interrupt] Since:4 years]
Results:
|
| 75/2 |
2026/11/18 01:25 |
llm |
expert |
0m
Model:
Tokens:
input: 97703
output: 28
thoughts: 400
Thoughts: **Analyzing the Safety of `migrate_disable()` in Interrupt Context**
Okay, first things first, let's see if we should involve CodeExpert here. Wait, I have to remember the instructions. We can't ask it about syzkaller or syzlang specifically. So let's see, should I use this tool for a question about whether `migrate_disable()` is intended/safe to be called from a softirq/interrupt context? And, more importantly, whether the data race on `current->migration_disabled` can actually cause corruption.
Hmm, let's think this through carefully before asking. *Can* `migrate_disable()` and `migrate_enable()` even be called in softirq context? Okay, now, let's consider PREEMPT_RT. In PREEMPT_RT, softirqs are threaded, so does that change things? They're handled by `ksoftirqd/X` or per-softirq threads. That's right! But what about non-RT kernels?
Okay, let's look at `migrate_disable()` itself:
Wait a second, there's `this_rq_pinned()`! That's a key. What exactly is `this_rq_pinned()`? Ah, it's equivalent to `this_rq()->nr_pinned`. Okay, now what's the purpose of `nr_pinned` within `struct rq`? I need to understand this. Better to check where `nr_pinned` is used in the kernel more broadly.
|
| 76/2 |
2026/11/18 01:25 |
tool |
grepper |
0m
Args: map[Expression:nr_pinned PathPrefixes:[kernel/sched/]]
Results: map[Output:kernel/sched/core.c=2512=static inline bool rq_has_pinned_tasks(struct rq *rq)
kernel/sched/core.c-2513-{
kernel/sched/core.c:2514: return rq->nr_pinned;
kernel/sched/core.c-2515-}
--
kernel/sched/rq-offsets.c=7=int main(void)
kernel/sched/rq-offsets.c-8-{
kernel/sched/rq-offsets.c:9: DEFINE(RQ_nr_pinned, offsetof(struct rq, nr_pinned));
kernel/sched/rq-offsets.c-10-
--
kernel/sched/sched.h=1179=struct rq {
--
kernel/sched/sched.h-1386-
kernel/sched/sched.h:1387: unsigned int nr_pinned;
kernel/sched/sched.h-1388- unsigned int push_busy;
]
|
| 77/2 |
2026/11/18 01:25 |
llm |
expert |
0m
Model:
Tokens:
input: 98707
output: 32
thoughts: 40
Thoughts: Okay, here's my interpretation of this thought process, keeping in mind my expertise:
**Investigating the Usage of `rq->nr_pinned` in the Linux Kernel's Scheduler**
Alright, so I'm digging into the scheduler code again, and I've come across this `rq->nr_pinned` variable. My initial understanding is that it's only ever *read* within the `rq_has_pinned_tasks(struct rq *rq)` function. That's a key observation, a good starting point for analysis. Now, the next question immediately springs to mind, and it's fundamental: *where* is `rq_has_pinned_tasks` itself being called? Knowing the call sites of that function will give me crucial context. This should tell me what operations are interested in pinned tasks. Understanding the broader use of `nr_pinned` is the next, logical step in dissecting this part of the scheduler's behavior. I need to trace the control flow to determine the conditions under which a task is considered pinned.
|
| 78/2 |
2026/11/18 01:26 |
tool |
grepper |
0m
Args: map[Expression:rq_has_pinned_tasks PathPrefixes:[kernel/sched/]]
Results: map[Output:kernel/sched/core.c=2510=EXPORT_SYMBOL_GPL(migrate_enable);
kernel/sched/core.c-2511-
kernel/sched/core.c:2512:static inline bool rq_has_pinned_tasks(struct rq *rq)
kernel/sched/core.c-2513-{
--
kernel/sched/core.c=8435=static void balance_push(struct rq *rq)
--
kernel/sched/core.c-8471- */
kernel/sched/core.c:8472: if (!rq->nr_running && !rq_has_pinned_tasks(rq) &&
kernel/sched/core.c-8473- rcuwait_active(&rq->hotplug_wait)) {
--
kernel/sched/core.c=8520=static void balance_hotplug_wait(void)
--
kernel/sched/core.c-8524- rcuwait_wait_event(&rq->hotplug_wait,
kernel/sched/core.c:8525: rq->nr_running == 1 && !rq_has_pinned_tasks(rq),
kernel/sched/core.c-8526- TASK_UNINTERRUPTIBLE);
--
kernel/sched/core.c=8833=int sched_cpu_dying(unsigned int cpu)
--
kernel/sched/core.c-8842- update_rq_clock(rq);
kernel/sched/core.c:8843: if (rq->nr_running != 1 || rq_has_pinned_tasks(rq)) {
kernel/sched/core.c-8844- WARN(true, "Dying CPU not properly vacated!");
]
|
| 79/2 |
2026/11/18 01:26 |
llm |
expert |
0m
Model:
Tokens:
input: 99441
output: 162
thoughts: 673
Thoughts: **Analyzing Kernel Migration Disabling/Enabling**
Okay, so I'm looking at this `rq->nr_pinned` usage for CPU hotplug, specifically through `rq_has_pinned_tasks(rq)`. The warning "Dying CPU not properly vacated!" triggers if `rq_has_pinned_tasks(rq)` is non-zero during the CPU's death. That's a key observation.
Now, I'm digging deeper into `__migrate_disable()` and `__migrate_enable()`. The code's using `migration_disabled` within the `task_struct`. I see the increment/decrement logic and the checks for potential overflows, especially under `CONFIG_DEBUG_PREEMPT`. The `guard(preempt)()` calls seem important, ensuring some sort of atomic-ish execution or mutual exclusion, and the `cpus_ptr` shenanigans are also worth watching. The `barrier()` before clearing `migration_disabled` is another significant detail to note. It clearly shows the intention to sequence the memory operations correctly. The associated `this_rq_pinned()` operations are another side-effect to consider.
Now, a critical question pops up: can I safely call `migrate_disable()` and `migrate_enable()` from an interrupt or softirq context, particularly through these BPF-related functions, `bpf_prog_run_pin_on_cpu` or `bpf_prog_run_clear_cb` within `packet_rcv` or `run_filter`? The interaction between task execution and softirqs within the same CPU looks like a potential minefield. What if a task on CPU 0 is interrupted by a softirq on CPU 0 that *also* calls these migration functions, while the task itself is executing either `migrate_enable()` or `migrate_disable()`? That's a data race on `current->migration_disabled`, a classic concurrency issue. Is it harmful, is it benign? I'm going to formulate a clear question for CodeExpert to delve deeper.
|
| 80/2 |
2026/11/18 01:26 |
tool |
codeexpert |
15m
Args: map[Question:Can migrate_disable() / migrate_enable() be called from softirq/interrupt context?
Specifically, in packet_rcv -> run_filter -> bpf_prog_run_clear_cb -> bpf_prog_run_pin_on_cpu -> migrate_disable()/migrate_enable():
When a task on CPU 0 is interrupted by an interrupt/softirq that calls migrate_disable() and migrate_enable(), both operating on current->migration_disabled (e.g. while the task was executing __migrate_enable()), is this data race harmful or benign? Can current->migration_disabled or rq->nr_pinned get corrupted or lead to any failure/WARN_ON?]
Results: map[Answer:Based on analysis of the Linux kernel scheduler (`kernel/sched/core.c`, `include/linux/sched.h`) and BPF filter implementations (`include/linux/filter.h`, `net/packet/af_packet.c`):
### 1. Can `migrate_disable()` / `migrate_enable()` be called from softirq/interrupt context?
**No, they are not intended or safe to be called from softirq or interrupt context.**
* `migrate_disable()` and `migrate_enable()` are per-task primitives designed strictly for preemptible task context. They operate on `current->migration_disabled`. In softirq and interrupt contexts, `current` is simply whichever arbitrary task was interrupted by the IRQ on that CPU.
* Furthermore, softirqs and interrupt handlers are already inherently bound to the CPU executing them and cannot migrate across CPUs. Invoking `migrate_disable()` in softirq context is redundant and violates the design contract of `migrate_disable()`.
* The call chain `packet_rcv -> run_filter -> bpf_prog_run_clear_cb -> bpf_prog_run_pin_on_cpu` mistakenly invokes `migrate_disable()` / `migrate_enable()` in softirq context because `bpf_prog_run_clear_cb()` unconditionally uses `bpf_prog_run_pin_on_cpu()`.
---
### 2. Is this race harmful or benign? Can `current->migration_disabled` or `rq->nr_pinned` get corrupted or lead to `WARN_ON`?
**The race is HARMFUL. It can corrupt `rq->nr_pinned`, corrupt `current->migration_disabled`, and trigger `WARN_ON_ONCE`.**
Specific failure mechanisms:
1. **`rq->nr_pinned` Underflow / Corruption:**
In `__migrate_enable()`:
```c
barrier();
p->migration_disabled = 0;
/* <-- WINDOW OF VULNERABILITY */
this_rq_pinned()--;
```
`guard(preempt)()` disables preemption, but does *not* disable hardware interrupts. If a hardirq interrupts the CPU right after `p->migration_disabled = 0;` but before `this_rq_pinned()--;`, softirq processing (e.g. on `irq_exit()`) can run `packet_rcv`:
- Inside the softirq, `migrate_disable()` inspects `p = current`. Because `p->migration_disabled` was already set to 0 by the task, it takes the slow path, increments `this_rq_pinned()++`, and sets `p->migration_disabled = 1`.
- When the BPF filter finishes, `migrate_enable()` in the softirq sets `p->migration_disabled = 0` and decrements `this_rq_pinned()--`.
- Control returns to the interrupted task, which now executes its pending `this_rq_pinned()--;`.
- **Result:** `this_rq_pinned()` (i.e., `rq->nr_pinned`) is decremented one extra time without a matching increment. Since `rq->nr_pinned` is an `unsigned int`, it underflows to `UINT_MAX` (4,294,967,295), permanently corrupting the runqueue's pinned task count and breaking scheduler migration/load-balancing logic.
2. **`current->migration_disabled` Corruption via Non-Atomic Updates:**
- `migration_disabled` is an `unsigned short` modified with plain `++` and `--` without atomic operations or interrupt disabling.
- If a softirq interrupts while the task is executing `p->migration_disabled--` (when nesting count > 1) or `p->migration_disabled++`, concurrent modification causes lost updates or incorrect values.
3. **`WARN_ON_ONCE` Triggers:**
- With `CONFIG_DEBUG_PREEMPT=y`, `__migrate_enable()` contains:
```c
if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
return;
```
and `__migrate_disable()` checks:
```c
WARN_ON_ONCE((s16)p->migration_disabled < 0);
```
Underflow or desynchronization between the interrupted task and the softirq will trigger these warnings.
4. **Inadvertent Affinity Restoration (`___migrate_enable()`):**
- If the interrupted task had `p->cpus_ptr != &p->cpus_mask` (e.g. pending affinity change while migration was disabled), `migrate_enable()` running in the softirq will call `___migrate_enable()`, which calls `__set_cpus_allowed_ptr()` with task/runqueue locks from softirq context for the interrupted task.
---
### 3. Caveats and Unverified Parts
- Due to research tool limits, we did not verify whether a specific fix patch for `bpf_prog_run_clear_cb` / `packet_rcv` has been merged in the very latest upstream git HEAD or mailing lists (e.g., whether `bpf_prog_run_clear_cb` should use `bpf_prog_run` instead of `bpf_prog_run_pin_on_cpu` when in softirq context).
- On `PREEMPT_RT`, softirqs run in dedicated per-CPU ksoftirqd threads where `current` is the ksoftirqd task rather than the interrupted user/kernel thread; the analysis above applies primarily to standard (non-RT) kernels where softirqs run in interrupt context interrupting arbitrary tasks.]
|
| 81/3 |
2026/11/18 01:26 |
agent |
codeexpert |
15m
Model:
Results: map[Answer:Based on analysis of the Linux kernel scheduler (`kernel/sched/core.c`, `include/linux/sched.h`) and BPF filter implementations (`include/linux/filter.h`, `net/packet/af_packet.c`):
### 1. Can `migrate_disable()` / `migrate_enable()` be called from softirq/interrupt context?
**No, they are not intended or safe to be called from softirq or interrupt context.**
* `migrate_disable()` and `migrate_enable()` are per-task primitives designed strictly for preemptible task context. They operate on `current->migration_disabled`. In softirq and interrupt contexts, `current` is simply whichever arbitrary task was interrupted by the IRQ on that CPU.
* Furthermore, softirqs and interrupt handlers are already inherently bound to the CPU executing them and cannot migrate across CPUs. Invoking `migrate_disable()` in softirq context is redundant and violates the design contract of `migrate_disable()`.
* The call chain `packet_rcv -> run_filter -> bpf_prog_run_clear_cb -> bpf_prog_run_pin_on_cpu` mistakenly invokes `migrate_disable()` / `migrate_enable()` in softirq context because `bpf_prog_run_clear_cb()` unconditionally uses `bpf_prog_run_pin_on_cpu()`.
---
### 2. Is this race harmful or benign? Can `current->migration_disabled` or `rq->nr_pinned` get corrupted or lead to `WARN_ON`?
**The race is HARMFUL. It can corrupt `rq->nr_pinned`, corrupt `current->migration_disabled`, and trigger `WARN_ON_ONCE`.**
Specific failure mechanisms:
1. **`rq->nr_pinned` Underflow / Corruption:**
In `__migrate_enable()`:
```c
barrier();
p->migration_disabled = 0;
/* <-- WINDOW OF VULNERABILITY */
this_rq_pinned()--;
```
`guard(preempt)()` disables preemption, but does *not* disable hardware interrupts. If a hardirq interrupts the CPU right after `p->migration_disabled = 0;` but before `this_rq_pinned()--;`, softirq processing (e.g. on `irq_exit()`) can run `packet_rcv`:
- Inside the softirq, `migrate_disable()` inspects `p = current`. Because `p->migration_disabled` was already set to 0 by the task, it takes the slow path, increments `this_rq_pinned()++`, and sets `p->migration_disabled = 1`.
- When the BPF filter finishes, `migrate_enable()` in the softirq sets `p->migration_disabled = 0` and decrements `this_rq_pinned()--`.
- Control returns to the interrupted task, which now executes its pending `this_rq_pinned()--;`.
- **Result:** `this_rq_pinned()` (i.e., `rq->nr_pinned`) is decremented one extra time without a matching increment. Since `rq->nr_pinned` is an `unsigned int`, it underflows to `UINT_MAX` (4,294,967,295), permanently corrupting the runqueue's pinned task count and breaking scheduler migration/load-balancing logic.
2. **`current->migration_disabled` Corruption via Non-Atomic Updates:**
- `migration_disabled` is an `unsigned short` modified with plain `++` and `--` without atomic operations or interrupt disabling.
- If a softirq interrupts while the task is executing `p->migration_disabled--` (when nesting count > 1) or `p->migration_disabled++`, concurrent modification causes lost updates or incorrect values.
3. **`WARN_ON_ONCE` Triggers:**
- With `CONFIG_DEBUG_PREEMPT=y`, `__migrate_enable()` contains:
```c
if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
return;
```
and `__migrate_disable()` checks:
```c
WARN_ON_ONCE((s16)p->migration_disabled < 0);
```
Underflow or desynchronization between the interrupted task and the softirq will trigger these warnings.
4. **Inadvertent Affinity Restoration (`___migrate_enable()`):**
- If the interrupted task had `p->cpus_ptr != &p->cpus_mask` (e.g. pending affinity change while migration was disabled), `migrate_enable()` running in the softirq will call `___migrate_enable()`, which calls `__set_cpus_allowed_ptr()` with task/runqueue locks from softirq context for the interrupted task.
---
### 3. Caveats and Unverified Parts
- Due to research tool limits, we did not verify whether a specific fix patch for `bpf_prog_run_clear_cb` / `packet_rcv` has been merged in the very latest upstream git HEAD or mailing lists (e.g., whether `bpf_prog_run_clear_cb` should use `bpf_prog_run` instead of `bpf_prog_run_pin_on_cpu` when in softirq context).
- On `PREEMPT_RT`, softirqs run in dedicated per-CPU ksoftirqd threads where `current` is the ksoftirqd task rather than the interrupted user/kernel thread; the analysis above applies primarily to standard (non-RT) kernels where softirqs run in interrupt context interrupting arbitrary tasks.]
Instruction:
You are a capable Linux kernel developer tasked with researching complex questions
about kernel source code. You will be given a concrete question, and need to provide
a concrete answer.
Research the question using targeted, efficient tool calls. Don't make assumptions,
or rely on your previous knowledge about the kernel source code, use available tools
to access the actual source code. Keep your tool calls focused and limited to the
scope of the question. Do not explore tangential code paths or perform unbounded searches.
Use all available sources of information:
- kernel source code
- documentation in the Documentation dir in the source tree
- git commits descriptions, git blame
Do not guess file names or file paths and attempt to read them without
verifying their existence first using content search or directory listing tools.
If a file, symbol, or directory is not found via content search (grepper) or
directory listing (codesearch-dir-index), treat it as completely absent.
Do not attempt to guess alternative names, extensions, or directories.
Tool Selection Guidelines:
1. For C functions, structs, and variables: Prefer codesearch-definition-source
or codesearch-struct-layout FIRST to retrieve clean, exact definitions
without line-number guessing.
2. If symbol lookup fails (e.g., preprocessor macros, macro-generated code, or
disabled #ifdef branches), fall back to read-file or grepper.
3. For file headers, #include directives, preprocessor macro definitions, and
non-C files (Kconfig, Makefiles, docs): Use read-file or grepper directly.
4. Be concise and targeted. Answer the question as directly and concisely as possible
once you have located the relevant code or definitions. If a symbol or function cannot
be found after targeted searches, conclude with the best available information rather
than continuing repetitive searches.
Do NOT use git-log to search for the presence or existence of files in the
repository. git-log is only for tracing commit history of files that are
already present in the current checkout. If a file does not exist in the
current checkout, it cannot be used for reproduction.
Avoid running broad git-log queries (such as searches on the entire repo)
without a specific 'PathPrefix' to restrict the scope, as these are very
expensive and will time out.
If a git-log tool call times out, do not retry the query with the same broad
scope. You must specify a tighter, narrower 'PathPrefix' for subsequent queries.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Can migrate_disable() / migrate_enable() be called from softirq/interrupt context?
Specifically, in packet_rcv -> run_filter -> bpf_prog_run_clear_cb -> bpf_prog_run_pin_on_cpu -> migrate_disable()/migrate_enable():
When a task on CPU 0 is interrupted by an interrupt/softirq that calls migrate_disable() and migrate_enable(), both operating on current->migration_disabled (e.g. while the task was executing __migrate_enable()), is this data race harmful or benign? Can current->migration_disabled or rq->nr_pinned get corrupted or lead to any failure/WARN_ON?
|
| 82/4 |
2026/11/18 01:26 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 4433
output: 109
thoughts: 202
Thoughts: Okay, let's break this down.
**Analyzing Potential Data Races in Kernel Migration Operations**
Alright, the question's pretty specific: Can calls to `migrate_disable()` and `migrate_enable()` cause issues if they're invoked from within an interrupt or softirq context? Specifically, the concern arises in this `bpf_prog_run_pin_on_cpu` path, down through `packet_rcv`, `run_filter`, etc. The scenario is: a task on CPU 0 gets interrupted by something β an interrupt or softirq β and that interrupt/softirq then calls the migration functions, potentially while the original task was *also* in the process of calling `migrate_enable()` (or disabling). My gut reaction is that this *feels* like a potential race condition, but we need to verify. My primary concern is if there's any corruption of `current->migration_disabled` or `rq->nr_pinned`, or a trigger for a `WARN_ON`.
My next step is clear: I need to get into the kernel source. I'll need to meticulously inspect the implementations of `migrate_disable()` and `migrate_enable()`. I want to see how these functions interact with the global state, how they manipulate `current->migration_disabled` and anything relating to the runqueue (`rq`) and how pinned tasks are managed. I'll be looking for any uses of spinlocks, atomic operations, or other synchronization primitives that might be involved. I'll need to trace the control flow carefully to understand if the interrupt context is adequately protected from the original task's execution. Ultimately, the question is whether the architecture safely handles the concurrent modifications of these critical data structures.
|
| 83/4 |
2026/11/18 01:26 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:kernel/sched/core.c Name:migrate_disable]
Results: map[SourceCode:2500: void migrate_disable(void)
2501: {
2502: __migrate_disable();
2503: }
SourceFile:kernel/sched/core.c]
|
| 84/4 |
2026/11/18 01:26 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:kernel/sched/core.c Name:migrate_enable]
Results: map[SourceCode:2506: void migrate_enable(void)
2507: {
2508: __migrate_enable();
2509: }
SourceFile:kernel/sched/core.c]
|
| 85/4 |
2026/11/18 01:26 |
tool |
grepper |
0m
Args: map[Expression:void migrate_disable|void migrate_enable PathPrefixes:[kernel/sched/ include/linux/]]
Results: map[Output:include/linux/sched.h=2500=static inline void __migrate_enable(void) { }
--
include/linux/sched.h-2509-#ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
include/linux/sched.h:2510:static __always_inline void migrate_disable(void)
include/linux/sched.h-2511-{
--
include/linux/sched.h-2514-
include/linux/sched.h:2515:static __always_inline void migrate_enable(void)
include/linux/sched.h-2516-{
--
include/linux/sched.h-2519-#else /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
include/linux/sched.h:2520:extern void migrate_disable(void);
include/linux/sched.h:2521:extern void migrate_enable(void);
include/linux/sched.h-2522-#endif /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
--
include/linux/sched.h-2524-#else /* MODULE */
include/linux/sched.h:2525:extern void migrate_disable(void);
include/linux/sched.h:2526:extern void migrate_enable(void);
include/linux/sched.h-2527-#endif /* MODULE */
--
kernel/sched/core.c=2469=do_set_cpus_allowed(struct task_struct *p, struct affinity_context *ctx);
kernel/sched/core.c-2470-
kernel/sched/core.c:2471:static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
kernel/sched/core.c-2472-{
--
kernel/sched/core.c=2498=EXPORT_SYMBOL_GPL(___migrate_enable);
kernel/sched/core.c-2499-
kernel/sched/core.c:2500:void migrate_disable(void)
kernel/sched/core.c-2501-{
--
kernel/sched/core.c=2504=EXPORT_SYMBOL_GPL(migrate_disable);
kernel/sched/core.c-2505-
kernel/sched/core.c:2506:void migrate_enable(void)
kernel/sched/core.c-2507-{
]
|
| 86/4 |
2026/11/18 01:26 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 5851
output: 72
thoughts: 73
Thoughts: Here's my take:
**Investigating the Migration Disable/Enable Mechanisms**
Alright, let's dive into the guts of the scheduler. I need to refresh my memory on how the migration disable and enable mechanisms actually function. My first stop will be `include/linux/sched.h`, specifically around lines 2490 to 2530. I'm hoping to get a clear picture of the basic definitions of `__migrate_disable`, `__migrate_enable`, and those related symbols. Understanding the data structures involved is crucial.
Then, I'll jump over to the implementation, which should be in `kernel/sched/core.c`. I'll be targeting the vicinity of lines 2450 to 2520, where I expect to find the actual code that implements those disable and enable functions. I need to see how the kernel interacts with the task's state. I need to be sure that I remember how the atomic operations are working.
|
| 87/4 |
2026/11/18 01:26 |
tool |
read-file |
0m
Args: map[File:include/linux/sched.h FirstLine:2480 LineCount:60]
Results: map[Contents:2480: {
2481: struct task_struct *p = current;
2482:
2483: if (p->migration_disabled) {
2484: #ifdef CONFIG_DEBUG_PREEMPT
2485: /*
2486: *Warn about overflow half-way through the range.
2487: */
2488: WARN_ON_ONCE((s16)p->migration_disabled < 0);
2489: #endif
2490: p->migration_disabled++;
2491: return;
2492: }
2493:
2494: guard(preempt)();
2495: this_rq_pinned()++;
2496: p->migration_disabled = 1;
2497: }
2498: #else /* !COMPILE_OFFSETS */
2499: static inline void __migrate_disable(void) { }
2500: static inline void __migrate_enable(void) { }
2501: #endif /* !COMPILE_OFFSETS */
2502:
2503: /*
2504: * So that it is possible to not export the runqueues variable, define and
2505: * export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
2506: * them for the modules. The macro "INSTANTIATE_EXPORTED_MIGRATE_DISABLE" will
2507: * be defined in kernel/sched/core.c.
2508: */
2509: #ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
2510: static __always_inline void migrate_disable(void)
2511: {
2512: __migrate_disable();
2513: }
2514:
2515: static __always_inline void migrate_enable(void)
2516: {
2517: __migrate_enable();
2518: }
2519: #else /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
2520: extern void migrate_disable(void);
2521: extern void migrate_enable(void);
2522: #endif /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
2523:
2524: #else /* MODULE */
2525: extern void migrate_disable(void);
2526: extern void migrate_enable(void);
2527: #endif /* MODULE */
2528:
2529: DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
2530:
2531: #endif
]
|
| 88/4 |
2026/11/18 01:26 |
tool |
read-file |
0m
Args: map[File:kernel/sched/core.c FirstLine:2440 LineCount:70]
Results: map[Contents:2440: /*
2441: * It's not enough that it's not actively running,
2442: * it must be off the runqueue _entirely_, and not
2443: * preempted!
2444: *
2445: * So if it was still runnable (but just not actively
2446: * running right now), it's preempted, and we should
2447: * yield - it could be a while.
2448: */
2449: if (unlikely(queued)) {
2450: ktime_t to = NSEC_PER_SEC / HZ;
2451:
2452: set_current_state(TASK_UNINTERRUPTIBLE);
2453: schedule_hrtimeout(&to, HRTIMER_MODE_REL_HARD);
2454: continue;
2455: }
2456:
2457: /*
2458: * Ahh, all good. It wasn't running, and it wasn't
2459: * runnable, which means that it will never become
2460: * running in the future either. We're all done!
2461: */
2462: break;
2463: }
2464:
2465: return ncsw;
2466: }
2467:
2468: static void
2469: do_set_cpus_allowed(struct task_struct *p, struct affinity_context *ctx);
2470:
2471: static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
2472: {
2473: struct affinity_context ac = {
2474: .new_mask = cpumask_of(rq->cpu),
2475: .flags = SCA_MIGRATE_DISABLE,
2476: };
2477:
2478: if (likely(!p->migration_disabled))
2479: return;
2480:
2481: if (p->cpus_ptr != &p->cpus_mask)
2482: return;
2483:
2484: scoped_guard (task_rq_lock, p)
2485: do_set_cpus_allowed(p, &ac);
2486: }
2487:
2488: void ___migrate_enable(void)
2489: {
2490: struct task_struct *p = current;
2491: struct affinity_context ac = {
2492: .new_mask = &p->cpus_mask,
2493: .flags = SCA_MIGRATE_ENABLE,
2494: };
2495:
2496: __set_cpus_allowed_ptr(p, &ac);
2497: }
2498: EXPORT_SYMBOL_GPL(___migrate_enable);
2499:
2500: void migrate_disable(void)
2501: {
2502: __migrate_disable();
2503: }
2504: EXPORT_SYMBOL_GPL(migrate_disable);
2505:
2506: void migrate_enable(void)
2507: {
2508: __migrate_enable();
2509: }
]
|
| 89/4 |
2026/11/18 01:26 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 7877
output: 36
thoughts: 28
Thoughts: Okay, let's dive into it.
**Delving into the Migration Enable Function**
Right, so to truly understand what's happening with this `__migrate_enable()` function, I need to go back to the source. Specifically, I need to navigate into `include/linux/sched.h`. That's where the core scheduling definitions reside, and it's where this function's declaration or definition *should* be. I'm expecting to find the nitty-gritty details of how this migration enabling process is architected. Time to go spelunking! I want to understand the exact implementation before drawing any conclusions.
|
| 90/4 |
2026/11/18 01:26 |
tool |
read-file |
0m
Args: map[File:include/linux/sched.h FirstLine:2440 LineCount:45]
Results: map[Contents:2440: #define this_rq_raw() PERCPU_PTR(&runqueues)
2441: #endif
2442: #define this_rq_pinned() (*(unsigned int *)((void *)this_rq_raw() + RQ_nr_pinned))
2443:
2444: static inline void __migrate_enable(void)
2445: {
2446: struct task_struct *p = current;
2447:
2448: #ifdef CONFIG_DEBUG_PREEMPT
2449: /*
2450: * Check both overflow from migrate_disable() and superfluous
2451: * migrate_enable().
2452: */
2453: if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
2454: return;
2455: #endif
2456:
2457: if (p->migration_disabled > 1) {
2458: p->migration_disabled--;
2459: return;
2460: }
2461:
2462: /*
2463: * Ensure stop_task runs either before or after this, and that
2464: * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
2465: */
2466: guard(preempt)();
2467: if (unlikely(p->cpus_ptr != &p->cpus_mask))
2468: ___migrate_enable();
2469: /*
2470: * Mustn't clear migration_disabled() until cpus_ptr points back at the
2471: * regular cpus_mask, otherwise things that race (eg.
2472: * select_fallback_rq) get confused.
2473: */
2474: barrier();
2475: p->migration_disabled = 0;
2476: this_rq_pinned()--;
2477: }
2478:
2479: static inline void __migrate_disable(void)
2480: {
2481: struct task_struct *p = current;
2482:
2483: if (p->migration_disabled) {
2484: #ifdef CONFIG_DEBUG_PREEMPT
]
|
| 91/4 |
2026/11/18 01:26 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 8718
output: 26
thoughts: 95
Thoughts: Okay, here's my assessment of the situation:
**Diving into the Kernel Scheduler and BPF Interaction**
Alright, that's interesting. I've spotted something significant. My attention is immediately drawn to lines 2444-2497 within `include/linux/sched.h`. That's where things are likely to get interesting concerning the scheduler. I'm thinking of how the BPF programs interact with this core scheduler. I want to understand exactly how the scheduling decisions are made, especially when BPF programs get involved.
Specifically, I need to understand the connection between `bpf_prog_run_pin_on_cpu`, `run_filter`, `bpf_prog_run_clear_cb` and the network receive function `packet_rcv`. They have to be linked somewhere, I need to know the mechanism.
I'm zeroing in on `bpf_prog_run_pin_on_cpu` as a starting point. That seems like a critical piece of the puzzle, and a good place to start my investigation. I need to understand the function's purpose within this workflow.
|
| 92/4 |
2026/11/18 01:26 |
tool |
grepper |
0m
Args: map[Expression:bpf_prog_run_pin_on_cpu]
Results: map[Output:include/linux/filter.h=818=static __always_inline u32 bpf_prog_run(const struct bpf_prog *prog, const void *ctx)
--
include/linux/filter.h-830- */
include/linux/filter.h:831:static inline u32 bpf_prog_run_pin_on_cpu(const struct bpf_prog *prog,
include/linux/filter.h-832- const void *ctx)
--
include/linux/filter.h=1110=static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
--
include/linux/filter.h-1118-
include/linux/filter.h:1119: res = bpf_prog_run_pin_on_cpu(prog, skb);
include/linux/filter.h-1120- return res;
--
io_uring/bpf_filter.c=57=int __io_uring_run_bpf_filters(struct io_bpf_filter __rcu **filters,
--
io_uring/bpf_filter.c-87- return -EACCES;
io_uring/bpf_filter.c:88: ret = bpf_prog_run_pin_on_cpu(filter->prog, &bpf_ctx);
io_uring/bpf_filter.c-89- if (!ret)
--
kernel/seccomp.c=406=static u32 seccomp_run_filters(const struct seccomp_data *sd,
--
kernel/seccomp.c-425- for (; f; f = f->prev) {
kernel/seccomp.c:426: u32 cur_ret = bpf_prog_run_pin_on_cpu(f->prog, sd);
kernel/seccomp.c-427-
--
net/bpf/test_run.c=1684=int bpf_prog_test_run_syscall(struct bpf_prog *prog,
--
net/bpf/test_run.c-1711- rcu_read_lock_trace();
net/bpf/test_run.c:1712: retval = bpf_prog_run_pin_on_cpu(prog, ctx);
net/bpf/test_run.c-1713- rcu_read_unlock_trace();
--
net/core/flow_dissector.c=1004=u32 bpf_flow_dissect(struct bpf_prog *prog, struct bpf_flow_dissector *ctx,
--
net/core/flow_dissector.c-1023-
net/core/flow_dissector.c:1024: result = bpf_prog_run_pin_on_cpu(prog, ctx);
net/core/flow_dissector.c-1025-
--
net/core/skmsg.c=920=int sk_psock_msg_verdict(struct sock *sk, struct sk_psock *psock,
--
net/core/skmsg.c-934- msg->sk = sk;
net/core/skmsg.c:935: ret = bpf_prog_run_pin_on_cpu(prog, msg);
net/core/skmsg.c-936- msg->sk = NULL;
--
net/core/skmsg.c=1079=static void sk_psock_strp_read(struct strparser *strp, struct sk_buff *skb)
--
net/core/skmsg.c-1097- skb_bpf_redirect_clear(skb);
net/core/skmsg.c:1098: ret = bpf_prog_run_pin_on_cpu(prog, skb);
net/core/skmsg.c-1099- skb_bpf_set_strparser(skb);
--
net/core/skmsg.c=1113=static int sk_psock_strp_parse(struct strparser *strp, struct sk_buff *skb)
--
net/core/skmsg.c-1122- skb->sk = psock->sk;
net/core/skmsg.c:1123: ret = bpf_prog_run_pin_on_cpu(prog, skb);
net/core/skmsg.c-1124- skb->sk = NULL;
--
net/core/skmsg.c=1202=static int sk_psock_verdict_recv(struct sock *sk, struct sk_buff *skb)
--
net/core/skmsg.c-1222- skb_bpf_redirect_clear(skb);
net/core/skmsg.c:1223: ret = bpf_prog_run_pin_on_cpu(prog, skb);
net/core/skmsg.c-1224- ret = sk_psock_map_verd(ret, skb_bpf_redirect_fetch(skb));
--
net/kcm/kcmsock.c=389=static int kcm_parse_func_strparser(struct strparser *strp, struct sk_buff *skb)
--
net/kcm/kcmsock.c-395- rcu_read_lock();
net/kcm/kcmsock.c:396: res = bpf_prog_run_pin_on_cpu(prog, skb);
net/kcm/kcmsock.c-397- rcu_read_unlock();
--
net/netfilter/nf_bpf_link.c=11=static unsigned int nf_hook_run_bpf(void *bpf_prog, struct sk_buff *skb,
--
net/netfilter/nf_bpf_link.c-19-
net/netfilter/nf_bpf_link.c:20: return bpf_prog_run_pin_on_cpu(prog, &ctx);
net/netfilter/nf_bpf_link.c-21-}
]
|
| 93/4 |
2026/11/18 01:26 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 10617
output: 35
thoughts: 20
Thoughts: Here's a first-person summary, tailored for an expert audience, based on your instructions:
**Diving into BPF Filtering**
Okay, time to get my hands dirty. Let's see what's actually going on under the hood with BPF filtering. I'm thinking a good starting point to refresh my memory would be the `include/linux/filter.h` file. Specifically, I'm going to jump in around line 830. I'm interested in seeing how the kernel defines and interacts with the various BPF instructions and helper functions. I need to understand the data structures and macros in this area to correctly debug a potential issue I've been seeing in a production environment. I'm going to pay close attention to the way the BPF programs are constructed and how the kernel validates them, to see if anything is immediately apparent to me. I'm expecting some standard definitions for instructions and perhaps some structures to represent program blocks. Let's dig in.
|
| 94/4 |
2026/11/18 01:26 |
tool |
read-file |
0m
Args: map[File:include/linux/filter.h FirstLine:820 LineCount:40]
Results: map[Contents: 820: return __bpf_prog_run(prog, ctx, bpf_dispatcher_nop_func);
821: }
822:
823: /*
824: * Use in preemptible and therefore migratable context to make sure that
825: * the execution of the BPF program runs on one CPU.
826: *
827: * This uses migrate_disable/enable() explicitly to document that the
828: * invocation of a BPF program does not require reentrancy protection
829: * against a BPF program which is invoked from a preempting task.
830: */
831: static inline u32 bpf_prog_run_pin_on_cpu(const struct bpf_prog *prog,
832: const void *ctx)
833: {
834: u32 ret;
835:
836: migrate_disable();
837: ret = bpf_prog_run(prog, ctx);
838: migrate_enable();
839: return ret;
840: }
841:
842: static inline bool is_stack_arg_ldx(const struct bpf_insn *insn)
843: {
844: return insn->code == (BPF_LDX | BPF_MEM | BPF_DW) &&
845: insn->src_reg == BPF_REG_PARAMS &&
846: insn->off > 0 && insn->off % 8 == 0;
847: }
848:
849: static inline bool is_stack_arg_st(const struct bpf_insn *insn)
850: {
851: return insn->code == (BPF_ST | BPF_MEM | BPF_DW) &&
852: insn->dst_reg == BPF_REG_PARAMS &&
853: insn->off < 0 && insn->off % 8 == 0;
854: }
855:
856: static inline bool is_stack_arg_stx(const struct bpf_insn *insn)
857: {
858: return insn->code == (BPF_STX | BPF_MEM | BPF_DW) &&
859: insn->dst_reg == BPF_REG_PARAMS &&
]
|
| 95/4 |
2026/11/18 01:26 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 11563
output: 28
thoughts: 68
Thoughts: Okay, here's my take on those thoughts, framed as my own:
**Investigating BPF Program Execution in a Preemptible Context**
Alright, let's break this down. The comment at lines 823-830 emphasizes the need to keep a BPF program running on a single CPU, particularly in a preemptible environment, which screams migration concerns. The core idea is to prevent the program from jumping around CPUs, which can introduce all sorts of performance and coherence issues. My immediate thought is, "Okay, that makes sense." We want to avoid any context switching across CPUs when a packet is processed.
Now, that brings up a key question: Does `packet_rcv` even *call* `run_filter`? The comment's relevance hinges on that. If `packet_rcv` is the entry point for packet processing, and `run_filter` is the BPF program execution, there needs to be a call path. I can't just assume. I need to trace the flow. I need to verify how `run_filter` is integrated into the packet reception path. So, let's dive into `packet_rcv`. Let's get the context, see the call graph, and confirm the relationship. It's the only way to be sure if this comment applies.
|
| 96/4 |
2026/11/18 01:27 |
tool |
grepper |
0m
Args: map[Expression:run_filter PathPrefixes:[net/packet/]]
Results: map[Output:net/packet/af_packet.c=1971=static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg,
--
net/packet/af_packet.c-2101-
net/packet/af_packet.c:2102:static unsigned int run_filter(struct sk_buff *skb,
net/packet/af_packet.c-2103- const struct sock *sk,
--
net/packet/af_packet.c=2144=static int packet_rcv(struct sk_buff *skb, struct net_device *dev,
--
net/packet/af_packet.c-2183-
net/packet/af_packet.c:2184: res = run_filter(skb, sk, snaplen);
net/packet/af_packet.c-2185- if (!res)
--
net/packet/af_packet.c=2257=static int tpacket_rcv(struct sk_buff *skb, struct net_device *dev,
--
net/packet/af_packet.c-2303-
net/packet/af_packet.c:2304: res = run_filter(skb, sk, snaplen);
net/packet/af_packet.c-2305- if (!res)
]
|
| 97/4 |
2026/11/18 01:27 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 12283
output: 38
thoughts: 21
Thoughts: Here's a summary of my thought process:
**Diving into the `af_packet.c` Filter**
Okay, so I need to understand how this packet filtering is working in the networking stack. My initial thought is to zero in on `run_filter` within `net/packet/af_packet.c`. That function is probably the heart of how packets are evaluated against any user-defined BPF filters, or any other kind of filter that is being used, for that matter. I want to see exactly how the filter bytecode gets executed, how the data gets accessed, what registers are involved, and how the results determine whether a packet is accepted or dropped. I need to refresh my understanding of the execution path, potential performance bottlenecks, and any areas where optimizations might be possible. This will be my starting point to understand the overall packet flow and filter processing.
|
| 98/4 |
2026/11/18 01:27 |
tool |
read-file |
0m
Args: map[File:net/packet/af_packet.c FirstLine:2100 LineCount:40]
Results: map[Contents:2100: }
2101:
2102: static unsigned int run_filter(struct sk_buff *skb,
2103: const struct sock *sk,
2104: unsigned int res)
2105: {
2106: struct sk_filter *filter;
2107:
2108: rcu_read_lock();
2109: filter = rcu_dereference(sk->sk_filter);
2110: if (filter != NULL)
2111: res = bpf_prog_run_clear_cb(filter->prog, skb);
2112: rcu_read_unlock();
2113:
2114: return res;
2115: }
2116:
2117: static int packet_rcv_vnet(struct msghdr *msg, const struct sk_buff *skb,
2118: size_t *len, int vnet_hdr_sz)
2119: {
2120: struct virtio_net_hdr_mrg_rxbuf vnet_hdr = { .num_buffers = 0 };
2121:
2122: if (*len < vnet_hdr_sz)
2123: return -EINVAL;
2124: *len -= vnet_hdr_sz;
2125:
2126: if (virtio_net_hdr_from_skb(skb, (struct virtio_net_hdr *)&vnet_hdr, vio_le(), true, 0))
2127: return -EINVAL;
2128:
2129: return memcpy_to_msg(msg, (void *)&vnet_hdr, vnet_hdr_sz);
2130: }
2131:
2132: /*
2133: * This function makes lazy skb cloning in hope that most of packets
2134: * are discarded by BPF.
2135: *
2136: * Note tricky part: we DO mangle shared skb! skb->data, skb->len
2137: * and skb->cb are mangled. It works because (and until) packets
2138: * falling here are owned by current CPU. Output packets are cloned
2139: * by dev_queue_xmit_nit(), input packets are processed by net_bh
]
|
| 99/4 |
2026/11/18 01:27 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 13169
output: 31
thoughts: 331
Thoughts: **Analyzing a Potential Concurrency Issue in Kernel Packet Processing**
Okay, let's break down this call trace: `packet_rcv` is the starting point, leading into `run_filter`, then `bpf_prog_run_clear_cb`, `bpf_prog_run_pin_on_cpu`, and finally hitting `migrate_disable()` and `migrate_enable()`. That makes sense; `packet_rcv` is right in the heart of packet reception, happening within a softirq (specifically, `NET_RX_SOFTIRQ`, and called from things like `net_rx_action` or `napi_poll`), or even packet transmission paths like `dev_queue_xmit_nit`. The critical point the user is raising is whether those calls to `migrate_disable()`/`migrate_enable()` within this packet processing path, which can exist in softirq or even process context, are problematic.
Essentially, the question is whether a data race could occur. The scenario they're concerned about is a task on CPU 0 being interrupted by an interrupt or softirq. If that interrupt/softirq then calls `migrate_disable()` and `migrate_enable()`, both acting on `current->migration_disabled` while the task was, perhaps, in the middle of a `__migrate_enable()` operation, is there a risk of corruption? Could it corrupt `current->migration_disabled` or the `rq->nr_pinned` counter, potentially leading to errors or even a `WARN_ON()`?
Given the sensitivity of those migration primitives, I need to check if this is even plausible. I'm going to dig into the git log for any past discussions, commits, bug reports, or lore entries related to `migrate_disable`, `bpf_prog_run_pin_on_cpu`, and all related components. This smells like a subtle concurrency issue that someone might have tripped over before. Let's see if there's any prior art on this!
|
| 100/4 |
2026/11/18 01:27 |
tool |
git-log |
0m
Args: map[Count:10 MessageRegexps:[migrate_disable softirq]]
Results: map[Output:1872e75375c4 bpf: Fix race in devmap on PREEMPT_RT
869c63d5975d bpf: Fix race in cpumap on PREEMPT_RT
]
|
| 101/4 |
2026/11/18 01:27 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 14022
output: 34
thoughts: 33
Thoughts: Here's my take on those thoughts:
**Analyzing Kernel Commit Messages for eBPF and Migration Constraints**
Okay, so I need to get a handle on how these recent kernel changes might affect eBPF program execution, specifically around CPU pinning and migration constraints. My initial thought is to dive into the commit messages. The keywords that come immediately to mind are related to the core functionality I'm interested in: I need to search for occurrences of `bpf_prog_run_pin_on_cpu`. This likely points directly to changes in the mechanisms for running pinned eBPF programs. I also have to consider the system-level impacts. Therefore I'll incorporate a search for `migration_disabled`, since this would indicate attempts to circumvent this behavior. Finally, I'll need to know about the status of pinned programs in general, so a search for `nr_pinned` seems like a solid inclusion. This should give me a good overview of the relevant modifications and hopefully allow me to infer any potential edge cases or performance implications introduced.
|
| 102/4 |
2026/11/18 01:27 |
tool |
git-log |
0m
Args: map[Count:10 MessageRegexps:[bpf_prog_run_pin_on_cpu]]
Results: map[Output:b0346dd64e49 net: kcm: Hold RCU read lock while running BPF parser
31e2f36d3821 bpf, sockmap: reject a packet-modifying SK_SKB stream parser
a48802fb2cd2 bpf, sockmap: fix integer overflow in bpf_msg_pop_data() bounds check
f3f34ca45b96 bpf, sockmap: Fix wrong rsge offset in bpf_msg_push_data()
3c94f241f776 udp: clear skb->dev before running a sockmap verdict
f6606a44bc43 bpf, arm64: Emit BTI for indirect jump target
785d4625d3e0 io_uring/bpf_filter: use bpf_prog_run_pin_on_cpu() to prevent migration
17ce3e5949bc bpf: Disable migration in nf_hook_run_bpf().
4580f4e0ebdf bpf: Fix deadlock between rcu_tasks_trace and event_mutex.
22c7fa171a02 bpf: Reject variable offset alu on PTR_TO_FLOW_KEYS
]
|
| 103/4 |
2026/11/18 01:27 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 14614
output: 34
thoughts: 19
Thoughts: Okay, here's my summarized thought process, presented as if I'm thinking it:
**Digging into the Scheduler's Migration Disable Logic**
Alright, time to dive in. I need to understand how `migration_disabled` is being handled within the kernel's scheduler. My first move? I'm going to conduct a targeted search. I'm going to leverage my familiarity with the kernel source code organization and pinpoint the relevant area. I know the scheduler code lives within `kernel/sched/`, so that's where I'll focus the initial search. And I'm going to specifically look for instances where `migration_disabled` is mentioned in commit messages. Why commit messages? Well, they often provide valuable context around why a change was made, what the intent was, and any potential implications. They can be a goldmine of information, especially when dealing with such fundamental aspects of the scheduler. This should give me a solid foundation for understanding the use and purpose of `migration_disabled` within the scheduler. This focused approach will hopefully help me quickly understand its use within the kernel.
|
| 104/4 |
2026/11/18 01:28 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[migration_disabled] PathPrefix:kernel/sched]
Results: map[Output:7a9df7667963 sched_ext: Fix stall when a task enqueued with SCX_ENQ_LAST lands back on its CPU's local DSQ
dae5c0292080 sched/rt,dl: Skip migrate-disabled tasks when picking a push candidate
5c94a3ab6ed9 sched_ext: Check remote rq eligibility under task's rq lock
0c4a59df370b sched_ext: Fix is_bpf_migration_disabled() false negative on non-PREEMPT_RCU
55ed11b181c4 sched_ext: idle: Handle migration-disabled tasks in BPF code
06efc9fe0b8d sched_ext: idle: Handle migration-disabled tasks in idle selection
9fd5da7989ba sched/smp: Use the SMP version of is_migration_disabled()
cc39454c341e sched_ext: Remove scx_ops_enq_* static_keys
f3f08c3acfb8 sched_ext: Fix incorrect assumption about migration disabled tasks in task_can_run_on_remote_rq()
eace54dff051 sched_ext: Add SCX_EV_ENQ_SKIP_MIGRATION_DISABLED
32966821574c sched_ext: Fix migration disabled handling in targeted dispatches
2fa0fbeb69ed sched_ext: Implement auto local dispatching of migration disabled tasks
0366017e0973 sched_ext: Use task_can_run_on_remote_rq() test in dispatch_to_local_dsq()
0ec8d5aed4d1 sched/core: Add WARN_ON_ONCE() to check overflow for migrate_disable()
feffe5bb274d sched/rt: Fix bad task migration for rt tasks
9d0df3779745 sched: Trigger warning if ->migration_disabled counter underflows.
]
|
| 105/4 |
2026/11/18 01:28 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 15375
output: 38
thoughts: 26
Thoughts: Okay, let's dive into this. Here's how I'd approach that request, keeping in mind my own deep understanding of the Linux kernel internals:
**Investigating the Evolution of `__migrate_enable` in `include/linux/sched.h`**
Alright, my internal systems are firing up. Let's get to the heart of this. The user wants to see the commit history specifically for modifications to `__migrate_enable` within `include/linux/sched.h`. That's a very targeted request, and I understand the significance.
I immediately start thinking about how to efficiently query the repository. I'll need to use `git log` or a similar tool. My initial approach is to craft the appropriate command to filter down to that file and, within that file, focus on lines that contain the `__migrate_enable` symbol. I'll probably start with the more obvious grep and git calls, ensuring the output is readable.
I know `include/linux/sched.h` is a core file in the kernel scheduler. Therefore, any changes to a migration enable function have likely had a ripple effect across core scheduling logic and potentially the NUMA/migration systems. I anticipate that this might lead to relevant patches, especially those touching memory management, task placement, and load balancing.
I'll be looking for several key things as I review the commit history:
* **The reasoning behind the change:** Was it a bug fix? An optimization? An architectural improvement? Understanding the "why" is crucial.
* **The context of the change:** What other code did the patch touch? Did it introduce new dependencies or modify existing ones? I know that even small changes can affect a huge area.
* **Performance implications:** Did the change claim any performance benefits or regressions? I need to analyze any performance test data to see if any claims were made.
* **Compatibility concerns:** Did the change impact older kernels or require any special handling?
* **The reviewers:** I will be checking who reviewed the patch for any insights into its quality.
I'm prepared to dig deep into the commit messages and patch diffs. This is my kind of challenge; understanding the "why" behind kernel changes is what I thrive on. I'll need to sift through the data carefully, looking for those key insights that explain the evolution of `__migrate_enable`. I'm ready to learn.
|
| 106/4 |
2026/11/18 01:28 |
tool |
git-log |
0m
Args: map[Count:20 MessageRegexps:[migrate_enable] PathPrefix:include/linux/sched.h]
Results: map[Output:14a7f2392f42 bpf: Make migrate_disable always inline to avoid partial inlining
378b7708194f sched: Make migrate_{en,dis}able() inline
6d337eab041d sched: Fix migrate_disable() vs set_cpus_allowed_ptr()
]
|
| 107/4 |
2026/11/18 01:28 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 16042
output: 27
thoughts: 58
Thoughts: Okay, let's dive into this code. I want to take a look at these commits specifically. I'll start by using `git show` on commit `14a7f2392f42`. Then, I'll move on to `378b7708194f`, and finally, I'll examine `6d337eab041d`. I want to understand the changes made in those specific revisions, and how they relate to the bigger picture. This will give me a very close view of the recent project history.
|
| 108/4 |
2026/11/18 01:28 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 14a7f2392f42bbb71c1a5ea68930006221fcd80a
Author: Yonghong Song <yonghong.song@linux.dev>
Date: Wed Oct 29 11:36:46 2025 -0700
bpf: Make migrate_disable always inline to avoid partial inlining
The build fails with llvm 21/22:
$ make LLVM=1 -j
...
LD vmlinux.o
GEN .vmlinux.objs
...
BTF .tmp_vmlinux1.btf.o
...
AS .tmp_vmlinux2.kallsyms.o
LD vmlinux.unstripped
BTFIDS vmlinux.unstripped
WARN: resolve_btfids: unresolved symbol migrate_enable
WARN: resolve_btfids: unresolved symbol migrate_disable
make[2]: *** [vmlinux.unstripped] Error 255
make[2]: *** Deleting file 'vmlinux.unstripped'
make[1]: *** [Makefile:1242: vmlinux] Error 2
make: *** [Makefile:248: __sub-make] Error 2
Two functions with identical names but different addresses are
considered ambiguous and removed by "pahole" from vmlinux BTF.
Later resolve_btfids warns since it cannot find them.
Commit 378b7708194f ("sched: Make migrate_{en,dis}able() inline") made
them inlineable in most places, but in vmlinux built with llvm 21 and 22
there are four symbols for migrate_{enable,disable}:
three static functions and one global function.
Fix the issue by marking migrate_{enable,disable} as always inline.
The alternative is to mark them as notrace/nokprobe which is more
drastic. Only bpf programs are prevented from attaching to these
functions. The rest of the tracing shouldn't be affected.
[note: Peter ok-ed the patch, Alexei rewrote commit log]
Fixes: 378b7708194f ("sched: Make migrate_{en,dis}able() inline")
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Acked-by: Menglong Dong <menglong.dong@linux.dev>
Link: https://lore.kernel.org/r/20251029183646.3811774-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
diff --git a/include/linux/sched.h b/include/linux/sched.h
index cbb7340c5866..b469878de25c 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2407,12 +2407,12 @@ static inline void __migrate_enable(void) { }
* be defined in kernel/sched/core.c.
*/
#ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
-static inline void migrate_disable(void)
+static __always_inline void migrate_disable(void)
{
__migrate_disable();
}
-static inline void migrate_enable(void)
+static __always_inline void migrate_enable(void)
{
__migrate_enable();
}
]
|
| 109/4 |
2026/11/18 01:28 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 17156
output: 27
thoughts: 32
Thoughts: Here's my analysis of the provided text, rendered in the first person, assuming an expert user:
**Initial Assessment: Code Object Inspection**
Okay, let's dive right in. The text is prompting me to examine two specific code objects, identified by their seemingly arbitrary hexadecimal IDs: `378b7708194f` and `6d337eab041d`. Given the nature of my work, I'm already mentally running through potential scenarios. Are these hashes representing versions of source code, object files, database records, network packets, memory addresses, or something entirely different? The context is missing, but the core task is clear: I need to investigate what information is associated with these identifiers. Depending on the environment, my next steps could range from checking a version control system to decompiling a binary, to looking up entries in a security log. Time to find out what kind of system I'm working with.
|
| 110/4 |
2026/11/18 01:28 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 378b7708194fff77c9020392067329931c3fcc04
Author: Menglong Dong <menglong8.dong@gmail.com>
Date: Wed Sep 17 14:09:15 2025 +0800
sched: Make migrate_{en,dis}able() inline
For now, migrate_enable and migrate_disable are global, which makes them
become hotspots in some case. Take BPF for example, the function calling
to migrate_enable and migrate_disable in BPF trampoline can introduce
significant overhead, and following is the 'perf top' of FENTRY's
benchmark (./tools/testing/selftests/bpf/bench trig-fentry):
54.63% bpf_prog_2dcccf652aac1793_bench_trigger_fentry [k]
bpf_prog_2dcccf652aac1793_bench_trigger_fentry
10.43% [kernel] [k] migrate_enable
10.07% bpf_trampoline_6442517037 [k] bpf_trampoline_6442517037
8.06% [kernel] [k] __bpf_prog_exit_recur
4.11% libc.so.6 [.] syscall
2.15% [kernel] [k] entry_SYSCALL_64
1.48% [kernel] [k] memchr_inv
1.32% [kernel] [k] fput
1.16% [kernel] [k] _copy_to_user
0.73% [kernel] [k] bpf_prog_test_run_raw_tp
So in this commit, we make migrate_enable/migrate_disable inline to obtain
better performance. The struct rq is defined internally in
kernel/sched/sched.h, and the field "nr_pinned" is accessed in
migrate_enable/migrate_disable, which makes it hard to make them inline.
Alexei Starovoitov suggests to generate the offset of "nr_pinned" in [1],
so we can define the migrate_enable/migrate_disable in
include/linux/sched.h and access "this_rq()->nr_pinned" with
"(void *)this_rq() + RQ_nr_pinned".
The offset of "nr_pinned" is generated in include/generated/rq-offsets.h
by kernel/sched/rq-offsets.c.
Generally speaking, we move the definition of migrate_enable and
migrate_disable to include/linux/sched.h from kernel/sched/core.c. The
calling to __set_cpus_allowed_ptr() is leaved in ___migrate_enable().
The "struct rq" is not available in include/linux/sched.h, so we can't
access the "runqueues" with this_cpu_ptr(), as the compilation will fail
in this_cpu_ptr() -> raw_cpu_ptr() -> __verify_pcpu_ptr():
typeof((ptr) + 0)
So we introduce the this_rq_raw() and access the runqueues with
arch_raw_cpu_ptr/PERCPU_PTR directly.
The variable "runqueues" is not visible in the kernel modules, and export
it is not a good idea. As Peter Zijlstra advised in [2], we define and
export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
them for the modules.
Before this patch, the performance of BPF FENTRY is:
fentry : 113.030 Β± 0.149M/s
fentry : 112.501 Β± 0.187M/s
fentry : 112.828 Β± 0.267M/s
fentry : 115.287 Β± 0.241M/s
After this patch, the performance of BPF FENTRY increases to:
fentry : 143.644 Β± 0.670M/s
fentry : 149.764 Β± 0.362M/s
fentry : 149.642 Β± 0.156M/s
fentry : 145.263 Β± 0.221M/s
Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/bpf/CAADnVQ+5sEDKHdsJY5ZsfGDO_1SEhhQWHrt2SMBG5SYyQ+jt7w@mail.gmail.com/ [1]
Link: https://lore.kernel.org/all/20250819123214.GH4067720@noisy.programming.kicks-ass.net/ [2]
diff --git a/Kbuild b/Kbuild
index f327ca86990c..13324b4bbe23 100644
--- a/Kbuild
+++ b/Kbuild
@@ -34,13 +34,24 @@ arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file)
$(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s FORCE
$(call filechk,offsets,__ASM_OFFSETS_H__)
+# Generate rq-offsets.h
+
+rq-offsets-file := include/generated/rq-offsets.h
+
+targets += kernel/sched/rq-offsets.s
+
+kernel/sched/rq-offsets.s: $(offsets-file)
+
+$(rq-offsets-file): kernel/sched/rq-offsets.s FORCE
+ $(call filechk,offsets,__RQ_OFFSETS_H__)
+
# Check for missing system calls
quiet_cmd_syscalls = CALL $<
cmd_syscalls = $(CONFIG_SHELL) $< $(CC) $(c_flags) $(missing_syscalls_flags)
PHONY += missing-syscalls
-missing-syscalls: scripts/checksyscalls.sh $(offsets-file)
+missing-syscalls: scripts/checksyscalls.sh $(rq-offsets-file)
$(call cmd,syscalls)
# Check the manual modification of atomic headers
diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index 1fad1c8a4c76..92237c319035 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -424,8 +424,6 @@ static inline void preempt_notifier_init(struct preempt_notifier *notifier,
* work-conserving schedulers.
*
*/
-extern void migrate_disable(void);
-extern void migrate_enable(void);
/**
* preempt_disable_nested - Disable preemption inside a normally preempt disabled section
@@ -471,7 +469,6 @@ static __always_inline void preempt_enable_nested(void)
DEFINE_LOCK_GUARD_0(preempt, preempt_disable(), preempt_enable())
DEFINE_LOCK_GUARD_0(preempt_notrace, preempt_disable_notrace(), preempt_enable_notrace())
-DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
#ifdef CONFIG_PREEMPT_DYNAMIC
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 644a01bdae70..d60ecaccdffc 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -49,6 +49,9 @@
#include <linux/tracepoint-defs.h>
#include <linux/unwind_deferred_types.h>
#include <asm/kmap_size.h>
+#ifndef COMPILE_OFFSETS
+#include <generated/rq-offsets.h>
+#endif
/* task_struct member predeclarations (sorted alphabetically): */
struct audit_context;
@@ -2317,4 +2320,114 @@ static __always_inline void alloc_tag_restore(struct alloc_tag *tag, struct allo
#define alloc_tag_restore(_tag, _old) do {} while (0)
#endif
+#ifndef MODULE
+#ifndef COMPILE_OFFSETS
+
+extern void ___migrate_enable(void);
+
+struct rq;
+DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
+
+/*
+ * The "struct rq" is not available here, so we can't access the
+ * "runqueues" with this_cpu_ptr(), as the compilation will fail in
+ * this_cpu_ptr() -> raw_cpu_ptr() -> __verify_pcpu_ptr():
+ * typeof((ptr) + 0)
+ *
+ * So use arch_raw_cpu_ptr()/PERCPU_PTR() directly here.
+ */
+#ifdef CONFIG_SMP
+#define this_rq_raw() arch_raw_cpu_ptr(&runqueues)
+#else
+#define this_rq_raw() PERCPU_PTR(&runqueues)
+#endif
+#define this_rq_pinned() (*(unsigned int *)((void *)this_rq_raw() + RQ_nr_pinned))
+
+static inline void __migrate_enable(void)
+{
+ struct task_struct *p = current;
+
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ * Check both overflow from migrate_disable() and superfluous
+ * migrate_enable().
+ */
+ if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
+ return;
+#endif
+
+ if (p->migration_disabled > 1) {
+ p->migration_disabled--;
+ return;
+ }
+
+ /*
+ * Ensure stop_task runs either before or after this, and that
+ * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
+ */
+ guard(preempt)();
+ if (unlikely(p->cpus_ptr != &p->cpus_mask))
+ ___migrate_enable();
+ /*
+ * Mustn't clear migration_disabled() until cpus_ptr points back at the
+ * regular cpus_mask, otherwise things that race (eg.
+ * select_fallback_rq) get confused.
+ */
+ barrier();
+ p->migration_disabled = 0;
+ this_rq_pinned()--;
+}
+
+static inline void __migrate_disable(void)
+{
+ struct task_struct *p = current;
+
+ if (p->migration_disabled) {
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ *Warn about overflow half-way through the range.
+ */
+ WARN_ON_ONCE((s16)p->migration_disabled < 0);
+#endif
+ p->migration_disabled++;
+ return;
+ }
+
+ guard(preempt)();
+ this_rq_pinned()++;
+ p->migration_disabled = 1;
+}
+#else /* !COMPILE_OFFSETS */
+static inline void __migrate_disable(void) { }
+static inline void __migrate_enable(void) { }
+#endif /* !COMPILE_OFFSETS */
+
+/*
+ * So that it is possible to not export the runqueues variable, define and
+ * export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
+ * them for the modules. The macro "INSTANTIATE_EXPORTED_MIGRATE_DISABLE" will
+ * be defined in kernel/sched/core.c.
+ */
+#ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
+static inline void migrate_disable(void)
+{
+ __migrate_disable();
+}
+
+static inline void migrate_enable(void)
+{
+ __migrate_enable();
+}
+#else /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
+extern void migrate_disable(void);
+extern void migrate_enable(void);
+#endif /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
+
+#else /* MODULE */
+extern void migrate_disable(void);
+extern void migrate_enable(void);
+#endif /* MODULE */
+
+DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
+
#endif
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c4f69a9e9af6..de9078a9df3a 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -23855,6 +23855,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
BTF_SET_START(btf_id_deny)
BTF_ID_UNUSED
#ifdef CONFIG_SMP
+BTF_ID(func, ___migrate_enable)
BTF_ID(func, migrate_disable)
BTF_ID(func, migrate_enable)
#endif
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index feb750aae71b..7f1e5cb94c53 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7,6 +7,8 @@
* Copyright (C) 1991-2002 Linus Torvalds
* Copyright (C) 1998-2024 Ingo Molnar, Red Hat
*/
+#define INSTANTIATE_EXPORTED_MIGRATE_DISABLE
+#include <linux/sched.h>
#include <linux/highmem.h>
#include <linux/hrtimer_api.h>
#include <linux/ktime_api.h>
@@ -2381,28 +2383,7 @@ static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
__do_set_cpus_allowed(p, &ac);
}
-void migrate_disable(void)
-{
- struct task_struct *p = current;
-
- if (p->migration_disabled) {
-#ifdef CONFIG_DEBUG_PREEMPT
- /*
- *Warn about overflow half-way through the range.
- */
- WARN_ON_ONCE((s16)p->migration_disabled < 0);
-#endif
- p->migration_disabled++;
- return;
- }
-
- guard(preempt)();
- this_rq()->nr_pinned++;
- p->migration_disabled = 1;
-}
-EXPORT_SYMBOL_GPL(migrate_disable);
-
-void migrate_enable(void)
+void ___migrate_enable(void)
{
struct task_struct *p = current;
struct affinity_context ac = {
@@ -2410,35 +2391,19 @@ void migrate_enable(void)
.flags = SCA_MIGRATE_ENABLE,
};
-#ifdef CONFIG_DEBUG_PREEMPT
- /*
- * Check both overflow from migrate_disable() and superfluous
- * migrate_enable().
- */
- if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
- return;
-#endif
+ __set_cpus_allowed_ptr(p, &ac);
+}
+EXPORT_SYMBOL_GPL(___migrate_enable);
- if (p->migration_disabled > 1) {
- p->migration_disabled--;
- return;
- }
+void migrate_disable(void)
+{
+ __migrate_disable();
+}
+EXPORT_SYMBOL_GPL(migrate_disable);
- /*
- * Ensure stop_task runs either before or after this, and that
- * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
- */
- guard(preempt)();
- if (p->cpus_ptr != &p->cpus_mask)
- __set_cpus_allowed_ptr(p, &ac);
- /*
- * Mustn't clear migration_disabled() until cpus_ptr points back at the
- * regular cpus_mask, otherwise things that race (eg.
- * select_fallback_rq) get confused.
- */
- barrier();
- p->migration_disabled = 0;
- this_rq()->nr_pinned--;
+void migrate_enable(void)
+{
+ __migrate_enable();
}
EXPORT_SYMBOL_GPL(migrate_enable);
diff --git a/kernel/sched/rq-offsets.c b/kernel/sched/rq-offsets.c
new file mode 100644
index 000000000000..a23747bbe25b
--- /dev/null
+++ b/kernel/sched/rq-offsets.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+#define COMPILE_OFFSETS
+#include <linux/kbuild.h>
+#include <linux/types.h>
+#include "sched.h"
+
+int main(void)
+{
+ DEFINE(RQ_nr_pinned, offsetof(struct rq, nr_pinned));
+
+ return 0;
+}
]
|
| 111/4 |
2026/11/18 01:28 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 21911
output: 26
thoughts: 18
Thoughts: Okay, here's my interpretation of what's going through my mind as I approach commit `6d337eab041d`, presented as a first-person account suitable for an expert:
**Diving into `6d337eab041d`**
Alright, let's pull up that commit hash: `6d337eab041d`. Immediately, my mental radar starts scanning. I'm not going to approach this blindly. I'm already forming a hypothesis based on the context, what I was working on, and the general project flow. I'm expecting something⦠hopefully coherent! Depending on the project, I may already know what kind of functionality might be altered by this. Is it a bug fix? Feature enhancement? Refactor? A dependency update? I'm anticipating the diff will give me the initial signals to confirm or reject. I'm looking for the key lines, the telltale patterns - maybe a specific function call, a changed data structure, or a newly added test. I'm also preparing for the possibility of a rabbit hole. If the changes are extensive, or if the reasoning behind them is obscure, I need to stay calm and start building a causal chain to understand the rationale. If there's an associated issue number or pull request, I'll be sure to reference it as I go. Ultimately, I'm aiming for a solid understanding of *why* this change happened and what its impact is. Let's see what we've got.
|
| 112/4 |
2026/11/18 01:28 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 6d337eab041d56bb8f0e7794f39906c21054c512
Author: Peter Zijlstra <peterz@infradead.org>
Date: Fri Sep 18 17:24:31 2020 +0200
sched: Fix migrate_disable() vs set_cpus_allowed_ptr()
Concurrent migrate_disable() and set_cpus_allowed_ptr() has
interesting features. We rely on set_cpus_allowed_ptr() to not return
until the task runs inside the provided mask. This expectation is
exported to userspace.
This means that any set_cpus_allowed_ptr() caller must wait until
migrate_enable() allows migrations.
At the same time, we don't want migrate_enable() to schedule, due to
patterns like:
preempt_disable();
migrate_disable();
...
migrate_enable();
preempt_enable();
And:
raw_spin_lock(&B);
spin_unlock(&A);
this means that when migrate_enable() must restore the affinity
mask, it cannot wait for completion thereof. Luck will have it that
that is exactly the case where there is a pending
set_cpus_allowed_ptr(), so let that provide storage for the async stop
machine.
Much thanks to Valentin who used TLA+ most effective and found lots of
'interesting' cases.
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Reviewed-by: Daniel Bristot de Oliveira <bristot@redhat.com>
Link: https://lkml.kernel.org/r/20201023102346.921768277@infradead.org
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 0732356c0eca..90a0c92741d7 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -714,6 +714,7 @@ struct task_struct {
int nr_cpus_allowed;
const cpumask_t *cpus_ptr;
cpumask_t cpus_mask;
+ void *migration_pending;
#if defined(CONFIG_SMP) && defined(CONFIG_PREEMPT_RT)
int migration_disabled;
#endif
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 6a3f1c2e185b..0efc1e41bb60 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1732,15 +1732,26 @@ void migrate_enable(void)
{
struct task_struct *p = current;
- if (--p->migration_disabled)
+ if (p->migration_disabled > 1) {
+ p->migration_disabled--;
return;
+ }
+ /*
+ * Ensure stop_task runs either before or after this, and that
+ * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
+ */
+ preempt_disable();
+ if (p->cpus_ptr != &p->cpus_mask)
+ __set_cpus_allowed_ptr(p, &p->cpus_mask, SCA_MIGRATE_ENABLE);
+ /*
+ * Mustn't clear migration_disabled() until cpus_ptr points back at the
+ * regular cpus_mask, otherwise things that race (eg.
+ * select_fallback_rq) get confused.
+ */
barrier();
-
- if (p->cpus_ptr == &p->cpus_mask)
- return;
-
- __set_cpus_allowed_ptr(p, &p->cpus_mask, SCA_MIGRATE_ENABLE);
+ p->migration_disabled = 0;
+ preempt_enable();
}
EXPORT_SYMBOL_GPL(migrate_enable);
@@ -1805,8 +1816,16 @@ static struct rq *move_queued_task(struct rq *rq, struct rq_flags *rf,
}
struct migration_arg {
- struct task_struct *task;
- int dest_cpu;
+ struct task_struct *task;
+ int dest_cpu;
+ struct set_affinity_pending *pending;
+};
+
+struct set_affinity_pending {
+ refcount_t refs;
+ struct completion done;
+ struct cpu_stop_work stop_work;
+ struct migration_arg arg;
};
/*
@@ -1838,16 +1857,19 @@ static struct rq *__migrate_task(struct rq *rq, struct rq_flags *rf,
*/
static int migration_cpu_stop(void *data)
{
+ struct set_affinity_pending *pending;
struct migration_arg *arg = data;
struct task_struct *p = arg->task;
+ int dest_cpu = arg->dest_cpu;
struct rq *rq = this_rq();
+ bool complete = false;
struct rq_flags rf;
/*
* The original target CPU might have gone down and we might
* be on another CPU but it doesn't matter.
*/
- local_irq_disable();
+ local_irq_save(rf.flags);
/*
* We need to explicitly wake pending tasks before running
* __migrate_task() such that we will not miss enforcing cpus_ptr
@@ -1857,21 +1879,83 @@ static int migration_cpu_stop(void *data)
raw_spin_lock(&p->pi_lock);
rq_lock(rq, &rf);
+
+ pending = p->migration_pending;
/*
* If task_rq(p) != rq, it cannot be migrated here, because we're
* holding rq->lock, if p->on_rq == 0 it cannot get enqueued because
* we're holding p->pi_lock.
*/
if (task_rq(p) == rq) {
+ if (is_migration_disabled(p))
+ goto out;
+
+ if (pending) {
+ p->migration_pending = NULL;
+ complete = true;
+ }
+
+ /* migrate_enable() -- we must not race against SCA */
+ if (dest_cpu < 0) {
+ /*
+ * When this was migrate_enable() but we no longer
+ * have a @pending, a concurrent SCA 'fixed' things
+ * and we should be valid again. Nothing to do.
+ */
+ if (!pending) {
+ WARN_ON_ONCE(!is_cpu_allowed(p, cpu_of(rq)));
+ goto out;
+ }
+
+ dest_cpu = cpumask_any_distribute(&p->cpus_mask);
+ }
+
if (task_on_rq_queued(p))
- rq = __migrate_task(rq, &rf, p, arg->dest_cpu);
+ rq = __migrate_task(rq, &rf, p, dest_cpu);
else
- p->wake_cpu = arg->dest_cpu;
+ p->wake_cpu = dest_cpu;
+
+ } else if (dest_cpu < 0) {
+ /*
+ * This happens when we get migrated between migrate_enable()'s
+ * preempt_enable() and scheduling the stopper task. At that
+ * point we're a regular task again and not current anymore.
+ *
+ * A !PREEMPT kernel has a giant hole here, which makes it far
+ * more likely.
+ */
+
+ /*
+ * When this was migrate_enable() but we no longer have an
+ * @pending, a concurrent SCA 'fixed' things and we should be
+ * valid again. Nothing to do.
+ */
+ if (!pending) {
+ WARN_ON_ONCE(!is_cpu_allowed(p, cpu_of(rq)));
+ goto out;
+ }
+
+ /*
+ * When migrate_enable() hits a rq mis-match we can't reliably
+ * determine is_migration_disabled() and so have to chase after
+ * it.
+ */
+ task_rq_unlock(rq, p, &rf);
+ stop_one_cpu_nowait(task_cpu(p), migration_cpu_stop,
+ &pending->arg, &pending->stop_work);
+ return 0;
}
- rq_unlock(rq, &rf);
- raw_spin_unlock(&p->pi_lock);
+out:
+ task_rq_unlock(rq, p, &rf);
+
+ if (complete)
+ complete_all(&pending->done);
+
+ /* For pending->{arg,stop_work} */
+ pending = arg->pending;
+ if (pending && refcount_dec_and_test(&pending->refs))
+ wake_up_var(&pending->refs);
- local_irq_enable();
return 0;
}
@@ -1940,6 +2024,112 @@ void do_set_cpus_allowed(struct task_struct *p, const struct cpumask *new_mask)
__do_set_cpus_allowed(p, new_mask, 0);
}
+/*
+ * This function is wildly self concurrent, consider at least 3 times.
+ */
+static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf,
+ int dest_cpu, unsigned int flags)
+{
+ struct set_affinity_pending my_pending = { }, *pending = NULL;
+ struct migration_arg arg = {
+ .task = p,
+ .dest_cpu = dest_cpu,
+ };
+ bool complete = false;
+
+ /* Can the task run on the task's current CPU? If so, we're done */
+ if (cpumask_test_cpu(task_cpu(p), &p->cpus_mask)) {
+ pending = p->migration_pending;
+ if (pending) {
+ refcount_inc(&pending->refs);
+ p->migration_pending = NULL;
+ complete = true;
+ }
+ task_rq_unlock(rq, p, rf);
+
+ if (complete)
+ goto do_complete;
+
+ return 0;
+ }
+
+ if (!(flags & SCA_MIGRATE_ENABLE)) {
+ /* serialized by p->pi_lock */
+ if (!p->migration_pending) {
+ refcount_set(&my_pending.refs, 1);
+ init_completion(&my_pending.done);
+ p->migration_pending = &my_pending;
+ } else {
+ pending = p->migration_pending;
+ refcount_inc(&pending->refs);
+ }
+ }
+ pending = p->migration_pending;
+ /*
+ * - !MIGRATE_ENABLE:
+ * we'll have installed a pending if there wasn't one already.
+ *
+ * - MIGRATE_ENABLE:
+ * we're here because the current CPU isn't matching anymore,
+ * the only way that can happen is because of a concurrent
+ * set_cpus_allowed_ptr() call, which should then still be
+ * pending completion.
+ *
+ * Either way, we really should have a @pending here.
+ */
+ if (WARN_ON_ONCE(!pending)) {
+ task_rq_unlock(rq, p, rf);
+ return -EINVAL;
+ }
+
+ if (flags & SCA_MIGRATE_ENABLE) {
+
+ refcount_inc(&pending->refs); /* pending->{arg,stop_work} */
+ task_rq_unlock(rq, p, rf);
+
+ pending->arg = (struct migration_arg) {
+ .task = p,
+ .dest_cpu = -1,
+ .pending = pending,
+ };
+
+ stop_one_cpu_nowait(cpu_of(rq), migration_cpu_stop,
+ &pending->arg, &pending->stop_work);
+
+ return 0;
+ }
+
+ if (task_running(rq, p) || p->state == TASK_WAKING) {
+
+ task_rq_unlock(rq, p, rf);
+ stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg);
+
+ } else {
+
+ if (!is_migration_disabled(p)) {
+ if (task_on_rq_queued(p))
+ rq = move_queued_task(rq, rf, p, dest_cpu);
+
+ p->migration_pending = NULL;
+ complete = true;
+ }
+ task_rq_unlock(rq, p, rf);
+
+do_complete:
+ if (complete)
+ complete_all(&pending->done);
+ }
+
+ wait_for_completion(&pending->done);
+
+ if (refcount_dec_and_test(&pending->refs))
+ wake_up_var(&pending->refs);
+
+ wait_var_event(&my_pending.refs, !refcount_read(&my_pending.refs));
+
+ return 0;
+}
+
/*
* Change a given task's CPU affinity. Migrate the thread to a
* proper CPU and schedule it away if the CPU it's executing on
@@ -2009,23 +2199,8 @@ static int __set_cpus_allowed_ptr(struct task_struct *p,
p->nr_cpus_allowed != 1);
}
- /* Can the task run on the task's current CPU? If so, we're done */
- if (cpumask_test_cpu(task_cpu(p), new_mask))
- goto out;
+ return affine_move_task(rq, p, &rf, dest_cpu, flags);
- if (task_running(rq, p) || p->state == TASK_WAKING) {
- struct migration_arg arg = { p, dest_cpu };
- /* Need help from migration thread: drop lock and wait. */
- task_rq_unlock(rq, p, &rf);
- stop_one_cpu(cpu_of(rq), migration_cpu_stop, &arg);
- return 0;
- } else if (task_on_rq_queued(p)) {
- /*
- * OK, since we're going to drop the lock immediately
- * afterwards anyway.
- */
- rq = move_queued_task(rq, &rf, p, dest_cpu);
- }
out:
task_rq_unlock(rq, p, &rf);
@@ -3205,6 +3380,7 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p)
init_numa_balancing(clone_flags, p);
#ifdef CONFIG_SMP
p->wake_entry.u_flags = CSD_TYPE_TTWU;
+ p->migration_pending = NULL;
#endif
}
]
|
| 113/4 |
2026/11/18 01:28 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 26318
output: 26
thoughts: 31
Thoughts: Okay, here's my interpretation of what's going through my mind, delivered as a first-person summary, keeping in mind the expert audience:
**Analyzing Commits: 9d0df3779745 and 0ec8d5aed4d1**
Alright, let's dive in. I need to get a clear picture of the changes made in commits `9d0df3779745` and `0ec8d5aed4d1`. First, I'll quickly recall context. What was the branching strategy involved? What was the general direction the project was heading in at this stage? That will help me frame these specific commits quickly. I'll mentally pull up the associated tickets or stories, if they exist, to understand the problem domain this code addresses.
Now, let me start at the first commit, `9d0df3779745`. I'll want to grasp the high-level modifications. Did it focus on a specific module or feature, or was it more of a refactoring effort? I need to analyze any new dependencies introduced, or changes to existing ones. What do the tests look like? Are they comprehensive, and do they cover the new functionality or fixes? I will be specifically looking for potential regressions here, and any edge cases that were, or weren't, addressed.
Next, I'll switch my attention to commit `0ec8d5aed4d1`. I'll begin by analyzing its diff, keeping in mind the prior commit. Are these changes complementary, or building upon the work in `9d0df3779745`? Is there a clear logical progression, or is there a divergence of intent? I will be looking for any conflicts or overlaps in the code. I am specifically interested in how the design and implementation evolved, and how each addresses existing problems in the code.
As I analyze both, I'll be forming a mental model of the impact of these changes on the system's architecture and performance. Any potential performance bottlenecks will be immediately flagged for investigation. I want to assess the quality of the implementation. Is the code clean, readable, and maintainable? Does it adhere to best practices for the language and framework being used? Are there any architectural changes? What are the implications of those changes?
Finally, after I have analyzed these commits individually and with respect to each other, I can assess the overall impact, risk, and value added to the project. Are these changes production ready? Are there any hidden gotchas or potential issues that need further investigation or follow-up? Are there any potential dependencies on either commit?
|
| 114/4 |
2026/11/18 01:28 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 9d0df37797453f168afdb2e6fd0353c73718ae9a
Author: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Date: Mon Nov 29 18:46:44 2021 +0100
sched: Trigger warning if ->migration_disabled counter underflows.
If migrate_enable() is used more often than its counter part then it
remains undetected and rq::nr_pinned will underflow, too.
Add a warning if migrate_enable() is attempted if without a matching a
migrate_disable().
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/r/20211129174654.668506-2-bigeasy@linutronix.de
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 3c9b0fda64ac..300218ad98a2 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2173,6 +2173,9 @@ void migrate_enable(void)
return;
}
+ if (WARN_ON_ONCE(!p->migration_disabled))
+ return;
+
/*
* Ensure stop_task runs either before or after this, and that
* __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
]
|
| 115/4 |
2026/11/18 01:28 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 27396
output: 25
thoughts: 17
Thoughts: Okay, here's my interpretation of what's going through my mind, delivered as requested:
**Deconstructing Commit 0ec8d5aed4d1**
Alright, let's dive into commit `0ec8d5aed4d1`. First thing's first, I need to get a high-level sense of the scope. Is this a major refactor, a bug fix, a new feature, or something else entirely? I'm already mentally running through the typical areas I'd look for clues β the commit message itself, the file changes, and any related pull requests. My experience tells me where to focus my attention based on the project's architecture and coding style. I'll likely start by glancing at the biggest changes, then zoom in on the specific lines that were added, modified, or removed. I'll try to understand the rationale behind the changes and how they fit into the broader context of the system. I'm looking for potential regressions, performance impacts, or unexpected side effects. Are the changes well-tested? Do the tests cover the new functionality or fixes adequately? I'll probably cross-reference the commit with my knowledge of the codebase, recalling any recent issues or discussions that might be relevant. Finally, I'll assess whether the changes are well-implemented, maintainable, and aligned with the project's goals. I'm essentially performing a mental code review, trying to anticipate any problems before they arise.
|
| 116/4 |
2026/11/18 01:28 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 0ec8d5aed4d14055aab4e2746def33f8b0d409c3
Author: Peilin He <he.peilin@zte.com.cn>
Date: Tue Jul 16 10:42:44 2024 +0800
sched/core: Add WARN_ON_ONCE() to check overflow for migrate_disable()
Background
==========
When repeated migrate_disable() calls are made with missing the
corresponding migrate_enable() calls, there is a risk of
'migration_disabled' going upper overflow because
'migration_disabled' is a type of unsigned short whose max value is
65535.
In PREEMPT_RT kernel, if 'migration_disabled' goes upper overflow, it may
make the migrate_disable() ineffective within local_lock_irqsave(). This
is because, during the scheduling procedure, the value of
'migration_disabled' will be checked, which can trigger CPU migration.
Consequently, the count of 'rcu_read_lock_nesting' may leak due to
local_lock_irqsave() and local_unlock_irqrestore() occurring on different
CPUs.
Usecase
========
For example, When I developed a driver, I encountered a warning like
"WARNING: CPU: 4 PID: 260 at kernel/rcu/tree_plugin.h:315
rcu_note_context_switch+0xa8/0x4e8" warning. It took me half a month
to locate this issue. Ultimately, I discovered that the lack of upper
overflow detection mechanism in migrate_disable() was the root cause,
leading to a significant amount of time spent on problem localization.
If the upper overflow detection mechanism was added to migrate_disable(),
the root cause could be very quickly and easily identified.
Effect
======
Using WARN_ON_ONCE() to check if 'migration_disabled' is upper overflow
can help developers identify the issue quickly.
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Peilin He<he.peilin@zte.com.cn>
Signed-off-by: xu xin <xu.xin16@zte.com.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Yunkai Zhang <zhang.yunkai@zte.com.cn>
Reviewed-by: Qiang Tu <tu.qiang35@zte.com.cn>
Reviewed-by: Kun Jiang <jiang.kun2@zte.com.cn>
Reviewed-by: Fan Yu <fan.yu9@zte.com.cn>
Link: https://lkml.kernel.org/r/20240716104244764N2jD8gnBpnsLjCDnQGQ8c@zte.com.cn
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 2c61b4fcb15e..db5823fcc31c 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2233,6 +2233,12 @@ void migrate_disable(void)
struct task_struct *p = current;
if (p->migration_disabled) {
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ *Warn about overflow half-way through the range.
+ */
+ WARN_ON_ONCE((s16)p->migration_disabled < 0);
+#endif
p->migration_disabled++;
return;
}
@@ -2251,14 +2257,20 @@ void migrate_enable(void)
.flags = SCA_MIGRATE_ENABLE,
};
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ * Check both overflow from migrate_disable() and superfluous
+ * migrate_enable().
+ */
+ if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
+ return;
+#endif
+
if (p->migration_disabled > 1) {
p->migration_disabled--;
return;
}
- if (WARN_ON_ONCE(!p->migration_disabled))
- return;
-
/*
* Ensure stop_task runs either before or after this, and that
* __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
]
|
| 117/4 |
2026/11/18 01:28 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 28924
output: 26
thoughts: 669
Thoughts: **Analyzing `migrate_disable()` and `migrate_enable()`**
Okay, let's dive into these functions, focusing on `migrate_disable()` and `migrate_enable()`. I need to understand this meticulously. I'm seeing that `__migrate_disable()` increments `p->migration_disabled` and, importantly, increments `this_rq_pinned()`. `__migrate_enable()` decrements `migration_disabled` and calls `___migrate_enable()` under certain conditions. The `guard(preempt)()` calls are interesting; they're hinting at some preemption implications here. I should pay close attention to that.
Now, wait a minute! It is key to remember that `current` is the current task that *was* running on CPU 0. If an interrupt or softirq arrives, who is `current`? In interrupt/softirq context, `current` is the task that got interrupted. This is crucial for understanding how these functions behave in different contexts.
So, in hardirq or softirq context, unless we're in ksoftirqd, `current` refers to the interrupted task. Now, can `migrate_disable()` or `migrate_enable()` even be called from interrupt or softirq context? I should clarify that. This has potential implications for the interrupt context and the functions' behavior. I need to know where these functions are valid.
Alright, the next step is to consult the documentation and any inline comments. I'll search the relevant parts of the kernel and the Documentation directory. I need to understand any stated restrictions or best practices around using `migrate_disable()` and `migrate_enable()`. It is also important to know how these functions should be used when an interrupt has arrived. I will see if the comments describe how it is intended to be used and in what context.
|
| 118/4 |
2026/11/18 01:29 |
tool |
grepper |
0m
Args: map[Expression:migrate_disable PathPrefixes:[Documentation/]]
Results: map[Output:Documentation/locking/locktypes.rst=451=preemption. The following substitution works on both kernels::
--
Documentation/locking/locktypes.rst-454-
Documentation/locking/locktypes.rst:455: migrate_disable();
Documentation/locking/locktypes.rst-456- p = this_cpu_ptr(&var1);
--
Documentation/locking/locktypes.rst-459-
Documentation/locking/locktypes.rst:460:migrate_disable() ensures that the task is pinned on the current CPU which
Documentation/locking/locktypes.rst-461-in turn guarantees that the per-CPU access to var1 and var2 are staying on
Documentation/locking/locktypes.rst=462=the same CPU while the task remains preemptible.
Documentation/locking/locktypes.rst-463-
Documentation/locking/locktypes.rst:464:The migrate_disable() substitution is not valid for the following
Documentation/locking/locktypes.rst-465-scenario::
--
Documentation/locking/locktypes.rst-470-
Documentation/locking/locktypes.rst:471: migrate_disable();
Documentation/locking/locktypes.rst-472- p = this_cpu_ptr(&var1);
--
Documentation/locking/locktypes.rst-474-
Documentation/locking/locktypes.rst:475:This breaks because migrate_disable() does not protect against reentrancy from
Documentation/locking/locktypes.rst-476-a preempting task. A correct substitution for this case is::
--
Documentation/translations/it_IT/locking/locktypes.rst=443=prelazione. La seguente sostituzione funzionerΓ su entrambe i kernel::
--
Documentation/translations/it_IT/locking/locktypes.rst-446-
Documentation/translations/it_IT/locking/locktypes.rst:447: migrate_disable();
Documentation/translations/it_IT/locking/locktypes.rst-448- p = this_cpu_ptr(&var1);
--
Documentation/translations/it_IT/locking/locktypes.rst-451-
Documentation/translations/it_IT/locking/locktypes.rst:452:La funzione migrate_disable() assicura che il processo venga tenuto sulla CPU
Documentation/translations/it_IT/locking/locktypes.rst-453-corrente, e di conseguenza garantisce che gli accessi per-CPU alle variabili var1 e
Documentation/translations/it_IT/locking/locktypes.rst=454=var2 rimangano sulla stessa CPU fintanto che il processo rimane prelabile.
Documentation/translations/it_IT/locking/locktypes.rst-455-
Documentation/translations/it_IT/locking/locktypes.rst:456:La sostituzione con migrate_disable() non funzionerΓ nel seguente scenario::
Documentation/translations/it_IT/locking/locktypes.rst-457-
--
Documentation/translations/it_IT/locking/locktypes.rst-461-
Documentation/translations/it_IT/locking/locktypes.rst:462: migrate_disable();
Documentation/translations/it_IT/locking/locktypes.rst-463- p = this_cpu_ptr(&var1);
--
Documentation/translations/it_IT/locking/locktypes.rst-465-
Documentation/translations/it_IT/locking/locktypes.rst:466:Questo non funziona perchΓ© migrate_disable() non protegge dal ritorno da un
Documentation/translations/it_IT/locking/locktypes.rst-467-processo che aveva avuto il diritto di prelazione. Una sostituzione piΓΉ adatta
]
|
| 119/4 |
2026/11/18 01:29 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 30897
output: 32
thoughts: 19
Thoughts: Okay, here's the summary, expressed in the first person, assuming an expert user:
**Investigating Migration Disablement**
Right, let's drill down further into this migration issue. Given the context, I think it's wise to focus our search terms to pinpoint the potential source of the problem. I'm going to leverage my existing knowledge of the system and its architecture. My immediate thought is to check for any mention of the `cant_migrate` flag. That's a strong indicator of a specific, defined reason why a migration might be blocked. Or maybe something's in the comments, an offhand note about `migrate_disable`. Sometimes the documentation is thin, and the true reason is only buried in developer comments. These are the kinds of specific keywords I'm hunting for, the breadcrumbs that might lead me to the root cause. This methodical approach will hopefully reveal something useful, and quickly. I'll need to know whether any prior troubleshooting steps have been taken, as well.
|
| 120/4 |
2026/11/18 01:29 |
tool |
grepper |
0m
Args: map[Expression:migrate_disable PathPrefixes:[include/linux/preempt.h]]
Results: map[Output:include/linux/preempt.h=387=static inline void preempt_notifier_init(struct preempt_notifier *notifier,
--
include/linux/preempt.h-402- * becomes one of the M highest priority tasks), it might still have to wait
include/linux/preempt.h:403: * for the preemptee's migrate_disable() section to complete. Thereby suffering
include/linux/preempt.h:404: * a reduction in bandwidth in the exact duration of the migrate_disable()
include/linux/preempt.h-405- * section.
include/linux/preempt.h-406- *
include/linux/preempt.h:407: * Per this argument, the change from preempt_disable() to migrate_disable()
include/linux/preempt.h-408- * gets us:
--
include/linux/preempt.h-415- * by the ability to push the higher priority task away, which might itself be
include/linux/preempt.h:416: * in a migrate_disable() section, reducing its available bandwidth.
include/linux/preempt.h-417- *
--
include/linux/preempt.h-427- * migration. This turns out to break a bunch of per-cpu usage. To this end,
include/linux/preempt.h:428: * all these primitives employ migrate_disable() to restore this implicit
include/linux/preempt.h-429- * assumption.
--
include/linux/preempt.h-434- *
include/linux/preempt.h:435: * The end goal must be to get rid of migrate_disable(), alternatively we need
include/linux/preempt.h-436- * a schedulability theory that does not depend on arbitrary migration.
--
include/linux/preempt.h-441- * The implementation is particularly tricky since existing code patterns
include/linux/preempt.h:442: * dictate neither migrate_disable() nor migrate_enable() is allowed to block.
include/linux/preempt.h-443- * This means that it cannot use cpus_read_lock() to serialize against hotplug,
--
include/linux/preempt.h-448- * Note: even non-work-conserving schedulers like semi-partitioned depends on
include/linux/preempt.h:449: * migration, so migrate_disable() is not only a problem for
include/linux/preempt.h-450- * work-conserving schedulers.
]
|
| 121/4 |
2026/11/18 01:29 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 31785
output: 36
thoughts: 31
Thoughts: Okay, here's my take on those lines of thought, geared towards a fellow kernel expert:
**Preemption Shenanigans in `preempt.h`**
Alright, let's dive into the core of preemption logic. I'm going to take a look at `include/linux/preempt.h` around lines 380-455. I know this section houses the critical comment block defining how preemption works within the kernel. I need to refresh my understanding and maybe spot any nuanced details I've missed previously. I want to zero in on how it affects the different states and how the `preempt_count` works. It's time to refresh and review those low-level details. Let's see what gems we can unearth within that particular area of the header file.
|
| 122/4 |
2026/11/18 01:29 |
tool |
read-file |
0m
Args: map[File:include/linux/preempt.h FirstLine:380 LineCount:80]
Results: map[Contents: 380: };
381:
382: void preempt_notifier_inc(void);
383: void preempt_notifier_dec(void);
384: void preempt_notifier_register(struct preempt_notifier *notifier);
385: void preempt_notifier_unregister(struct preempt_notifier *notifier);
386:
387: static inline void preempt_notifier_init(struct preempt_notifier *notifier,
388: struct preempt_ops *ops)
389: {
390: /* INIT_HLIST_NODE() open coded, to avoid dependency on list.h */
391: notifier->link.next = NULL;
392: notifier->link.pprev = NULL;
393: notifier->ops = ops;
394: }
395:
396: #endif
397:
398: /*
399: * Migrate-Disable and why it is undesired.
400: *
401: * When a preempted task becomes eligible to run under the ideal model (IOW it
402: * becomes one of the M highest priority tasks), it might still have to wait
403: * for the preemptee's migrate_disable() section to complete. Thereby suffering
404: * a reduction in bandwidth in the exact duration of the migrate_disable()
405: * section.
406: *
407: * Per this argument, the change from preempt_disable() to migrate_disable()
408: * gets us:
409: *
410: * - a higher priority tasks gains reduced wake-up latency; with preempt_disable()
411: * it would have had to wait for the lower priority task.
412: *
413: * - a lower priority tasks; which under preempt_disable() could've instantly
414: * migrated away when another CPU becomes available, is now constrained
415: * by the ability to push the higher priority task away, which might itself be
416: * in a migrate_disable() section, reducing its available bandwidth.
417: *
418: * IOW it trades latency / moves the interference term, but it stays in the
419: * system, and as long as it remains unbounded, the system is not fully
420: * deterministic.
421: *
422: *
423: * The reason we have it anyway.
424: *
425: * PREEMPT_RT breaks a number of assumptions traditionally held. By forcing a
426: * number of primitives into becoming preemptible, they would also allow
427: * migration. This turns out to break a bunch of per-cpu usage. To this end,
428: * all these primitives employ migrate_disable() to restore this implicit
429: * assumption.
430: *
431: * This is a 'temporary' work-around at best. The correct solution is getting
432: * rid of the above assumptions and reworking the code to employ explicit
433: * per-cpu locking or short preempt-disable regions.
434: *
435: * The end goal must be to get rid of migrate_disable(), alternatively we need
436: * a schedulability theory that does not depend on arbitrary migration.
437: *
438: *
439: * Notes on the implementation.
440: *
441: * The implementation is particularly tricky since existing code patterns
442: * dictate neither migrate_disable() nor migrate_enable() is allowed to block.
443: * This means that it cannot use cpus_read_lock() to serialize against hotplug,
444: * nor can it easily migrate itself into a pending affinity mask change on
445: * migrate_enable().
446: *
447: *
448: * Note: even non-work-conserving schedulers like semi-partitioned depends on
449: * migration, so migrate_disable() is not only a problem for
450: * work-conserving schedulers.
451: *
452: */
453:
454: /**
455: * preempt_disable_nested - Disable preemption inside a normally preempt disabled section
456: *
457: * Use for code which requires preemption protection inside a critical
458: * section which has preemption disabled implicitly on non-PREEMPT_RT
459: * enabled kernels, by e.g.:
]
|
| 123/4 |
2026/11/18 01:29 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 33280
output: 34
thoughts: 71
Thoughts: Okay, here's my summary, tailored as requested:
**Investigating Potential BPF Performance Bottlenecks**
Alright, let's dive in. My initial thought is to zero in on potential performance issues related to the `bpf_prog_run_pin_on_cpu` function. Given my understanding of BPF internals, I suspect this function, and its interactions with other kernel subsystems, is the likely culprit, so let's start by digging deep.
First, I need to understand the scenarios where this function could be problematic, so I'll start by querying the codebase. I'll search the relevant lore resources β documentation, mailing list archives, bug reports, and especially the Git history β using a specific, combined search query to narrow the results. My search parameters will focus on `bpf_prog_run_pin_on_cpu`, while also incorporating terms related to contention, like `migrate_disable`, or data flow, like `packet_rcv` or latency caused by interrupt processing `softirq`. This should provide some valuable information.
Second, I need to see what's changed and whether we've already tried to solve this problem. I'll use the Git commit messages and use a specific search to see if this function is mentioned in any commit, for example, `bpf_prog_run_pin_on_cpu`. This will indicate if this function has undergone performance optimization attempts in the past.
|
| 124/4 |
2026/11/18 01:29 |
tool |
git-log |
10m
Error:
Args: map[CodeRegexp:bpf_prog_run_pin_on_cpu Count:10]
Results:
|
| 125/4 |
2026/11/18 01:39 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 33723
output: 47
thoughts: 31
Thoughts: Okay, here's my take on those thoughts, framed as a first-person expert summary:
**Analyzing `bpf_prog_run_pin_on_cpu` in the Kernel's Filter Header**
Right, let's dive into this. The mental checklist starts with, "Okay, `bpf_prog_run_pin_on_cpu`... that's a very specific function name." My gut tells me this likely relates to running a BPF program that's been "pinned" to a specific CPU core. The "pinning" part immediately raises questions about thread affinity, NUMA considerations, and potential performance implications around cache locality. Is it about controlling context switching? Load balancing? Precise scheduling of eBPF programs?
My immediate goal: locate the declaration of this function within `include/linux/filter.h`. It is a well-known header file. I know that if this is a publicly accessible interface, it will be defined there. The function name itself suggests a function related to BPF, and given the context, I expect to find its definition, and hopefully, some comments explaining its purpose, arguments, and return value. More than that, I am hoping to also glean information about where it is actually used. That will guide my next steps.
I'll be looking for clues in the surrounding code. Comments would be useful. I'm keen to understand the constraints and use cases it's designed for. I'm thinking: Why pin a BPF program? What are the potential advantages? And critically, what are the potential *disadvantages* or gotchas? This function might offer a highly-optimized execution path, but I need to understand its limitations to avoid surprises. My interest is in understanding how eBPF programs interact with the CPU and its resources in a more controlled fashion. And, what are the implications on any underlying data structures, error-handling and overall stability? Armed with that knowledge, I can start to evaluate its impact on performance, security, and overall system behaviour.
|
| 126/4 |
2026/11/18 01:39 |
tool |
git-log |
0m
Args: map[Count:10 MessageRegexps:[bpf_prog_run_pin_on_cpu] PathPrefix:include/linux/filter.h]
Results: map[Output:f941eadd8d6d bpf: Avoid races in __bpf_prog_run() for 32bit arches
fb7dd8bca013 bpf: Refactor BPF_PROG_RUN into a function
3d9f773cf287 bpf: Use bpf_prog_run_pin_on_cpu() at simple call sites.
3c58482a382b bpf: Provide bpf_prog_run_pin_on_cpu() helper
]
|
| 127/4 |
2026/11/18 01:39 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 34359
output: 27
thoughts: 34
Thoughts: Okay, let's dive into this, shall we? I need to take a look at commits `3c58482a382b` and `3d9f773cf287`.
|
| 128/4 |
2026/11/18 01:39 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 3c58482a382bae89410439247152eb342e9872f7
Author: Thomas Gleixner <tglx@kernel.org>
Date: Mon Feb 24 15:01:41 2020 +0100
bpf: Provide bpf_prog_run_pin_on_cpu() helper
BPF programs require to run on one CPU to completion as they use per CPU
storage, but according to Alexei they don't need reentrancy protection as
obviously BPF programs running in thread context can always be 'preempted'
by hard and soft interrupts and instrumentation and the same program can
run concurrently on a different CPU.
The currently used mechanism to ensure CPUness is to wrap the invocation
into a preempt_disable/enable() pair. Disabling preemption is also
disabling migration for a task.
preempt_disable/enable() is used because there is no explicit way to
reliably disable only migration.
Provide a separate macro to invoke a BPF program which can be used in
migrateable task context.
It wraps BPF_PROG_RUN() in a migrate_disable/enable() pair which maps on
non RT enabled kernels to preempt_disable/enable(). On RT enabled kernels
this merely disables migration. Both methods ensure that the invoked BPF
program runs on one CPU to completion.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200224145643.474592620@linutronix.de
diff --git a/include/linux/filter.h b/include/linux/filter.h
index f349e2c0884c..38f60188bb26 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -576,8 +576,30 @@ DECLARE_STATIC_KEY_FALSE(bpf_stats_enabled_key);
} \
ret; })
-#define BPF_PROG_RUN(prog, ctx) __BPF_PROG_RUN(prog, ctx, \
- bpf_dispatcher_nopfunc)
+#define BPF_PROG_RUN(prog, ctx) \
+ __BPF_PROG_RUN(prog, ctx, bpf_dispatcher_nopfunc)
+
+/*
+ * Use in preemptible and therefore migratable context to make sure that
+ * the execution of the BPF program runs on one CPU.
+ *
+ * This uses migrate_disable/enable() explicitly to document that the
+ * invocation of a BPF program does not require reentrancy protection
+ * against a BPF program which is invoked from a preempting task.
+ *
+ * For non RT enabled kernels migrate_disable/enable() maps to
+ * preempt_disable/enable(), i.e. it disables also preemption.
+ */
+static inline u32 bpf_prog_run_pin_on_cpu(const struct bpf_prog *prog,
+ const void *ctx)
+{
+ u32 ret;
+
+ migrate_disable();
+ ret = __BPF_PROG_RUN(prog, ctx, bpf_dispatcher_nopfunc);
+ migrate_enable();
+ return ret;
+}
#define BPF_SKB_CB_LEN QDISC_CB_PRIV_LEN
]
|
| 129/4 |
2026/11/18 01:39 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 35384
output: 38
thoughts: 202
Thoughts: **Investigating the Implications of a BPF Program Invocation Macro**
Okay, so I'm looking at this commit description, and it's proposing a macro for invoking BPF programs in a migrateable task context. The description states it wraps `BPF_PROG_RUN()` with `migrate_disable/enable()`. Hmm, I see it notes the mapping to `preempt_disable/enable()` on non-RT kernels. *That* immediately triggered a flag. I recall this from back in 2020: wasn't `migrate_disable()` just an alias for `preempt_disable()` on those non-RT systems?
The question now is, did that implementation remain true, or did things change?
I need to confirm if `migrate_disable()` is *still* mapped to `preempt_disable()` on non-RT kernels, or if it has since become a distinct function even in those contexts? I need to know when `migrate_disable()` became a "real" function. Right, let's go straight to the source. Time to search the git log for occurrences of `migrate_disable` within `kernel/sched/core.c`. Let's see what the history reveals.
|
| 130/4 |
2026/11/18 01:39 |
tool |
git-log |
0m
Args: map[Count:10 MessageRegexps:[migrate_disable] PathPrefix:kernel/sched/core.c]
Results: map[Output:0d032a43ebeb sched/mmcid: Prevent pointless work in mm_update_cpus_allowed()
942b8db96500 sched: Fix migrate_disable_switch() locking
378b7708194f sched: Make migrate_{en,dis}able() inline
0ec8d5aed4d1 sched/core: Add WARN_ON_ONCE() to check overflow for migrate_disable()
f0498d2a54e7 sched: Fix stop_one_cpu_nowait() vs hotplug
9d0df3779745 sched: Trigger warning if ->migration_disabled counter underflows.
8d713b699e84 sched: Make might_sleep() output less confusing
475ea6c60279 sched: Don't defer CPU pick to migration_cpu_stop()
50caf9c14b14 sched: Simplify set_affinity_pending refcounts
9e81889c7648 sched: Fix affine_move_task() self-concurrency
]
|
| 131/4 |
2026/11/18 01:39 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 36154
output: 38
thoughts: 45
Thoughts: Okay, here's my interpretation of that thought process, from an expert's perspective:
**Pinpointing the Activation of `migrate_disable` for Symmetric Multiprocessing**
Alright, I need to track down the introduction of the `migrate_disable` feature *specifically* for SMP systems, not just within the PREEMPT_RT context. The broader SMP use case is what I'm interested in here, as the nuances will be very different. The `migrate_disable` setting has implications, and I need to see when that functionality became available for general SMP use.
My strategy? I'll dive into the kernel source, specifically `kernel/sched/core.c`, as that's where I expect to find the relevant scheduling logic. I'll employ a targeted search. I'm going to look for commits where the term `migration_disabled` is mentioned in that file. That should give me a good history of the feature's development and reveal the specific commits that introduced or altered its SMP-related behavior. It's a pretty standard approach, but it's a good starting point to start the investigative process. From there I'll be able to work through any related commits and the potential evolution of the code.
|
| 132/4 |
2026/11/18 01:39 |
tool |
git-log |
0m
Args: map[Count:10 MessageRegexps:[migration_disabled] PathPrefix:kernel/sched/core.c]
Results: map[Output:0ec8d5aed4d1 sched/core: Add WARN_ON_ONCE() to check overflow for migrate_disable()
9d0df3779745 sched: Trigger warning if ->migration_disabled counter underflows.
]
|
| 133/4 |
2026/11/18 01:39 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 36565
output: 34
thoughts: 65
Thoughts: Here's my summary, as if I were thinking these thoughts:
**Analyzing Kernel Sched Core History**
Right, let's dive into some kernel scheduling code. I need to refresh my memory on a specific change, so I'm going to jump back in the commit history to `8d713b699e84` or potentially even earlier within `kernel/sched/core.c`. I need to understand the evolution of this section, and that commit seems like a good starting point. To get a quick overview of recent changes leading up to that point, I'll run a `git log -n 5 9e81889c7648~1 kernel/sched/core.c` command. This will provide the last five commits from the parent of `9e81889c7648`, focusing specifically on the `kernel/sched/core.c` file. That should give me the necessary context to understand the relevant changes.
|
| 134/4 |
2026/11/18 01:40 |
tool |
git-log |
0m
Args: map[Count:5 PathPrefix:kernel/sched/core.c Since:5 years]
Results: map[Output:ec6f6731538e sched/core: Do not touch SMT siblings that have not joined the core yet
0965587d00b8 sched/core: Account I/O waits independently of delayed dequeue
7dbc307b6c72 sched/mmcid: Fix affinity-triggered switch back to per-task mode
b99b7f60c5b4 sched: Replace nr_iowait_cpu() with a bool accessor
118a2cf4ced6 sched/core: Call wakeup_preempt() for queued tasks during class change
]
|
| 135/4 |
2026/11/18 01:40 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 37026
output: 31
thoughts: 23
Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience:
**Examining Task Struct's Migration State**
Okay, hold on a second. Before I proceed, I need to ground myself and fully understand how the `migration_disabled` flag within the `task_struct` is currently defined. This is fundamental; I can't build upon something without a solid grasp of its current implementation and any potential side effects. I need to refresh my mental model of that specific field and its interaction with the scheduler and memory management. I'm going to quickly check the relevant header files and source code to see how the bit is allocated and any relevant locking semantics associated with it. I need to make sure I am working with the most up-to-date and authoritative source.
|
| 136/4 |
2026/11/18 01:40 |
tool |
grepper |
0m
Args: map[Expression:migration_disabled PathPrefixes:[include/linux/sched.h]]
Results: map[Output:include/linux/sched.h=836=struct task_struct {
--
include/linux/sched.h-947- void *migration_pending;
include/linux/sched.h:948: unsigned short migration_disabled;
include/linux/sched.h-949- unsigned short migration_flags;
--
include/linux/sched.h=2444=static inline void __migrate_enable(void)
--
include/linux/sched.h-2452- */
include/linux/sched.h:2453: if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
include/linux/sched.h-2454- return;
--
include/linux/sched.h-2456-
include/linux/sched.h:2457: if (p->migration_disabled > 1) {
include/linux/sched.h:2458: p->migration_disabled--;
include/linux/sched.h-2459- return;
--
include/linux/sched.h-2469- /*
include/linux/sched.h:2470: * Mustn't clear migration_disabled() until cpus_ptr points back at the
include/linux/sched.h-2471- * regular cpus_mask, otherwise things that race (eg.
--
include/linux/sched.h-2474- barrier();
include/linux/sched.h:2475: p->migration_disabled = 0;
include/linux/sched.h-2476- this_rq_pinned()--;
--
include/linux/sched.h=2479=static inline void __migrate_disable(void)
--
include/linux/sched.h-2482-
include/linux/sched.h:2483: if (p->migration_disabled) {
include/linux/sched.h-2484-#ifdef CONFIG_DEBUG_PREEMPT
--
include/linux/sched.h-2487- */
include/linux/sched.h:2488: WARN_ON_ONCE((s16)p->migration_disabled < 0);
include/linux/sched.h-2489-#endif
include/linux/sched.h:2490: p->migration_disabled++;
include/linux/sched.h-2491- return;
--
include/linux/sched.h-2495- this_rq_pinned()++;
include/linux/sched.h:2496: p->migration_disabled = 1;
include/linux/sched.h-2497-}
]
|
| 137/4 |
2026/11/18 01:40 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 37919
output: 41
thoughts: 851
Thoughts: **Analyzing Potential Data Races in Kernel Task Migration Disabling**
Okay, let's break this down systematically. I see a question about the safety of calling `migrate_disable()` and `migrate_enable()` from interrupt or softirq context, specifically in the context of a BPF program running within a packet receive path. Let's make sure our understanding is perfectly clear.
First, I need to remind myself of the definition of `migration_disabled` within `struct task_struct`. The code tells me it's always present, which makes sense for SMP systems. And I see the inline implementations of `__migrate_disable()` and `__migrate_enable()` in `sched.h`, and that `__migrate_enable()` contains a lot of careful handling. I need to fully grok the subtleties to determine if it is okay.
Now, the core of the problem: Can these functions, `migrate_disable()` and `migrate_enable()`, *legitimately* be called from interrupt/softirq context? That is the real question to resolve here. The example given: `packet_rcv` -> `run_filter` -> `bpf_prog_run_clear_cb` -> `bpf_prog_run_pin_on_cpu` -> `migrate_disable()/migrate_enable()` raises concerns.
The data race is the heart of the matter. Imagine a scenario where a task on CPU 0 is interrupted by an interrupt or softirq. If that interrupt, while operating on current->migration_disabled (e.g., during the execution of `__migrate_enable()`), calls `migrate_disable()` and `migrate_enable()`, can we end up with corrupted data or unexpected behavior? We need to determine if there is a real risk of corruption of `current->migration_disabled` or the `rq->nr_pinned` counter, or if any `WARN_ON()` calls are possible.
Specifically, I need to understand why `bpf_prog_run_clear_cb` is calling `bpf_prog_run_pin_on_cpu`. *That* is the critical piece here to see if this is intentional or not. Where is `bpf_prog_run_clear_cb` actually defined? I need to check the exact function definition and usage to understand its intent and flow of execution.
|
| 138/4 |
2026/11/18 01:40 |
tool |
codesearch-definition-source |
0m
Args: map[ContextFile:include/linux/filter.h Name:bpf_prog_run_clear_cb]
Results: map[SourceCode:1110: static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
1111: struct sk_buff *skb)
1112: {
1113: u8 *cb_data = bpf_skb_cb(skb);
1114: u32 res;
1115:
1116: if (unlikely(prog->cb_access))
1117: memset(cb_data, 0, BPF_SKB_CB_LEN);
1118:
1119: res = bpf_prog_run_pin_on_cpu(prog, skb);
1120: return res;
1121: }
SourceFile:include/linux/filter.h]
|
| 139/4 |
2026/11/18 01:40 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 39533
output: 33
thoughts: 1165
Thoughts: **Analyzing Potential Data Races in Kernel Context Switching**
Okay, so I'm diving into this BPF program execution path, specifically focusing on `bpf_prog_run_clear_cb` and its call to `bpf_prog_run_pin_on_cpu`. This leads me down a rabbit hole into `run_filter` in `net/packet/af_packet.c`, which, in turn, is triggered by `packet_rcv` and `tpacket_rcv`. These are registered as packet handlers using `dev_add_pack` and called within the `__netif_receive_skb_core()` or `dev_queue_xmit_nit()` functions. Now, `__netif_receive_skb_core()` runs in softirq context, like `napi_gro_receive`, `napi_poll`, and `net_rx_action`.
This is where it gets interesting. I need to keep in mind that I'm inside a softirq context here, meaning `in_interrupt()` and `in_softirq()` are both true. Preemption might *not* be disabled by `preempt_disable()` in softirq, however, in softirq (bottom half), `local_bh_disable()` / `in_softirq()` is true. An interrupt (hardirq) *can* preempt a softirq, as well as a process. Softirqs can also interrupt task context. In fact, if a hardirq fires while a task is running, the softirq will run on *that* CPU on behalf of the interrupted task after the hardirq is exited!
Okay, let's look at the given prompt again: "When a task on CPU 0 is interrupted by an interrupt/softirq that calls migrate_disable() and migrate_enable(), both operating on current->migration_disabled (e.g. while the task was executing __migrate_enable()), is this data race harmful or benign? Can current->migration_disabled or rq->nr_pinned get corrupted or lead to any failure/WARN_ON?"
Right, so let's step through this instruction by instruction. I need to understand what's happening when a hardirq or softirq interrupts a task that's messing with migration disabling/enabling.
Let's dissect `__migrate_enable()` and `__migrate_disable()` from `include/linux/sched.h`. Specifically, I need to focus on what happens in `__migrate_enable()`, and the use of `guard(preempt)()` in particular. I see `guard(preempt)()` which seems to disable preemption. I know it's *not* disabling interrupts or softirqs. `guard(preempt)()` is actually `preempt_disable()` with an RAII-style `preempt_enable()` at the end of the scope.
So, since I know `preempt_disable()` only disables preemption and not interrupts and softirqs, what happens when a hardirq occurs during `preempt_disable()`? In `irq_exit()`, `invoke_softirq()` is called. *Can* softirqs run if `preempt_count() != 0`? The key is `irq_exit()` and `invoke_softirq()`!
I need to dig into `kernel/softirq.c` now to understand the interaction between `preempt_count` and softirq execution. My goal is to determine if the data race mentioned in the prompt is a *real* problem. The potential for corruption of `current->migration_disabled` or `rq->nr_pinned` is the thing that I need to confirm as harmless, or not.
|
| 140/4 |
2026/11/18 01:40 |
tool |
grepper |
0m
Args: map[Expression:invoke_softirq PathPrefixes:[kernel/softirq.c]]
Results: map[Output:kernel/softirq.c=356=static inline bool should_wake_ksoftirqd(void)
--
kernel/softirq.c-360-
kernel/softirq.c:361:static inline void invoke_softirq(void)
kernel/softirq.c-362-{
--
kernel/softirq.c=377=void do_softirq_post_smp_call_flush(unsigned int was_pending)
--
kernel/softirq.c-382- WARN_ON_ONCE(was_pending != (is_pending & ~SCHED_SOFTIRQ_MASK));
kernel/softirq.c:383: invoke_softirq();
kernel/softirq.c-384- }
--
kernel/softirq.c=528=static inline bool should_wake_ksoftirqd(void)
--
kernel/softirq.c-532-
kernel/softirq.c:533:static inline void invoke_softirq(void)
kernel/softirq.c-534-{
--
kernel/softirq.c=767=static inline void __irq_exit_rcu(void)
--
kernel/softirq.c-785- hrtimer_rearm_deferred();
kernel/softirq.c:786: invoke_softirq();
kernel/softirq.c-787- }
]
|
| 141/4 |
2026/11/18 01:40 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 41911
output: 35
thoughts: 43
Thoughts: Here's my summary, as if I were thinking it:
**Diving into `softirq.c` for Analysis**
Okay, let's get down to brass tacks. I need to refresh my memory on the soft interrupt handling in the kernel, specifically within `kernel/softirq.c`. I'm going to focus on a few key sections to understand the core logic. First, I want to zero in on lines 360-400. That range usually deals with the scheduling aspects, right? I'm anticipating seeing the context switching mechanisms related to soft interrupts there. It's probably the hook to other schedulers that handle the softirq. Let's see if the code confirms my assumptions regarding the control flow and potential scheduling latencies introduced by softirqs.
Next, I'll jump to lines 530-550. This area likely handles the raising or enabling of soft interrupts. I need to understand how the system signals that a particular soft interrupt needs processing. Is it based on a counter? Are there specific flags or bits being set in a control register? I'm hoping to see a clear explanation of the interrupt acknowledge and the associated mechanism.
Finally, I'm going to check out lines 765-790. This area is likely the heart of the soft interrupt execution. I'll need to confirm my understanding of the execution environment - does it run in interrupt context or task context? Are there locks involved? What's the mechanism for handling nested or re-entrant soft interrupts? That should give me a good handle on the overall process flow and any potential race conditions.
|
| 142/4 |
2026/11/18 01:40 |
tool |
read-file |
0m
Args: map[File:kernel/softirq.c FirstLine:360 LineCount:40]
Results: map[Contents: 360:
361: static inline void invoke_softirq(void)
362: {
363: if (should_wake_ksoftirqd())
364: wakeup_softirqd();
365: }
366:
367: #define SCHED_SOFTIRQ_MASK BIT(SCHED_SOFTIRQ)
368:
369: /*
370: * flush_smp_call_function_queue() can raise a soft interrupt in a function
371: * call. On RT kernels this is undesired and the only known functionalities
372: * are in the block layer which is disabled on RT, and in the scheduler for
373: * idle load balancing. If soft interrupts get raised which haven't been
374: * raised before the flush, warn if it is not a SCHED_SOFTIRQ so it can be
375: * investigated.
376: */
377: void do_softirq_post_smp_call_flush(unsigned int was_pending)
378: {
379: unsigned int is_pending = local_softirq_pending();
380:
381: if (unlikely(was_pending != is_pending)) {
382: WARN_ON_ONCE(was_pending != (is_pending & ~SCHED_SOFTIRQ_MASK));
383: invoke_softirq();
384: }
385: }
386:
387: #else /* CONFIG_PREEMPT_RT */
388:
389: /*
390: * This one is for softirq.c-internal use, where hardirqs are disabled
391: * legitimately:
392: */
393: #ifdef CONFIG_TRACE_IRQFLAGS
394: void __local_bh_disable_ip(unsigned long ip, unsigned int cnt)
395: {
396: unsigned long flags;
397:
398: WARN_ON_ONCE(in_hardirq());
399:
]
|
| 143/4 |
2026/11/18 01:40 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 42933
output: 35
thoughts: 20
Thoughts: Okay, let's dive into this. Here's my take on those thoughts, as though they were my own:
**Analyzing `kernel/softirq.c` β Focusing on Line 765**
Alright, let's zero in on a specific point. I'm thinking we need to check out line 765 in `kernel/softirq.c`. Just reading the line number, I know we're likely in the heart of the soft interrupt handling mechanism. My initial guess is we're probably looking at a critical section in the `softirq` context, a check on whether to reschedule, or perhaps something related to enabling/disabling of a particular softirq. It is also possible that a particular critical section is being completed. With this location in mind I will read up and down the code to find the overall context. I will be looking for interactions, in the form of function calls or data structures. I am assuming a particular point of contention and wish to confirm the hypothesis.
|
| 144/4 |
2026/11/18 01:40 |
tool |
read-file |
0m
Args: map[File:kernel/softirq.c FirstLine:765 LineCount:40]
Results: map[Contents: 765: #endif
766:
767: static inline void __irq_exit_rcu(void)
768: {
769: #ifndef __ARCH_IRQ_EXIT_IRQS_DISABLED
770: local_irq_disable();
771: #else
772: lockdep_assert_irqs_disabled();
773: #endif
774: account_hardirq_exit(current);
775:
776: /*
777: * HARDIRQ_OFFSET is still set. Only the outermost interrupt handles
778: * softirqs, and only if it did not hit a softirq or BH disabled section.
779: */
780: if (irq_count() == HARDIRQ_OFFSET && local_softirq_pending()) {
781: /*
782: * If we left hrtimers unarmed, make sure to arm them now,
783: * before enabling interrupts to run softirq.
784: */
785: hrtimer_rearm_deferred();
786: invoke_softirq();
787: }
788:
789: /*
790: * Wake the timer thread even if the interrupt hit a softirq or a
791: * section with BHs disabled. Only the outermost interrupt can wake it.
792: */
793: if (IS_ENABLED(CONFIG_IRQ_FORCED_THREADING) && force_irqthreads() &&
794: local_timers_pending_force_th() &&
795: hardirq_count() == HARDIRQ_OFFSET)
796: wake_timersd();
797:
798: tick_irq_exit();
799: preempt_count_sub(HARDIRQ_OFFSET);
800: }
801:
802: /**
803: * irq_exit_rcu() - Exit an interrupt context without updating RCU
804: *
]
|
| 145/4 |
2026/11/18 01:40 |
llm |
codeexpert |
0m
Model:
Tokens:
input: 43903
output: 1290
thoughts: 6612
Thoughts: **Analysis of Migrate Disable/Enable in SoftIRQ Context**
I've carefully dissected the issue of `migrate_disable()` and `migrate_enable()` being called from within a softirq context, specifically the call chain from `packet_rcv` to `bpf_prog_run_pin_on_cpu`, which then calls the migration functions. My analysis clearly indicates a significant problem: These functions, designed for task context, are being invoked in an interruptible, potentially reentrant, and certainly *not* process context (i.e. on an interrupt) environment, leading to data races and potential system corruption.
Firstly, the core issue is that `migrate_disable()` and `migrate_enable()` are per-task primitives, modifying `current->migration_disabled` which is specific to *the currently running task.* In an interrupt or softirq context, `current` refers to the task that was interrupted, not a dedicated "interrupt task". These functions are meant to prevent that *particular* task from being migrated. Hardirqs are always bound to a specific CPU, and softirqs are generally (in non-RT) run on the interrupted task's stack on the CPU that handled the interrupt. These are not migratable contexts, thus these function calls are incorrect and misleading.
The primary concerns are about corruption of both `current->migration_disabled` (which, with a data race, can become negative) and the scheduler's `rq->nr_pinned` (the number of pinned tasks on a runqueue). Several race conditions are identified:
* **`rq->nr_pinned` Underflow:** If a hardirq interrupts a task between setting `migration_disabled = 0` and decrementing `nr_pinned` during `__migrate_enable()`, the softirq/interrupt's `migrate_disable()` will increment `nr_pinned`, leading to a double-decrement (by softirq and the resumed task) that causes `nr_pinned` to underflow. Because `nr_pinned` is an unsigned integer, the underflow causes a wrap-around to a large value, corrupting scheduler load-balancing.
* **Counter Corruption on `migration_disabled`:** The migration counter's increment/decrement operations can become corrupted due to softirq interrupts at the wrong time (i.e. in the middle of a non-atomic operation). This, again, can lead to negative or out-of-sync values for `migration_disabled`.
* **`WARN_ON_ONCE` Triggering:** The `__migrate_enable()` function includes a `WARN_ON_ONCE` check. Data races can trigger this by causing an unexpected zero or negative value for `migration_disabled`.
* **Inadvertent Affinity Changes:** The most subtle issue involves tasks with pending affinity changes (`p->cpus_ptr != &p->cpus_mask`). If `migrate_enable()` is invoked in the softirq for a task in this state, it can lead to `___migrate_enable()`, essentially altering the task's CPU affinity from interrupt context, and disrupting pending affinity changes in a way that is not properly synchronized.
In summary, calling `migrate_disable()` and `migrate_enable()` from softirq context is fundamentally flawed due to its task-specific design that is inappropriate in a non-task context. This is further exacerbated by potential corruption of shared scheduler data and the high likelihood of underflows, data corruption, and scheduler instability. It is HIGHLY likely that this should be considered a bug.
*Caveats*: I have not verified the Linux git history for recent patches specifically addressing this race in the `bpf_prog_run_clear_cb` or related functions. I have also skipped verifying whether PREEMPT\_RT behaves any differently.
|
| 146/4 |
2026/11/18 01:41 |
tool |
set-results |
0m
Args: map[Answer:Based on analysis of the Linux kernel scheduler (`kernel/sched/core.c`, `include/linux/sched.h`) and BPF filter implementations (`include/linux/filter.h`, `net/packet/af_packet.c`):
### 1. Can `migrate_disable()` / `migrate_enable()` be called from softirq/interrupt context?
**No, they are not intended or safe to be called from softirq or interrupt context.**
* `migrate_disable()` and `migrate_enable()` are per-task primitives designed strictly for preemptible task context. They operate on `current->migration_disabled`. In softirq and interrupt contexts, `current` is simply whichever arbitrary task was interrupted by the IRQ on that CPU.
* Furthermore, softirqs and interrupt handlers are already inherently bound to the CPU executing them and cannot migrate across CPUs. Invoking `migrate_disable()` in softirq context is redundant and violates the design contract of `migrate_disable()`.
* The call chain `packet_rcv -> run_filter -> bpf_prog_run_clear_cb -> bpf_prog_run_pin_on_cpu` mistakenly invokes `migrate_disable()` / `migrate_enable()` in softirq context because `bpf_prog_run_clear_cb()` unconditionally uses `bpf_prog_run_pin_on_cpu()`.
---
### 2. Is this race harmful or benign? Can `current->migration_disabled` or `rq->nr_pinned` get corrupted or lead to `WARN_ON`?
**The race is HARMFUL. It can corrupt `rq->nr_pinned`, corrupt `current->migration_disabled`, and trigger `WARN_ON_ONCE`.**
Specific failure mechanisms:
1. **`rq->nr_pinned` Underflow / Corruption:**
In `__migrate_enable()`:
```c
barrier();
p->migration_disabled = 0;
/* <-- WINDOW OF VULNERABILITY */
this_rq_pinned()--;
```
`guard(preempt)()` disables preemption, but does *not* disable hardware interrupts. If a hardirq interrupts the CPU right after `p->migration_disabled = 0;` but before `this_rq_pinned()--;`, softirq processing (e.g. on `irq_exit()`) can run `packet_rcv`:
- Inside the softirq, `migrate_disable()` inspects `p = current`. Because `p->migration_disabled` was already set to 0 by the task, it takes the slow path, increments `this_rq_pinned()++`, and sets `p->migration_disabled = 1`.
- When the BPF filter finishes, `migrate_enable()` in the softirq sets `p->migration_disabled = 0` and decrements `this_rq_pinned()--`.
- Control returns to the interrupted task, which now executes its pending `this_rq_pinned()--;`.
- **Result:** `this_rq_pinned()` (i.e., `rq->nr_pinned`) is decremented one extra time without a matching increment. Since `rq->nr_pinned` is an `unsigned int`, it underflows to `UINT_MAX` (4,294,967,295), permanently corrupting the runqueue's pinned task count and breaking scheduler migration/load-balancing logic.
2. **`current->migration_disabled` Corruption via Non-Atomic Updates:**
- `migration_disabled` is an `unsigned short` modified with plain `++` and `--` without atomic operations or interrupt disabling.
- If a softirq interrupts while the task is executing `p->migration_disabled--` (when nesting count > 1) or `p->migration_disabled++`, concurrent modification causes lost updates or incorrect values.
3. **`WARN_ON_ONCE` Triggers:**
- With `CONFIG_DEBUG_PREEMPT=y`, `__migrate_enable()` contains:
```c
if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
return;
```
and `__migrate_disable()` checks:
```c
WARN_ON_ONCE((s16)p->migration_disabled < 0);
```
Underflow or desynchronization between the interrupted task and the softirq will trigger these warnings.
4. **Inadvertent Affinity Restoration (`___migrate_enable()`):**
- If the interrupted task had `p->cpus_ptr != &p->cpus_mask` (e.g. pending affinity change while migration was disabled), `migrate_enable()` running in the softirq will call `___migrate_enable()`, which calls `__set_cpus_allowed_ptr()` with task/runqueue locks from softirq context for the interrupted task.
---
### 3. Caveats and Unverified Parts
- Due to research tool limits, we did not verify whether a specific fix patch for `bpf_prog_run_clear_cb` / `packet_rcv` has been merged in the very latest upstream git HEAD or mailing lists (e.g., whether `bpf_prog_run_clear_cb` should use `bpf_prog_run` instead of `bpf_prog_run_pin_on_cpu` when in softirq context).
- On `PREEMPT_RT`, softirqs run in dedicated per-CPU ksoftirqd threads where `current` is the ksoftirqd task rather than the interrupted user/kernel thread; the analysis above applies primarily to standard (non-RT) kernels where softirqs run in interrupt context interrupting arbitrary tasks.]
Results: map[Answer:Based on analysis of the Linux kernel scheduler (`kernel/sched/core.c`, `include/linux/sched.h`) and BPF filter implementations (`include/linux/filter.h`, `net/packet/af_packet.c`):
### 1. Can `migrate_disable()` / `migrate_enable()` be called from softirq/interrupt context?
**No, they are not intended or safe to be called from softirq or interrupt context.**
* `migrate_disable()` and `migrate_enable()` are per-task primitives designed strictly for preemptible task context. They operate on `current->migration_disabled`. In softirq and interrupt contexts, `current` is simply whichever arbitrary task was interrupted by the IRQ on that CPU.
* Furthermore, softirqs and interrupt handlers are already inherently bound to the CPU executing them and cannot migrate across CPUs. Invoking `migrate_disable()` in softirq context is redundant and violates the design contract of `migrate_disable()`.
* The call chain `packet_rcv -> run_filter -> bpf_prog_run_clear_cb -> bpf_prog_run_pin_on_cpu` mistakenly invokes `migrate_disable()` / `migrate_enable()` in softirq context because `bpf_prog_run_clear_cb()` unconditionally uses `bpf_prog_run_pin_on_cpu()`.
---
### 2. Is this race harmful or benign? Can `current->migration_disabled` or `rq->nr_pinned` get corrupted or lead to `WARN_ON`?
**The race is HARMFUL. It can corrupt `rq->nr_pinned`, corrupt `current->migration_disabled`, and trigger `WARN_ON_ONCE`.**
Specific failure mechanisms:
1. **`rq->nr_pinned` Underflow / Corruption:**
In `__migrate_enable()`:
```c
barrier();
p->migration_disabled = 0;
/* <-- WINDOW OF VULNERABILITY */
this_rq_pinned()--;
```
`guard(preempt)()` disables preemption, but does *not* disable hardware interrupts. If a hardirq interrupts the CPU right after `p->migration_disabled = 0;` but before `this_rq_pinned()--;`, softirq processing (e.g. on `irq_exit()`) can run `packet_rcv`:
- Inside the softirq, `migrate_disable()` inspects `p = current`. Because `p->migration_disabled` was already set to 0 by the task, it takes the slow path, increments `this_rq_pinned()++`, and sets `p->migration_disabled = 1`.
- When the BPF filter finishes, `migrate_enable()` in the softirq sets `p->migration_disabled = 0` and decrements `this_rq_pinned()--`.
- Control returns to the interrupted task, which now executes its pending `this_rq_pinned()--;`.
- **Result:** `this_rq_pinned()` (i.e., `rq->nr_pinned`) is decremented one extra time without a matching increment. Since `rq->nr_pinned` is an `unsigned int`, it underflows to `UINT_MAX` (4,294,967,295), permanently corrupting the runqueue's pinned task count and breaking scheduler migration/load-balancing logic.
2. **`current->migration_disabled` Corruption via Non-Atomic Updates:**
- `migration_disabled` is an `unsigned short` modified with plain `++` and `--` without atomic operations or interrupt disabling.
- If a softirq interrupts while the task is executing `p->migration_disabled--` (when nesting count > 1) or `p->migration_disabled++`, concurrent modification causes lost updates or incorrect values.
3. **`WARN_ON_ONCE` Triggers:**
- With `CONFIG_DEBUG_PREEMPT=y`, `__migrate_enable()` contains:
```c
if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
return;
```
and `__migrate_disable()` checks:
```c
WARN_ON_ONCE((s16)p->migration_disabled < 0);
```
Underflow or desynchronization between the interrupted task and the softirq will trigger these warnings.
4. **Inadvertent Affinity Restoration (`___migrate_enable()`):**
- If the interrupted task had `p->cpus_ptr != &p->cpus_mask` (e.g. pending affinity change while migration was disabled), `migrate_enable()` running in the softirq will call `___migrate_enable()`, which calls `__set_cpus_allowed_ptr()` with task/runqueue locks from softirq context for the interrupted task.
---
### 3. Caveats and Unverified Parts
- Due to research tool limits, we did not verify whether a specific fix patch for `bpf_prog_run_clear_cb` / `packet_rcv` has been merged in the very latest upstream git HEAD or mailing lists (e.g., whether `bpf_prog_run_clear_cb` should use `bpf_prog_run` instead of `bpf_prog_run_pin_on_cpu` when in softirq context).
- On `PREEMPT_RT`, softirqs run in dedicated per-CPU ksoftirqd threads where `current` is the ksoftirqd task rather than the interrupted user/kernel thread; the analysis above applies primarily to standard (non-RT) kernels where softirqs run in interrupt context interrupting arbitrary tasks.]
|
| 147/2 |
2026/11/18 01:41 |
llm |
expert |
0m
Model:
Tokens:
input: 101995
output: 34
thoughts: 5910
Thoughts: **Deep Dive into the `rq->nr_pinned` Underflow and Potential Data Races**
Okay, so I've been poring over this CodeExpert response, and I'm starting to get a clearer picture, but there are a few subtle points that need verification, especially with respect to the compiler and interrupts. The core of the issue seems to stem from a window of vulnerability when `p->migration_disabled` is being manipulated in conjunction with the `this_rq_pinned()` counter. The initial analysis of CodeExpert focuses on the possibility of a hard interrupt occurring precisely after `p->migration_disabled` is set to 0 but before the decrement of `this_rq_pinned()`.
My initial thought, and what CodeExpert explains, is that a hard interrupt would trigger a softirq, which could call `migrate_disable()`. This, in turn, could lead to an increment of `this_rq_pinned()` if the migration is actually disabled and the `p->migration_disabled` is 0. Then, when the softirq calls `migrate_enable()`, `this_rq_pinned()` gets decremented. If the interrupt returns to the original task that *also* decrements `this_rq_pinned()`, we might have an underflow. However, after tracing that scenario, it seems as though this *doesn't* lead to an underflow, since the softirq increment and decrement actually balance.
So, where is the underflow? I began to explore different scenarios. The underflow could potentially occur if `__migrate_disable()` is called, and an interrupt occurs *before* `p->migration_disabled` is set to 1. In this scenario, `this_rq_pinned()` *is* incremented, but the subsequent setting of `p->migration_disabled` might be missed, leading to a race where the count can be unbalanced. But again, after analysis, I believe the task and softirq increment/decrement again balanced out.
Next, I considered another scenario: what if a task *initially* has `p->migration_disabled == 2`? The `__migrate_enable()` function has a check in place for this. If `p->migration_disabled` is read as 2 and an interrupt occurs, the softirq would see the current value, increment it (to 3), and *then* decrement it back to 2, before resuming. However, the task would return, and decrement to 1, thus again leading to a possible data race and inconsistent counter.
The key question is the atomicity of the `p->migration_disabled` write, it's a 16-bit short, and so atomic. This means it is safe.
Next, I looked at a possible problem due to affinity, but I concluded that this isn't likely the issue.
Finally, I got to what CodeExpert said about the "window of vulnerability". I thought about how the KCSAN report showed the data race. The key is in the timing between a task and an interrupt accessing shared memory, and a possible compiler optimization. Specifically, in `__migrate_enable()`, if the compiler were to hoist or reload `p->migration_disabled`, that could create a race, but this is unlikely to happen.
Ultimately, I think the vulnerability lies in the race condition between `__migrate_enable` and `__migrate_disable` within the same CPU. The KCSAN report seems to confirm this.
|
| 148/2 |
2026/11/18 01:42 |
tool |
grepper |
0m
Args: map[Expression:in_interrupt|in_task PathPrefixes:[kernel/kcsan/]]
Results: map[Output:kernel/kcsan/core.c=200=static __always_inline struct kcsan_ctx *get_ctx(void)
--
kernel/kcsan/core.c-205- */
kernel/kcsan/core.c:206: return in_task() ? ¤t->kcsan_ctx : raw_cpu_ptr(&kcsan_cpu_ctx);
kernel/kcsan/core.c-207-}
--
kernel/kcsan/core.c=322=static void delay_access(int type)
kernel/kcsan/core.c-323-{
kernel/kcsan/core.c:324: unsigned int delay = in_task() ? kcsan_udelay_task : kcsan_udelay_interrupt;
kernel/kcsan/core.c-325- /* For certain access types, skew the random delay to be longer. */
--
kernel/kcsan/core.c=803=void __init kcsan_init(void)
--
kernel/kcsan/core.c-806-
kernel/kcsan/core.c:807: BUG_ON(!in_task());
kernel/kcsan/core.c-808-
--
kernel/kcsan/report.c=642=static struct access_info prepare_access_info(const volatile void *ptr, size_t size,
--
kernel/kcsan/report.c-648- .access_type = access_type,
kernel/kcsan/report.c:649: .task_pid = in_task() ? task_pid_nr(current) : -1,
kernel/kcsan/report.c-650- .cpu_id = raw_smp_processor_id(),
]
|
| 149/2 |
2026/11/18 01:42 |
llm |
expert |
0m
Model:
Tokens:
input: 109134
output: 33
thoughts: 108
Thoughts: **KCSAN Interrupt Handling - A Deep Dive**
Okay, so I'm looking into how Kernel Concurrency Sanitizer (KCSAN) handles data races, specifically with regards to interrupts. Let's see...I'm focusing on `kernel/kcsan/core.c` right now. Ah, `delay_access`...it sets up a watchpoint, then delays execution using `udelay`. Clever. But wait, if an interrupt fires *during* that `udelay` on the same CPU, and the interrupt handler touches the very memory that's being watched...KCSAN should detect that! That's the key.
Hold on, I feel like I'm missing something, let me double-check the documentation. There *has* to be more detail in there. Ah, yes! `Documentation/dev-tools/kcsan.rst`. I'm going to jump right to the section on data races and interrupt handling to verify my understanding and see if I'm missing a nuance. I want to make sure I'm not oversimplifying how KCSAN accounts for these very tricky concurrency scenarios. Time to refresh my memory and see how the team thought about these cases in detail.
|
| 150/2 |
2026/11/18 01:42 |
tool |
grepper |
0m
Args: map[Expression:interrupt PathPrefixes:[Documentation/dev-tools/kcsan.rst]]
Results: map[Output:Documentation/dev-tools/kcsan.rst=177=changed via the corresponding Kconfig options.
--
Documentation/dev-tools/kcsan.rst-189-
Documentation/dev-tools/kcsan.rst:190:* ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For
Documentation/dev-tools/kcsan.rst:191: interrupts, the microsecond delay to stall execution after a watchpoint has
Documentation/dev-tools/kcsan.rst-192- been set up. Interrupts have tighter latency requirements, and their delay
]
|
| 151/2 |
2026/11/18 01:42 |
llm |
expert |
0m
Model:
Tokens:
input: 109699
output: 34
thoughts: 22
Thoughts: Here's my thought process, summarized as you requested:
**KCSAN Documentation Search Strategy**
Okay, so I need to find something relevant in the KCSAN documentation. My immediate instinct is to look for information about how interrupts are handled or potentially affect the tool's behavior. The keyword "interrupt" seems like a sensible starting point. I should be able to narrow down the search by specifying the file, `Documentation/dev-tools/kcsan.rst`. I'll focus my search query on that specific file, as itβs the most logical place to begin looking for information specific to KCSAN. Let's see what turns up. Hopefully, this approach will quickly lead me to the relevant sections of the documentation.
|
| 152/2 |
2026/11/18 01:42 |
tool |
grepper |
0m
Args: map[Expression:interrupts PathPrefixes:[Documentation/dev-tools/kcsan.rst]]
Results: map[Output:Documentation/dev-tools/kcsan.rst=177=changed via the corresponding Kconfig options.
--
Documentation/dev-tools/kcsan.rst-190-* ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For
Documentation/dev-tools/kcsan.rst:191: interrupts, the microsecond delay to stall execution after a watchpoint has
Documentation/dev-tools/kcsan.rst-192- been set up. Interrupts have tighter latency requirements, and their delay
]
|
| 153/2 |
2026/11/18 01:42 |
llm |
expert |
0m
Model:
Tokens:
input: 110059
output: 37
thoughts: 85
Thoughts: Okay, here's my summary, framed as my internal thought process:
**Diving into the Code's Recent History**
Alright, I need to get a handle on recent activity related to these crucial BPF components. My immediate focus is on the `bpf_prog_run_pin_on_cpu` function and its related functions. I'll need to check the commits and discussions relating to related functions, such as `bpf_prog_run_clear_cb` and `run_filter`, especially in relation to the `af_packet` subsystem. This usually uncovers relevant changes.
The first thing to do is query git to find relevant changes. I'll start with this: `git log -S bpf_prog_run_clear_cb net/packet/af_packet.c`. This query should yield a list of commits that contain the string `bpf_prog_run_clear_cb`, and which have also touched the `af_packet.c` file. This narrows the scope significantly, so that I can zero-in on potentially critical context for the functions. This should give me a good starting point for understanding recent changes.
|
| 154/2 |
2026/11/18 01:43 |
tool |
git-log |
0m
Args: map[CodeRegexp:bpf_prog_run_clear_cb PathPrefix:net/packet/]
Results: map[Output:ff936a04e5f2 bpf: fix cb access in socket filter programs
]
|
| 155/2 |
2026/11/18 01:43 |
llm |
expert |
0m
Model:
Tokens:
input: 110463
output: 42
thoughts: 29
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience:
**Investigating a BPF Program Pinning Issue**
Alright, let's dive in. I suspect something might be amiss with how `bpf_prog_run_pin_on_cpu` is interacting with the filtering mechanisms, specifically given its potential impact on `include/linux/filter.h`. To get a handle on the recent changes and commits related to this function and file interaction, my first step is to trace the history. I'm going to kick off a `git log` search. I'll use the `-S` flag to specifically look for commits that *introduced* or *removed* the string "bpf\_prog\_run\_pin\_on\_cpu" within that include file. This direct search, with those precise keywords, should give me the most targeted results, avoiding extraneous noise. I'm hoping to quickly identify the potential culprits and understand the evolution of this particular interface within the BPF ecosystem. This targeted approach is much more efficient than a broad keyword search.
|
| 156/2 |
2026/11/18 01:43 |
tool |
git-log |
0m
Args: map[CodeRegexp:bpf_prog_run_pin_on_cpu PathPrefix:include/linux/filter.h]
Results: map[Output:3d9f773cf287 bpf: Use bpf_prog_run_pin_on_cpu() at simple call sites.
3c58482a382b bpf: Provide bpf_prog_run_pin_on_cpu() helper
]
|
| 157/2 |
2026/11/18 01:43 |
llm |
expert |
0m
Model:
Tokens:
input: 110838
output: 37
thoughts: 678
Thoughts: **Analyzing BPF Program Execution Context**
Okay, let's break this down. First, I'm taking a look at `bpf_prog_run_pin_on_cpu` in `include/linux/filter.h`. The comment here is key: *"Use in preemptible and therefore migratable context to make sure that the execution of the BPF program runs on one CPU."* It's *explicitly* stating that this function is designed for scenarios where preemption is possible, and the BPF program needs to stay on a single CPU.
Now, where is `bpf_prog_run_pin_on_cpu` actually used? I see it's called by `bpf_prog_run_clear_cb`. Then, I note the other function, `bpf_prog_run_save_cb`. This function also calls `migrate_disable()`. Interesting. And looking further, `bpf_prog_run_save_cb` is called in `sk_filter_trim_cap`, which, wait for it, is triggered during TCP/UDP receive, which means the execution context is in the softirq. Also, `packet_rcv` -> `run_filter` is called in the softirq. That's a red flag!
So, why are `bpf_prog_run_save_cb` and `bpf_prog_run_clear_cb` *also* calling `migrate_disable()`? That's what I want to understand. Time for a `git blame` on `include/linux/filter.h` around line 1104 to see who introduced that `migrate_disable()` call and what the reasoning was. I need to trace this back.
|
| 158/2 |
2026/11/18 01:43 |
tool |
git-blame |
0m
Args: map[End:1115 File:include/linux/filter.h Start:1095]
Results: map[Output:ff936a04e5f28 1095)
ff936a04e5f28 1096) return res;
ff936a04e5f28 1097) }
ff936a04e5f28 1098)
6cab5e90ab2bd 1099) static inline u32 bpf_prog_run_save_cb(const struct bpf_prog *prog,
6cab5e90ab2bd 1100) struct sk_buff *skb)
6cab5e90ab2bd 1101) {
6cab5e90ab2bd 1102) u32 res;
6cab5e90ab2bd 1103)
2a916f2f546ca 1104) migrate_disable();
6cab5e90ab2bd 1105) res = __bpf_prog_run_save_cb(prog, skb);
2a916f2f546ca 1106) migrate_enable();
6cab5e90ab2bd 1107) return res;
6cab5e90ab2bd 1108) }
6cab5e90ab2bd 1109)
ff936a04e5f28 1110) static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
ff936a04e5f28 1111) struct sk_buff *skb)
ff936a04e5f28 1112) {
01dd194c387af 1113) u8 *cb_data = bpf_skb_cb(skb);
6cab5e90ab2bd 1114) u32 res;
ff936a04e5f28 1115)
]
|
| 159/2 |
2026/11/18 01:43 |
llm |
expert |
0m
Model:
Tokens:
input: 112465
output: 27
thoughts: 51
Thoughts: Okay, here's my interpretation of those directives, rendered as a first-person account of the thought process, assuming I'm an expert in the field:
**Examining Recent Commits**
Right, let's dive into this. The first instruction is clear β I need to scrutinize commits `2a916f2f546ca` and `6cab5e90ab2bd`. I'll start with the former, commit `2a916f2f546ca`. I need to understand the change that commit introduces into the codebase. I need to understand what changed, and how it may impact the overall design and system stability. I need to quickly assess the code's impact and its potential side effects on other parts of the system.
|
| 160/2 |
2026/11/18 01:43 |
tool |
git-show |
0m
Args: map[Commit:2a916f2f546ca]
Results: map[Output:commit 2a916f2f546ca1c1e3323e2a4269307f6d9890eb
Author: David Miller <davem@davemloft.net>
Date: Mon Feb 24 15:01:46 2020 +0100
bpf: Use migrate_disable/enable in array macros and cgroup/lirc code.
Replace the preemption disable/enable with migrate_disable/enable() to
reflect the actual requirement and to allow PREEMPT_RT to substitute it
with an actual migration disable mechanism which does not disable
preemption.
Including the code paths that go via __bpf_prog_run_save_cb().
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20200224145643.998293311@linutronix.de
diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 49b1a70e12c8..76b3a0eb1502 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -885,7 +885,7 @@ int bpf_prog_array_copy(struct bpf_prog_array *old_array,
struct bpf_prog *_prog; \
struct bpf_prog_array *_array; \
u32 _ret = 1; \
- preempt_disable(); \
+ migrate_disable(); \
rcu_read_lock(); \
_array = rcu_dereference(array); \
if (unlikely(check_non_null && !_array))\
@@ -898,7 +898,7 @@ int bpf_prog_array_copy(struct bpf_prog_array *old_array,
} \
_out: \
rcu_read_unlock(); \
- preempt_enable(); \
+ migrate_enable(); \
_ret; \
})
@@ -932,7 +932,7 @@ _out: \
u32 ret; \
u32 _ret = 1; \
u32 _cn = 0; \
- preempt_disable(); \
+ migrate_disable(); \
rcu_read_lock(); \
_array = rcu_dereference(array); \
_item = &_array->items[0]; \
@@ -944,7 +944,7 @@ _out: \
_item++; \
} \
rcu_read_unlock(); \
- preempt_enable(); \
+ migrate_enable(); \
if (_ret) \
_ret = (_cn ? NET_XMIT_CN : NET_XMIT_SUCCESS); \
else \
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 9270de2a0df8..43b5e455d2f5 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -677,6 +677,7 @@ static inline u8 *bpf_skb_cb(struct sk_buff *skb)
return qdisc_skb_cb(skb)->data;
}
+/* Must be invoked with migration disabled */
static inline u32 __bpf_prog_run_save_cb(const struct bpf_prog *prog,
struct sk_buff *skb)
{
@@ -702,9 +703,9 @@ static inline u32 bpf_prog_run_save_cb(const struct bpf_prog *prog,
{
u32 res;
- preempt_disable();
+ migrate_disable();
res = __bpf_prog_run_save_cb(prog, skb);
- preempt_enable();
+ migrate_enable();
return res;
}
]
|
| 161/2 |
2026/11/18 01:43 |
llm |
expert |
0m
Model:
Tokens:
input: 113865
output: 36
thoughts: 3495
Thoughts: **Analyzing the Thread Safety of `migrate_disable()` in Softirq Context**
Okay, so here's the deal. Back in 2020, Miller and Gleixner replaced `preempt_disable()` with `migrate_disable()` in BPF contexts. Initially, on non-RT kernels, `migrate_disable()` was literally `preempt_disable()`. But then, things got interesting. A commit shifted the logic to use `current->migration_disabled`, and further inlining happened. The crucial point is that, unlike RT kernels where softirqs are threaded, on non-RT kernels, they run in interrupt context. This makes me uneasy, so I need to analyze this.
My concern is whether it's safe to call `migrate_disable()` and `migrate_enable()` within a softirq context on a non-RT kernel. I need to understand exactly what happens under the hood. I know that `migrate_disable()` increments `migration_disabled` and disables migration, while `migrate_enable()` decrements it, potentially re-enabling migration. And `this_rq_pinned` is used, too.
First, let's dissect the implementation: `__migrate_disable()` checks `migration_disabled`. If it's already set, it increments it. Otherwise, it uses `guard(preempt)` which I know disables preemption, and increments `this_rq_pinned` and sets `migration_disabled` to 1. Then `__migrate_enable()` checks if `migration_disabled > 1`, and decrements. It calls `guard(preempt)` again and, if `cpus_ptr` is not the same as `cpus_mask`, it calls `___migrate_enable()`, and finally, resets `migration_disabled` to zero and decrements `this_rq_pinned()`. The critical line is `this_rq_pinned()--;`. The barrier is there too.
Now I need to imagine the race conditions, the interleaved execution between a regular task and a softirq. Imagine this scenario: a task calls `migrate_enable()`, and an interrupt hits right after the `migration_disabled > 1` check. The interrupt triggers a softirq. The softirq calls `migrate_disable()`. The softirq's `migration_disabled` will be different to the original task, causing increments and decrements, leading to incorrect state! And how does the compiler optimize the comparison? What if it's interrupted mid-instruction? I saw that KCSAN already reported a data race on `p->migration_disabled`...
Let's assume a few cases. If the task calls `migrate_enable()` but has `migration_disabled` as 1, the softirq can get in and increment it, causing problems when the task resumes. If the increment happens *before* `p->migration_disabled` is updated, the issue is that it is incremented and then decremented again. This can lead to multiple increments and decrements and confusion about the state. Similarly, a race during a `migrate_disable` call can cause `migration_disabled` to be double-incremented and then possibly double-decremented, causing a subtle off-by-one error or worse.
What if the interrupt happens between `this_rq_pinned()++;` and `p->migration_disabled = 1;`? What happens if `migrate_enable()` is called with `p->migration_disabled == 0`? The order matters. I need to check every single instruction, in every possible order.
In the case where we can call `___migrate_enable()`, the order is still critical. I need to read the code for `___migrate_enable()` and understand exactly what it does with `cpus_ptr`. I see that the order of `set_cpus_allowed_ptr` and `migrate_enable()` is also critical, because if there's a race, `migration_disabled` can be left incorrectly set.
Basically, I need a meticulous, instruction-by-instruction analysis of all possible interleavings, especially around the crucial increment/decrement of `migration_disabled` and `this_rq_pinned()`, under the light of how `guard(preempt)` works, and the impact of the compiler and memory model. I need to be *absolutely* sure this is safe.
|
| 162/2 |
2026/11/18 01:44 |
tool |
read-file |
0m
Args: map[File:kernel/sched/core.c FirstLine:2385 LineCount:35]
Results: map[Contents:2385: * even sure that "rq" stays as the right runqueue!
2386: * But we don't care, since "task_on_cpu()" will
2387: * return false if the runqueue has changed and p
2388: * is actually now running somewhere else!
2389: */
2390: while (task_on_cpu(rq, p)) {
2391: if (!task_state_match(p, match_state))
2392: return 0;
2393: cpu_relax();
2394: }
2395:
2396: /*
2397: * Ok, time to look more closely! We need the rq
2398: * lock now, to be *sure*. If we're wrong, we'll
2399: * just go back and repeat.
2400: */
2401: rq = task_rq_lock(p, &rf);
2402: /*
2403: * If task is sched_delayed, force dequeue it, to avoid always
2404: * hitting the tick timeout in the queued case
2405: */
2406: if (p->se.sched_delayed)
2407: dequeue_task(rq, p, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
2408: trace_sched_wait_task(p);
2409: running = task_on_cpu(rq, p);
2410: queued = task_on_rq_queued(p);
2411: ncsw = 0;
2412: if ((match = __task_state_match(p, match_state))) {
2413: /*
2414: * When matching on p->saved_state, consider this task
2415: * still queued so it will wait.
2416: */
2417: if (match < 0)
2418: queued = 1;
2419: ncsw = p->nvcsw | LONG_MIN; /* sets MSB */
]
|
| 163/2 |
2026/11/18 01:44 |
llm |
expert |
0m
Model:
Tokens:
input: 118889
output: 30
thoughts: 20
Thoughts: Okay, here's my thought process rendered in the first person, assuming I'm an expert in kernel scheduling.
**Finding the Migration Enable Flag in the Scheduler**
Alright, time to get to the heart of this scheduling puzzle. I need to find where the `___migrate_enable` variable is defined and used. That sounds like something that would be buried deep within the scheduler's core logic. So, my initial hunch is that it's likely located in `kernel/sched/core.c`. Let's start there. That file holds the fundamental data structures and functions that orchestrate how the kernel handles process scheduling. That's the most logical place to begin my search. Let's see what that file reveals.
|
| 164/2 |
2026/11/18 01:44 |
tool |
grepper |
0m
Args: map[Expression:void ___migrate_enable PathPrefixes:[kernel/sched/]]
Results: map[Output:kernel/sched/core.c=2471=static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
--
kernel/sched/core.c-2487-
kernel/sched/core.c:2488:void ___migrate_enable(void)
kernel/sched/core.c-2489-{
]
|
| 165/2 |
2026/11/18 01:44 |
llm |
expert |
0m
Model:
Tokens:
input: 119190
output: 36
thoughts: 18
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person:
**Analyzing the Migration Enable Flag**
Alright, my internal thought process starts with a specific directive: let's zoom in on `___migrate_enable`. I'm immediately drawn to line 2488. I'm operating on the assumption that this variable or flag, `___migrate_enable`, is critically important to migration functionality within this codebase. The fact that I'm focusing on it suggests I'm likely debugging, reviewing a code change related to migrations, or trying to understand the control flow related to data movement within this system. I'm probably looking to determine its current state, how it's being set, what conditions influence it, and the impact it has on the migration process. My experience tells me this is likely a boolean variable, but depending on the context, could also be an enumerated type or a bitmask, hence my immediate desire to examine the code. Understanding its role will inform my next steps, whether that's to trace its usage, identify potential issues, or verify that it's behaving as expected within the greater migration process.
|
| 166/2 |
2026/11/18 01:44 |
tool |
read-file |
0m
Args: map[File:kernel/sched/core.c FirstLine:2485 LineCount:25]
Results: map[Contents:2485: do_set_cpus_allowed(p, &ac);
2486: }
2487:
2488: void ___migrate_enable(void)
2489: {
2490: struct task_struct *p = current;
2491: struct affinity_context ac = {
2492: .new_mask = &p->cpus_mask,
2493: .flags = SCA_MIGRATE_ENABLE,
2494: };
2495:
2496: __set_cpus_allowed_ptr(p, &ac);
2497: }
2498: EXPORT_SYMBOL_GPL(___migrate_enable);
2499:
2500: void migrate_disable(void)
2501: {
2502: __migrate_disable();
2503: }
2504: EXPORT_SYMBOL_GPL(migrate_disable);
2505:
2506: void migrate_enable(void)
2507: {
2508: __migrate_enable();
2509: }
]
|
| 167/2 |
2026/11/18 01:44 |
llm |
expert |
0m
Model:
Tokens:
input: 119813
output: 35
thoughts: 18
Thoughts: Okay, let's dive into this. Here's my take on those thoughts, as if they're my own:
**Investigating `__set_cpus_allowed_ptr`**
Alright, time to get to the heart of what's going on. My immediate thought is, "Let's crack open the hood and see what this `__set_cpus_allowed_ptr` function is really doing!" I know what it *sounds* like, based on the name - setting the allowed CPU affinity mask. But I'm not going to take that at face value. I need to understand the *implementation details*. What low-level shenanigans is it up to? Is it a simple bitwise operation, or is it digging into some deeper kernel structures? Are there any subtle side effects or race conditions I should be aware of? I want the full picture, the gritty details. I'm especially keen to see how it interacts with other CPU management mechanisms. Is there any chance of conflicts, or unexpected behavior when combined with other methods of setting affinity? Itβs not enough to know *what* it does; I need to understand *how* and *why* it does it. Let's see if there are any gotchas in there!
|
| 168/2 |
2026/11/18 01:44 |
tool |
grepper |
0m
Args: map[Expression:int __set_cpus_allowed_ptr PathPrefixes:[kernel/sched/]]
Results: map[Output:kernel/sched/core.c=3005=static int affine_move_task(struct rq *rq, struct task_struct *p, struct rq_flags *rf,
--
kernel/sched/core.c-3157- */
kernel/sched/core.c:3158:static int __set_cpus_allowed_ptr_locked(struct task_struct *p,
kernel/sched/core.c-3159- struct affinity_context *ctx,
--
kernel/sched/core.c-3242- */
kernel/sched/core.c:3243:int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx)
kernel/sched/core.c-3244-{
--
kernel/sched/sched.h=2979=extern void sched_balance_trigger(struct rq *rq);
kernel/sched/sched.h-2980-
kernel/sched/sched.h:2981:extern int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx);
kernel/sched/sched.h-2982-extern void set_cpus_allowed_common(struct task_struct *p, struct affinity_context *ctx);
]
|
| 169/2 |
2026/11/18 01:44 |
llm |
expert |
0m
Model:
Tokens:
input: 120449
output: 36
thoughts: 32
Thoughts: Okay, here's my interpretation of that thought process, tailored for an expert audience, and written in the first person:
**Diving into the CPU Affinity Implementation**
Alright, let's get down to brass tacks. I'm thinking, I need to understand how the CPU affinity is being handled at a fundamental level within the scheduler. My gut feeling tells me the `__set_cpus_allowed_ptr` function is going to be central to this, and I'm directing my attention specifically to line 3243 within `kernel/sched/core.c`. I expect to find the core logic for setting the CPU affinity mask of a task there. Given the naming, it's likely a pointer-based assignment, which means I should pay close attention to any locking or memory synchronization that's being used. I'll be looking for nuances in how the mask is copied, how the scheduler is notified of the change, and any potential race conditions that might be present in the logic. Time to dig in and get a grip on what's going on under the hood!
|
| 170/2 |
2026/11/18 01:44 |
tool |
read-file |
0m
Args: map[File:kernel/sched/core.c FirstLine:3240 LineCount:40]
Results: map[Contents:3240: * task must not exit() & deallocate itself prematurely. The
3241: * call is not atomic; no spinlocks may be held.
3242: */
3243: int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx)
3244: {
3245: struct rq_flags rf;
3246: struct rq *rq;
3247:
3248: rq = task_rq_lock(p, &rf);
3249: /*
3250: * Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_*
3251: * flags are set.
3252: */
3253: if (p->user_cpus_ptr &&
3254: !(ctx->flags & (SCA_USER | SCA_MIGRATE_ENABLE | SCA_MIGRATE_DISABLE)) &&
3255: cpumask_and(rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr))
3256: ctx->new_mask = rq->scratch_mask;
3257:
3258: return __set_cpus_allowed_ptr_locked(p, ctx, rq, &rf);
3259: }
3260:
3261: int set_cpus_allowed_ptr(struct task_struct *p, const struct cpumask *new_mask)
3262: {
3263: struct affinity_context ac = {
3264: .new_mask = new_mask,
3265: .flags = 0,
3266: };
3267:
3268: return __set_cpus_allowed_ptr(p, &ac);
3269: }
3270: EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr);
3271:
3272: /*
3273: * Change a given task's CPU affinity to the intersection of its current
3274: * affinity mask and @subset_mask, writing the resulting mask to @new_mask.
3275: * If user_cpus_ptr is defined, use it as the basis for restricting CPU
3276: * affinity or use cpu_online_mask instead.
3277: *
3278: * If the resulting mask is empty, leave the affinity unchanged and return
3279: * -EINVAL.
]
|
| 171/2 |
2026/11/18 01:44 |
llm |
expert |
0m
Model:
Tokens:
input: 121417
output: 31
thoughts: 2886
Thoughts: **Data Race in BPF: A Deep Dive**
Okay, let's break this down. First, I see `task_rq_lock` using `raw_spin_rq_lock` and `raw_spin_lock_irqsave` on `pi_lock`. Interesting, but not the focus here. The real kicker is this KCSAN report. This data race in `bpf_flow_dissect` and `bpf_prog_run_clear_cb` demands attention.
The report highlights a write and read to the *same* memory location (0xffff8881046d0478), but what's jumping out at me is that the write is from an *interrupt* context and it calls `migrate_disable`, and the read is from a task context, which also calls `migrate_enable`. That's a classic data race scenario! Why is `bpf_prog_run_clear_cb` calling those scheduler primitives?
Let's look at the code: `bpf_prog_run_clear_cb` is in `include/linux/filter.h`. It does a `memset` and then calls `bpf_prog_run_pin_on_cpu`, but it's *not* calling `migrate_disable()` and `migrate_enable()` directly. Who *is* calling `bpf_prog_run_clear_cb`? Examining the call sites, I see `run_filter` in `af_packet.c` and a few other places like `fanout_demux_bpf` in `af_packet.c`, and the `tun` device driver. So it's not JUST that initial call path.
Wait a second. What about *other* BPF functions, such as `bpf_prog_run_save_cb`? Let's check where that's called. This is a common function called in various places. `sk_filter_trim_cap` is a key caller, and it's called on *every* packet. `sk_filter_trim_cap` -> `bpf_prog_run_save_cb` -> `migrate_disable()`. I am seeing that EVERY packet received is calling `migrate_disable()` through `bpf_prog_run_save_cb`. This is concerning! This is happening during packet reception, within a softirq context. This is what's causing the problem.
Why is this race *now* visible? Let's trace it back. The key is **commit 378b7708194f**! It made `migrate_disable()` and `migrate_enable()` *inline* functions. Now that these functions are inlined, the compiler is seeing this race where it didn't before. It's only now visible with KCSAN because the inlining made the code more directly accessible to the race detector. Before, `kernel/sched/core.c` compiled separately, so the tool was blind to this cross-compilation unit data race! We need to understand the implications of this everywhere!
|
| 172/2 |
2026/11/18 01:44 |
tool |
read-file |
0m
Args: map[File:kernel/sched/Makefile FirstLine:1 LineCount:40]
Results: map[Contents: 1: # SPDX-License-Identifier: GPL-2.0
2:
3: CONTEXT_ANALYSIS_core.o := y
4: CONTEXT_ANALYSIS_fair.o := y
5:
6: # The compilers are complaining about unused variables inside an if(0) scope
7: # block. This is daft, shut them up.
8: ccflags-y += $(call cc-disable-warning, unused-but-set-variable)
9:
10: # These files are disabled because they produce non-interesting flaky coverage
11: # that is not a function of syscall inputs. E.g. involuntary context switches.
12: KCOV_INSTRUMENT := n
13:
14: # Disable KCSAN to avoid excessive noise and performance degradation. To avoid
15: # false positives ensure barriers implied by sched functions are instrumented.
16: KCSAN_SANITIZE := n
17: KCSAN_INSTRUMENT_BARRIERS := y
18:
19: ifneq ($(CONFIG_SCHED_OMIT_FRAME_POINTER),y)
20: # According to Alan Modra <alan@linuxcare.com.au>, the -fno-omit-frame-pointer is
21: # needed for x86 only. Why this used to be enabled for all architectures is beyond
22: # me. I suspect most platforms don't need this, but until we know that for sure
23: # I turn this off for IA-64 only. Andreas Schwab says it's also needed on m68k
24: # to get a correct value for the wait-channel (WCHAN in ps). --davidm
25: CFLAGS_core.o := $(PROFILING) -fno-omit-frame-pointer
26: endif
27:
28: # Branch profiling isn't noinstr-safe
29: ifdef CONFIG_TRACE_BRANCH_PROFILING
30: CFLAGS_build_policy.o += -DDISABLE_BRANCH_PROFILING
31: CFLAGS_build_utility.o += -DDISABLE_BRANCH_PROFILING
32: endif
33: #
34: # Build efficiency:
35: #
36: # These compilation units have roughly the same size and complexity - so their
37: # build parallelizes well and finishes roughly at once:
38: #
39: obj-y += core.o
40: obj-y += fair.o
]
|
| 173/2 |
2026/11/18 01:44 |
llm |
expert |
1m
Model:
Tokens:
input: 125665
output: 24
thoughts: 12046
Thoughts: **Data Race Analysis: `migration_disabled` in Interrupt Context**
Okay, so KCSAN flagged a potential data race involving `p->migration_disabled`, specifically when accessed within an interrupt context. This happened because the `migrate_disable()` and `migrate_enable()` functions, now inlined, are called from BPF programs that can run in softirq context. Let's break this down systematically, as an expert in this area.
First, the context: we're dealing with a reference counter (`migration_disabled`) controlling task migration state. When `migration_disabled` is zero, migration is enabled; otherwise, it's disabled. The race involves an interrupt handler (running a BPF program) potentially modifying `p->migration_disabled` while the *interrupted* task is also accessing it. The core issue is that BPF calls `migrate_disable()` (and thus, increments `migration_disabled` via softirq) from context which already has migration effectively disabled.
The critical question, as always, is: is this **BENIGN** or **HARMFUL**? Let's consult the established protocol. We need to determine if this race causes "incorrect behavior" due to a "synchronization failure."
I've examined the code for `__migrate_disable()` and `__migrate_enable()` very carefully. The compiler's behavior and potential instruction interleaving were crucial. The key is to understand that the "race" *isn't* a true concurrency issue. The interrupt and task are strictly interleaved on the *same* CPU. KCSAN's delay and interrupt firing caused the race condition to appear. No concurrent execution of tasks occurs.
Here's the analysis:
1. **Strict Interleaving:** The interrupt/softirq handler and the interrupted task run *on the same CPU* and *in strictly interleaved fashion*. They do not run concurrently, only interleaved. This means one task is paused while the softirq runs to completion.
2. **Balanced `migrate_disable`/`migrate_enable`:** Any `migrate_disable()` call by a softirq (usually running a BPF program) is *always* matched by a `migrate_enable()` call before the softirq returns control. In essence, the interrupt handler is merely incrementing a counter and immediately decrementing it.
* If `migration_disabled == 1` in the task, in the softirq, it becomes 2 and comes back to 1. No state changes in the interrupted task's context, because the softirq guarantees this.
* If `migration_disabled == 0` in the task, the softirq increments the counter to 1, sets the `this_rq_pinned()` variable to 1, and back to zero at the end of execution.
3. **No Data Corruption:** I've checked transient states: even if an interrupt *interferes* in the middle of executing `__migrate_disable()` or `__migrate_enable()` instructions, the values will be restored to their original values after the softirq completes. I even traced through cases where the value is incremented twice in softirq and decremented twice (or vice-versa), etc. There is no state corruption.
4. **No Multi-CPU Concurrency:** `migration_disabled` is per-task and on one CPU only. So no other CPU could possibly interfere.
5. **No Compiler Hazards:** There are no obvious compiler hazards. There is no reordering and no inventing of writes or hoising out of loops. Barriers prevent the compiler from creating hazards.
Thus, based on these points, this appears to be a *classic* KCSAN false positive, or, as the protocol states: a **BENIGN** data race. Specifically, under the rubric:
* **Safe Overwrites & Plain Scalar Accesses:** Writing the same value already present, or aligned word-sized scalar accesses (e.g., `read (marked)` vs plain `write`) where observing either the old or new value is safe.
In my judgment, marking these accesses would add needless complexity without fixing an actual problem. The *logic* is sound, even if the instrumentation points this out, due to the artificial delays introduced. The race is an artifact of the instrumentation rather than a fundamental flaw. No change is needed.
|
| 174/2 |
2026/11/18 01:46 |
tool |
git-log |
10m
Error:
Args: map[CodeRegexp:data_race.*migration_disabled]
Results:
|
| 175/2 |
2026/11/18 01:56 |
llm |
expert-compressor |
0m
Model:
Reply: ### 1. Approaches Tried and Results
- **Trace Analysis of KCSAN Report:**
- Evaluated report: Write to `0xffff8881046d0478` (2 bytes) during softirq (`packet_rcv` -> `run_filter` -> `bpf_prog_run_clear_cb` -> `bpf_prog_run_pin_on_cpu` -> `migrate_disable`) vs Read from `0xffff8881046d0478` (2 bytes) by task 5152 (`bpf_prog_test_run_flow_dissector` -> `bpf_flow_dissect` -> `bpf_prog_run_pin_on_cpu` -> `migrate_enable`).
- Both accesses occur on CPU 0.
- Identified target memory: `0xffff8881046d0478` corresponds to `p->migration_disabled` in `struct task_struct` (type `unsigned short`, 2 bytes).
- **Inspection of `__migrate_disable()` and `__migrate_enable()` (`include/linux/sched.h`):**
- Examined implementation of `__migrate_disable()` and `__migrate_enable()`.
- Found that `__migrate_enable()` checks `if (p->migration_disabled > 1)` before entering the `guard(preempt)()` critical section.
- At line 2457 of `include/linux/sched.h`, `p->migration_disabled` is read without disabling preemption or local interrupts/bottom-halves (BH).
- Hardirq occurred during the delay/read inside `__migrate_enable()`, and upon irq exit, softirq (`__do_softirq`) ran on CPU 0 while `current` was task 5152.
- **Inspection of BPF Helpers Calling `migrate_{disable,enable}()`:**
- Checked `bpf_prog_run_pin_on_cpu()` in `include/linux/filter.h`: explicitly wraps `bpf_prog_run()` with `migrate_disable()` and `migrate_enable()`.
- Checked `bpf_prog_run_clear_cb()` and `bpf_prog_run_save_cb()` in `include/linux/filter.h`: both invoke `migrate_disable()` / `migrate_enable()`.
- Traced callers of `bpf_prog_run_{clear,save}_cb()` across `af_packet.c`, `net/core/filter.c`, `tun.c`, etc. They are directly invoked in softirq (network RX) and hardirq contexts.
- **Git History Archeology:**
- Traced commit `378b7708194f` ("sched: Make migrate_{en,dis}able() inline") which moved `migrate_{disable,enable}` into `include/linux/sched.h`.
- Traced commit `3c58482a382b` / `3d9f773cf287` which introduced `bpf_prog_run_pin_on_cpu()` under the assumption that `migrate_disable()` mapped to `preempt_disable()` on non-RT kernels.
- Traced commit `74d862b682f5` ("sched: Make migrate_disable/enable() independent of RT") which decoupled `migrate_disable()` from `preempt_disable()` for SMP, making `migrate_disable()` directly manipulate `current->migration_disabled` and `rq->nr_pinned`.
---
### 2. Verified Facts
- **Target Variable:** `current->migration_disabled` (`unsigned short` in `struct task_struct`).
- **Accessing Functions:**
- Read: `__migrate_enable()` (`include/linux/sched.h:2457`):
```c
if (p->migration_disabled > 1) {
p->migration_disabled--;
return;
}
```
- Write: `__migrate_disable()` (`include/linux/sched.h:2490` or `2496`):
```c
if (p->migration_disabled) {
...
p->migration_disabled++;
return;
}
...
p->migration_disabled = 1;
```
- **Context & Execution Invariants:**
- Both accesses are on CPU 0.
- The interrupted thread is task 5152 executing `bpf_prog_run_pin_on_cpu` in task context.
- The interrupter is a softirq running in the context of the interrupted task (`current == task 5152`).
- `migrate_disable()` and `migrate_enable()` are called from softirq context (e.g., packet receive path `packet_rcv` -> `run_filter`).
- In `__migrate_enable()`, `guard(preempt)()` is only acquired **after** the `p->migration_disabled > 1` branch check and return. Furthermore, `guard(preempt)()` only disables preemption; it does **not** disable softirqs or local interrupts.
- Hence, softirqs can interrupt `__migrate_enable()` and `__migrate_disable()` at any point before or within `guard(preempt)()`.
---
### 3. Hypotheses and Open Lines of Investigation
- **Hypothesis 1: Harmful Race / Logic Corruption Due to Reentrancy from Softirq:**
- If task $T$ has `migration_disabled == 1` and calls `__migrate_enable()`:
1. Task $T$ loads `p->migration_disabled` (sees 1). Condition `p->migration_disabled > 1` is false.
2. Before or right around `p->migration_disabled = 0` (or before `this_rq_pinned()--`), softirq interrupts on CPU 0.
3. Softirq runs `migrate_disable()` on `current` (which is $T$).
- If $T$ has not yet set `migration_disabled = 0`: softirq sees `p->migration_disabled == 1`, increments it to 2.
- Softirq runs BPF program, then calls `migrate_enable()`. Softirq sees `p->migration_disabled == 2`, decrements it to 1, and returns!
- Now softirq finishes. Task $T$ resumes in `__migrate_enable()` where it was about to zero `migration_disabled` or decrement `p->migration_disabled--`!
- Does $T$ clear `migration_disabled = 0` and decrement `this_rq_pinned()--`? If so, the task exits its outer `migrate_enable()` with `migration_disabled == 0` and `this_rq_pinned` decremented properly.
4. What if the compiler reloads `p->migration_disabled`?
- Since `p->migration_disabled` is a plain load without `READ_ONCE()`, the compiler could hoist, duplicate, or reload `p->migration_disabled`.
- If reloaded after the softirq increments it to 2, `__migrate_enable()` might take the `> 1` branch, decrementing it to 1 and returning early, skipping `p->migration_disabled = 0` and skipping `this_rq_pinned()--`, leaving the task permanently pinned!
5. What if softirq interrupts between `p->migration_disabled = 0` and `this_rq_pinned()--`?
- Softirq calls `migrate_disable()`: sees `p->migration_disabled == 0`.
- Executes `guard(preempt)()`, increments `this_rq_pinned()++`, sets `p->migration_disabled = 1`.
- Softirq calls `migrate_enable()`: checks `cpus_ptr`, sets `p->migration_disabled = 0`, decrements `this_rq_pinned()--`.
- Softirq returns. Task resumes: decrements `this_rq_pinned()--`! Underflow of `this_rq_pinned()`!
- `rq->nr_pinned` underflows from 0 to `UINT_MAX`, corrupting scheduler state (`rq_has_pinned_tasks()` becomes permanently true).
- **Hypothesis 2: `migrate_disable()` is fundamentally illegal or unsupported in Softirq/Interrupt Context:**
- `migrate_disable()` operates on `current`. When called in softirq, it mutates `current->migration_disabled` of whatever arbitrary task happened to be interrupted.
- If softirqs are supposed to be migration-safe simply by virtue of being in softirq (`in_interrupt()`), calling `migrate_disable()` in softirq is either completely redundant or erroneous.
---
### 4. Critical Files, Functions, and Code Snippets
- **`include/linux/sched.h:2444-2497`:**
```c
static inline void __migrate_enable(void)
{
struct task_struct *p = current;
#ifdef CONFIG_DEBUG_PREEMPT
if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
return;
#endif
if (p->migration_disabled > 1) {
p->migration_disabled--;
return;
}
guard(preempt)();
if (unlikely(p->cpus_ptr != &p->cpus_mask))
___migrate_enable();
barrier();
p->migration_disabled = 0;
this_rq_pinned()--;
}
static inline void __migrate_disable(void)
{
struct task_struct *p = current;
if (p->migration_disabled) {
#ifdef CONFIG_DEBUG_PREEMPT
WARN_ON_ONCE((s16)p->migration_disabled < 0);
#endif
p->migration_disabled++;
return;
}
guard(preempt)();
this_rq_pinned()++;
p->migration_disabled = 1;
}
```
- **`include/linux/filter.h:831-840, 1110-1121`:**
```c
static inline u32 bpf_prog_run_pin_on_cpu(const struct bpf_prog *prog,
const void *ctx)
{
u32 ret;
migrate_disable();
ret = bpf_prog_run(prog, ctx);
migrate_enable();
return ret;
}
static inline u32 bpf_prog_run_clear_cb(const struct bpf_prog *prog,
struct sk_buff *skb)
{
u8 *cb_data = bpf_skb_cb(skb);
u32 res;
if (unlikely(prog->cb_access))
memset(cb_data, 0, BPF_SKB_CB_LEN);
res = bpf_prog_run_pin_on_cpu(prog, skb);
return res;
}
```
- **`kernel/sched/core.c:2512-2515`:**
```c
static inline bool rq_has_pinned_tasks(struct rq *rq)
{
return rq->nr_pinned;
}
```
---
### 5. Potential Loops / Redundant Operations to Avoid
- Do not run more generic `git-log` searches without a tight `PathPrefix` (they timed out twice).
- Do not keep looking for other callers of `migration_disabled` across all scheduler extensions; the core scheduler behavior and the definition in `include/linux/sched.h` are already known.
- Stop researching background history. The code in front of us is current mainline code.
---
### 6. Concrete Next Steps
1. **Verify the underflow/corruption data flow:**
- Specifically trace the exact sequence:
```text
Task (CPU 0):
__migrate_enable():
barrier();
p->migration_disabled = 0;
<INTERRUPT / SOFTIRQ>
__migrate_disable():
p->migration_disabled is 0!
this_rq_pinned()++; // nr_pinned goes from 1 to 2
p->migration_disabled = 1;
bpf_prog_run(...);
__migrate_enable():
p->migration_disabled is 1 (not > 1)
p->migration_disabled = 0;
this_rq_pinned()--; // nr_pinned goes from 2 to 1
<RETURN FROM SOFTIRQ>
this_rq_pinned()--; // nr_pinned goes from 1 to 0 (or from 0 to underflow if initial nr_pinned was 0)
```
- Check if `p->migration_disabled` can also suffer a lost update or TOCTOU re-fetch leading to `WARN_ON_ONCE` or `nr_pinned` underflow/mismatch.
2. Check whether `cant_migrate()` or other checks enforce anything about softirq calling `migrate_disable()`.
3. Conclude whether this is **HARMFUL** (logic bug / concurrency hazard causing `nr_pinned` corruption, lost updates, or crash) or **BENIGN**.
4. Call `set-results` and provide the final report following the required protocol.
Tokens:
input: 67071
output: 3097
thoughts: 0
|
| 176/2 |
2026/11/18 01:56 |
llm |
expert |
0m
Model:
Tokens:
input: 50943
output: 38
thoughts: 31
Thoughts: Okay, here's my take on those thoughts, framed as a first-person expert summary:
**Digging into the Scheduler's History**
Alright, time to get my hands dirty. I'm thinking I need to delve into the evolution of the Linux scheduler. To do that effectively, I'm going to start by leveraging `git log`. My focus is going to be on the `kernel/sched/` directory, specifically anything that's changed in the scheduler core. But, of course, the scheduler's not an island. I know changes in `include/linux/sched.h` can also be extremely important, as that header file defines all the core structures and interfaces that the scheduler relies on, and exposes to the rest of the kernel. So, to get a holistic view, I'll restrict my `git log` search to changes within the `kernel/sched/` directory and also including changes to `include/linux/sched.h`. This should give me a good handle on its development and hopefully illuminate any specific concerns or questions I have. I'm expecting this will provide a targeted history I can really sink my teeth into.
|
| 177/2 |
2026/11/18 01:56 |
tool |
git-log |
0m
Args: map[Count:10 MessageRegexps:[migrate_disable] PathPrefix:include/linux/sched.h]
Results: map[Output:14a7f2392f42 bpf: Make migrate_disable always inline to avoid partial inlining
378b7708194f sched: Make migrate_{en,dis}able() inline
74d862b682f5 sched: Make migrate_disable/enable() independent of RT
a7c81556ec4d sched: Fix migrate_disable() vs rt/dl balancing
6d337eab041d sched: Fix migrate_disable() vs set_cpus_allowed_ptr()
af449901b84c sched: Add migrate_disable()
3bd3706251ee sched/core: Provide a pointer to the valid CPU mask
]
|
| 178/2 |
2026/11/18 01:56 |
llm |
expert |
0m
Model:
Tokens:
input: 51439
output: 27
thoughts: 43
Thoughts: Okay, here's my interpretation of that commit message, expressed as my internal thought process:
**Diving into that BPF Commit**
Alright, let's take a look at commit `14a7f2392f42`. Ah, "bpf: Make migrate_disable always inline to avoid partial inlining". Right, this is a classic optimization and potential performance trap in BPF land. My immediate instinct is to think about the implications on instruction scheduling and register pressure. "Partial inlining" immediately sets off alarm bells, especially in performance-critical code paths, like BPF. It's a problem I've seen pop up numerous times in the past. If `migrate_disable` isn't fully inlined, the compiler is forced to make a trade-off. It either inlines a portion of the function or it leaves it un-inlined. The former is not optimized to its full potential, while the latter incurs function call overhead, extra stack operations and likely increased register pressure.
The goal here is to ensure that the compiler makes the optimal decision in the situation. This commit's author probably observed either an issue with branch mispredictions or a degradation in performance due to this partial inlining behaviour. Iβd like to see the compiler output before and after this change, ideally disassembling the generated BPF code. The commit likely ensures `migrate_disable` is small and simple enough to be *always* inlined, or uses a compiler directive to enforce the inlining, to avoid any of these problems. It's a smart move to force full inlining in a low-level, high-frequency context. Let's see what the actual patch looks like. I'd be looking for the `inline` keyword, maybe a `__always_inline`, or some specific compiler attribute to enforce inlining. The devil is always in the details with this kind of optimization. I'd need to assess whether the added inline function size imposes a large performance cost elsewhere. I'd also be curious about the testing methodology; I assume it was benchmarked and the results are showing improvements, otherwise, it would be a risky change to make. Okay, let's check it out.
|
| 179/2 |
2026/11/18 01:57 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 14a7f2392f42bbb71c1a5ea68930006221fcd80a
Author: Yonghong Song <yonghong.song@linux.dev>
Date: Wed Oct 29 11:36:46 2025 -0700
bpf: Make migrate_disable always inline to avoid partial inlining
The build fails with llvm 21/22:
$ make LLVM=1 -j
...
LD vmlinux.o
GEN .vmlinux.objs
...
BTF .tmp_vmlinux1.btf.o
...
AS .tmp_vmlinux2.kallsyms.o
LD vmlinux.unstripped
BTFIDS vmlinux.unstripped
WARN: resolve_btfids: unresolved symbol migrate_enable
WARN: resolve_btfids: unresolved symbol migrate_disable
make[2]: *** [vmlinux.unstripped] Error 255
make[2]: *** Deleting file 'vmlinux.unstripped'
make[1]: *** [Makefile:1242: vmlinux] Error 2
make: *** [Makefile:248: __sub-make] Error 2
Two functions with identical names but different addresses are
considered ambiguous and removed by "pahole" from vmlinux BTF.
Later resolve_btfids warns since it cannot find them.
Commit 378b7708194f ("sched: Make migrate_{en,dis}able() inline") made
them inlineable in most places, but in vmlinux built with llvm 21 and 22
there are four symbols for migrate_{enable,disable}:
three static functions and one global function.
Fix the issue by marking migrate_{enable,disable} as always inline.
The alternative is to mark them as notrace/nokprobe which is more
drastic. Only bpf programs are prevented from attaching to these
functions. The rest of the tracing shouldn't be affected.
[note: Peter ok-ed the patch, Alexei rewrote commit log]
Fixes: 378b7708194f ("sched: Make migrate_{en,dis}able() inline")
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Acked-by: Menglong Dong <menglong.dong@linux.dev>
Link: https://lore.kernel.org/r/20251029183646.3811774-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
diff --git a/include/linux/sched.h b/include/linux/sched.h
index cbb7340c5866..b469878de25c 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2407,12 +2407,12 @@ static inline void __migrate_enable(void) { }
* be defined in kernel/sched/core.c.
*/
#ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
-static inline void migrate_disable(void)
+static __always_inline void migrate_disable(void)
{
__migrate_disable();
}
-static inline void migrate_enable(void)
+static __always_inline void migrate_enable(void)
{
__migrate_enable();
}
]
|
| 180/2 |
2026/11/18 01:57 |
llm |
expert |
0m
Model:
Tokens:
input: 52875
output: 27
thoughts: 34
Thoughts: Okay, here's my take on those thoughts, tailored for an expert audience:
**Investigating a Code Inline**
Alright, I need to dig into commit `378b7708194f`. The fact that something was done inline *immediately* raises a flag. We all know the potential pitfalls: increased complexity, reduced readability if not done with care, and the risk of tightly coupling things that ideally should be modularized. So, I want to understand the *reasoning* behind that choice. I'm going to look at the diff itself, of course, but the commit message is going to be crucial. A good message will explain the problem, the solution's rationale, and any potential tradeoffs considered. Hopefully, it's not a lazy "Fixed a bug" or even worse, no message at all! I'm hoping to understand the context. Was this a quick fix under pressure? Was it a conscious decision based on performance constraints or some other architectural consideration that wasn't immediately obvious from the code? The message should tell the story, and the code itself should *support* the story that it tells. I'm going to determine if the inline change was the best solution, or if there is something else that could be better.
|
| 181/2 |
2026/11/18 01:57 |
tool |
git-show |
0m
Args:
Results: map[Output:commit 378b7708194fff77c9020392067329931c3fcc04
Author: Menglong Dong <menglong8.dong@gmail.com>
Date: Wed Sep 17 14:09:15 2025 +0800
sched: Make migrate_{en,dis}able() inline
For now, migrate_enable and migrate_disable are global, which makes them
become hotspots in some case. Take BPF for example, the function calling
to migrate_enable and migrate_disable in BPF trampoline can introduce
significant overhead, and following is the 'perf top' of FENTRY's
benchmark (./tools/testing/selftests/bpf/bench trig-fentry):
54.63% bpf_prog_2dcccf652aac1793_bench_trigger_fentry [k]
bpf_prog_2dcccf652aac1793_bench_trigger_fentry
10.43% [kernel] [k] migrate_enable
10.07% bpf_trampoline_6442517037 [k] bpf_trampoline_6442517037
8.06% [kernel] [k] __bpf_prog_exit_recur
4.11% libc.so.6 [.] syscall
2.15% [kernel] [k] entry_SYSCALL_64
1.48% [kernel] [k] memchr_inv
1.32% [kernel] [k] fput
1.16% [kernel] [k] _copy_to_user
0.73% [kernel] [k] bpf_prog_test_run_raw_tp
So in this commit, we make migrate_enable/migrate_disable inline to obtain
better performance. The struct rq is defined internally in
kernel/sched/sched.h, and the field "nr_pinned" is accessed in
migrate_enable/migrate_disable, which makes it hard to make them inline.
Alexei Starovoitov suggests to generate the offset of "nr_pinned" in [1],
so we can define the migrate_enable/migrate_disable in
include/linux/sched.h and access "this_rq()->nr_pinned" with
"(void *)this_rq() + RQ_nr_pinned".
The offset of "nr_pinned" is generated in include/generated/rq-offsets.h
by kernel/sched/rq-offsets.c.
Generally speaking, we move the definition of migrate_enable and
migrate_disable to include/linux/sched.h from kernel/sched/core.c. The
calling to __set_cpus_allowed_ptr() is leaved in ___migrate_enable().
The "struct rq" is not available in include/linux/sched.h, so we can't
access the "runqueues" with this_cpu_ptr(), as the compilation will fail
in this_cpu_ptr() -> raw_cpu_ptr() -> __verify_pcpu_ptr():
typeof((ptr) + 0)
So we introduce the this_rq_raw() and access the runqueues with
arch_raw_cpu_ptr/PERCPU_PTR directly.
The variable "runqueues" is not visible in the kernel modules, and export
it is not a good idea. As Peter Zijlstra advised in [2], we define and
export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
them for the modules.
Before this patch, the performance of BPF FENTRY is:
fentry : 113.030 Β± 0.149M/s
fentry : 112.501 Β± 0.187M/s
fentry : 112.828 Β± 0.267M/s
fentry : 115.287 Β± 0.241M/s
After this patch, the performance of BPF FENTRY increases to:
fentry : 143.644 Β± 0.670M/s
fentry : 149.764 Β± 0.362M/s
fentry : 149.642 Β± 0.156M/s
fentry : 145.263 Β± 0.221M/s
Signed-off-by: Menglong Dong <dongml2@chinatelecom.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/bpf/CAADnVQ+5sEDKHdsJY5ZsfGDO_1SEhhQWHrt2SMBG5SYyQ+jt7w@mail.gmail.com/ [1]
Link: https://lore.kernel.org/all/20250819123214.GH4067720@noisy.programming.kicks-ass.net/ [2]
diff --git a/Kbuild b/Kbuild
index f327ca86990c..13324b4bbe23 100644
--- a/Kbuild
+++ b/Kbuild
@@ -34,13 +34,24 @@ arch/$(SRCARCH)/kernel/asm-offsets.s: $(timeconst-file) $(bounds-file)
$(offsets-file): arch/$(SRCARCH)/kernel/asm-offsets.s FORCE
$(call filechk,offsets,__ASM_OFFSETS_H__)
+# Generate rq-offsets.h
+
+rq-offsets-file := include/generated/rq-offsets.h
+
+targets += kernel/sched/rq-offsets.s
+
+kernel/sched/rq-offsets.s: $(offsets-file)
+
+$(rq-offsets-file): kernel/sched/rq-offsets.s FORCE
+ $(call filechk,offsets,__RQ_OFFSETS_H__)
+
# Check for missing system calls
quiet_cmd_syscalls = CALL $<
cmd_syscalls = $(CONFIG_SHELL) $< $(CC) $(c_flags) $(missing_syscalls_flags)
PHONY += missing-syscalls
-missing-syscalls: scripts/checksyscalls.sh $(offsets-file)
+missing-syscalls: scripts/checksyscalls.sh $(rq-offsets-file)
$(call cmd,syscalls)
# Check the manual modification of atomic headers
diff --git a/include/linux/preempt.h b/include/linux/preempt.h
index 1fad1c8a4c76..92237c319035 100644
--- a/include/linux/preempt.h
+++ b/include/linux/preempt.h
@@ -424,8 +424,6 @@ static inline void preempt_notifier_init(struct preempt_notifier *notifier,
* work-conserving schedulers.
*
*/
-extern void migrate_disable(void);
-extern void migrate_enable(void);
/**
* preempt_disable_nested - Disable preemption inside a normally preempt disabled section
@@ -471,7 +469,6 @@ static __always_inline void preempt_enable_nested(void)
DEFINE_LOCK_GUARD_0(preempt, preempt_disable(), preempt_enable())
DEFINE_LOCK_GUARD_0(preempt_notrace, preempt_disable_notrace(), preempt_enable_notrace())
-DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
#ifdef CONFIG_PREEMPT_DYNAMIC
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 644a01bdae70..d60ecaccdffc 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -49,6 +49,9 @@
#include <linux/tracepoint-defs.h>
#include <linux/unwind_deferred_types.h>
#include <asm/kmap_size.h>
+#ifndef COMPILE_OFFSETS
+#include <generated/rq-offsets.h>
+#endif
/* task_struct member predeclarations (sorted alphabetically): */
struct audit_context;
@@ -2317,4 +2320,114 @@ static __always_inline void alloc_tag_restore(struct alloc_tag *tag, struct allo
#define alloc_tag_restore(_tag, _old) do {} while (0)
#endif
+#ifndef MODULE
+#ifndef COMPILE_OFFSETS
+
+extern void ___migrate_enable(void);
+
+struct rq;
+DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues);
+
+/*
+ * The "struct rq" is not available here, so we can't access the
+ * "runqueues" with this_cpu_ptr(), as the compilation will fail in
+ * this_cpu_ptr() -> raw_cpu_ptr() -> __verify_pcpu_ptr():
+ * typeof((ptr) + 0)
+ *
+ * So use arch_raw_cpu_ptr()/PERCPU_PTR() directly here.
+ */
+#ifdef CONFIG_SMP
+#define this_rq_raw() arch_raw_cpu_ptr(&runqueues)
+#else
+#define this_rq_raw() PERCPU_PTR(&runqueues)
+#endif
+#define this_rq_pinned() (*(unsigned int *)((void *)this_rq_raw() + RQ_nr_pinned))
+
+static inline void __migrate_enable(void)
+{
+ struct task_struct *p = current;
+
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ * Check both overflow from migrate_disable() and superfluous
+ * migrate_enable().
+ */
+ if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
+ return;
+#endif
+
+ if (p->migration_disabled > 1) {
+ p->migration_disabled--;
+ return;
+ }
+
+ /*
+ * Ensure stop_task runs either before or after this, and that
+ * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
+ */
+ guard(preempt)();
+ if (unlikely(p->cpus_ptr != &p->cpus_mask))
+ ___migrate_enable();
+ /*
+ * Mustn't clear migration_disabled() until cpus_ptr points back at the
+ * regular cpus_mask, otherwise things that race (eg.
+ * select_fallback_rq) get confused.
+ */
+ barrier();
+ p->migration_disabled = 0;
+ this_rq_pinned()--;
+}
+
+static inline void __migrate_disable(void)
+{
+ struct task_struct *p = current;
+
+ if (p->migration_disabled) {
+#ifdef CONFIG_DEBUG_PREEMPT
+ /*
+ *Warn about overflow half-way through the range.
+ */
+ WARN_ON_ONCE((s16)p->migration_disabled < 0);
+#endif
+ p->migration_disabled++;
+ return;
+ }
+
+ guard(preempt)();
+ this_rq_pinned()++;
+ p->migration_disabled = 1;
+}
+#else /* !COMPILE_OFFSETS */
+static inline void __migrate_disable(void) { }
+static inline void __migrate_enable(void) { }
+#endif /* !COMPILE_OFFSETS */
+
+/*
+ * So that it is possible to not export the runqueues variable, define and
+ * export migrate_enable/migrate_disable in kernel/sched/core.c too, and use
+ * them for the modules. The macro "INSTANTIATE_EXPORTED_MIGRATE_DISABLE" will
+ * be defined in kernel/sched/core.c.
+ */
+#ifndef INSTANTIATE_EXPORTED_MIGRATE_DISABLE
+static inline void migrate_disable(void)
+{
+ __migrate_disable();
+}
+
+static inline void migrate_enable(void)
+{
+ __migrate_enable();
+}
+#else /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
+extern void migrate_disable(void);
+extern void migrate_enable(void);
+#endif /* INSTANTIATE_EXPORTED_MIGRATE_DISABLE */
+
+#else /* MODULE */
+extern void migrate_disable(void);
+extern void migrate_enable(void);
+#endif /* MODULE */
+
+DEFINE_LOCK_GUARD_0(migrate, migrate_disable(), migrate_enable())
+
#endif
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index c4f69a9e9af6..de9078a9df3a 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -23855,6 +23855,7 @@ int bpf_check_attach_target(struct bpf_verifier_log *log,
BTF_SET_START(btf_id_deny)
BTF_ID_UNUSED
#ifdef CONFIG_SMP
+BTF_ID(func, ___migrate_enable)
BTF_ID(func, migrate_disable)
BTF_ID(func, migrate_enable)
#endif
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index feb750aae71b..7f1e5cb94c53 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7,6 +7,8 @@
* Copyright (C) 1991-2002 Linus Torvalds
* Copyright (C) 1998-2024 Ingo Molnar, Red Hat
*/
+#define INSTANTIATE_EXPORTED_MIGRATE_DISABLE
+#include <linux/sched.h>
#include <linux/highmem.h>
#include <linux/hrtimer_api.h>
#include <linux/ktime_api.h>
@@ -2381,28 +2383,7 @@ static void migrate_disable_switch(struct rq *rq, struct task_struct *p)
__do_set_cpus_allowed(p, &ac);
}
-void migrate_disable(void)
-{
- struct task_struct *p = current;
-
- if (p->migration_disabled) {
-#ifdef CONFIG_DEBUG_PREEMPT
- /*
- *Warn about overflow half-way through the range.
- */
- WARN_ON_ONCE((s16)p->migration_disabled < 0);
-#endif
- p->migration_disabled++;
- return;
- }
-
- guard(preempt)();
- this_rq()->nr_pinned++;
- p->migration_disabled = 1;
-}
-EXPORT_SYMBOL_GPL(migrate_disable);
-
-void migrate_enable(void)
+void ___migrate_enable(void)
{
struct task_struct *p = current;
struct affinity_context ac = {
@@ -2410,35 +2391,19 @@ void migrate_enable(void)
.flags = SCA_MIGRATE_ENABLE,
};
-#ifdef CONFIG_DEBUG_PREEMPT
- /*
- * Check both overflow from migrate_disable() and superfluous
- * migrate_enable().
- */
- if (WARN_ON_ONCE((s16)p->migration_disabled <= 0))
- return;
-#endif
+ __set_cpus_allowed_ptr(p, &ac);
+}
+EXPORT_SYMBOL_GPL(___migrate_enable);
- if (p->migration_disabled > 1) {
- p->migration_disabled--;
- return;
- }
+void migrate_disable(void)
+{
+ __migrate_disable();
+}
+EXPORT_SYMBOL_GPL(migrate_disable);
- /*
- * Ensure stop_task runs either before or after this, and that
- * __set_cpus_allowed_ptr(SCA_MIGRATE_ENABLE) doesn't schedule().
- */
- guard(preempt)();
- if (p->cpus_ptr != &p->cpus_mask)
- __set_cpus_allowed_ptr(p, &ac);
- /*
- * Mustn't clear migration_disabled() until cpus_ptr points back at the
- * regular cpus_mask, otherwise things that race (eg.
- * select_fallback_rq) get confused.
- */
- barrier();
- p->migration_disabled = 0;
- this_rq()->nr_pinned--;
+void migrate_enable(void)
+{
+ __migrate_enable();
}
EXPORT_SYMBOL_GPL(migrate_enable);
diff --git a/kernel/sched/rq-offsets.c b/kernel/sched/rq-offsets.c
new file mode 100644
index 000000000000..a23747bbe25b
--- /dev/null
+++ b/kernel/sched/rq-offsets.c
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0
+#define COMPILE_OFFSETS
+#include <linux/kbuild.h>
+#include <linux/types.h>
+#include "sched.h"
+
+int main(void)
+{
+ DEFINE(RQ_nr_pinned, offsetof(struct rq, nr_pinned));
+
+ return 0;
+}
]
|
| 182/2 |
2026/11/18 01:57 |
llm |
expert |
0m
Model:
Tokens:
input: 57695
output: 15
thoughts: 1145
Thoughts: **Analyzing a Benign Data Race in `migrate_enable` and `migrate_disable`**
Okay, so commit 378b7708194f moved `migrate_enable` and `migrate_disable` to a header file, which has caused KCSAN to start flagging their use in callers like `net/core/flow_dissector.c` and `net/packet/af_packet.c`. My initial thought is to figure out if this is actually a problem.
First, I need to understand the synchronization context. Thread A, running a BPF program, calls `migrate_enable()`. This happens within a task context. Meanwhile, on the *same* CPU (CPU 0), an interrupt handler or softirq can trigger, and within that handler, `migrate_disable()` gets called. Crucially, *both* are accessing `current->migration_disabled`, which could be trouble, right? However, after careful consideration, I think this is benign.
The critical points are: everything is happening on the same CPU, CPU 0. The interrupt handler executes serially with respect to the task. It's strictly nested; any call to `migrate_disable()` in an interrupt is always matched by a `migrate_enable()` before returning to the task. The net modification to `p->migration_disabled` is therefore zero. Furthermore, the accesses are 16-bit word accesses (aligned), so they are atomic on x86, preventing torn reads/writes.
So, is this truly benign? Let's check the criteria. This seems to fall under the "**BENIGN (Truly Benign)**" category. It's a single-writer update (with concurrent read), the reads are a simple check. No "Fused Accesses" or "Ordering Violations" are relevant here. The value is safe (either the old or new value). I think.
This means the data race is *benign*. I need to recommend some annotation. I'll need to use `data_race()` on the accesses to `p->migration_disabled` in `__migrate_enable()` and `__migrate_disable()`. Alternatively, in the BPF code itself, it may be better to avoid calling `migrate_disable()`/`migrate_enable()` when in interrupt context (using `!in_task()`).
Let me make absolutely sure. The accesses are indeed on the same CPU. The softirq calls are strictly balanced and nested, with a net zero effect. The interrupted task resumes with the counter it expected. No memory safety violation, crash, or data corruption. Perfect.
Now, I'm ready to call `set-results`.
|
| 183/2 |
2026/11/18 01:57 |
tool |
set-results |
0m
Args:
Results: map[Benign:true FailureDetectableBy:]
|
| 184/2 |
2026/11/18 01:57 |
llm |
expert |
0m
Model:
Tokens:
input: 59443
output: 902
thoughts: 111
Thoughts: Okay, here's my understanding of the situation, tailored for my expert perspective:
**My Final Race Analysis and Recommendation**
Alright, `set-results` has been triggered. Time to solidify the findings and articulate them in a way that's both concise and informative, hitting all the crucial points. Here's how I'm thinking about it:
First, let's nail down the core of the problem with a clear **Race Summary:** *We're looking at a race condition between `[Function A]` and `[Function B]` operating on `[Variable/Field]`*. Straight to the point, no fluff.
Next, the **Synchronization context** is paramount. I need to explicitly state what locks are in play (or *not* in play, which is often more telling). I'll list the held locks and, more importantly, *explain the specific reasons why mutual exclusion isn't effectively preventing the race*. It's critical to demonstrate my understanding of the concurrency model and highlight the gap. Stating the context without explanation is not sufficient for a clear understanding.
Now for the definitive call: **Final Classification:** Is this a **BENIGN** race or a **HARMFUL** one? This is where the rubber meets the road. This decision hinges on my understanding of how the race impacts the data or execution.
Then, I'll need to provide solid **Reasoning**. I can't just declare a classification; I need to *justify* it. This is where I'll briefly explain *why* the race is structurally tolerant, focusing on why stale reads are acceptable, or why reentrancy isn't a problem, or why it may never be possible to occur. It's about demonstrating my deep understanding of the code's behavior under concurrency. It's the "how" and "why" of the conclusion.
Finally, and this is crucial for the overall lifecycle, I will prepare **Recommended Annotations**. What annotations are best to resolve this issue and why? If the race is benign, perhaps I'll suggest annotations that help document it and clarify its intent. If it's harmful, I'll recommend the *specific* annotations needed to remedy the issue, such as `synchronized`, `volatile`, or even more sophisticated concurrency constructs. This is where my expertise truly shines.
|
| 185/1 |
2026/11/18 01:57 |
action |
format-explanation |
0m
Results: map[Explanation:### Race Summary
- **Racing Functions:** `__migrate_enable()` (via `bpf_flow_dissect` /
`bpf_prog_run_pin_on_cpu`) vs. `__migrate_disable()` (via
`bpf_prog_run_clear_cb` / `run_filter` / `packet_rcv` in softirq context).
- **Target Variable:** `current->migration_disabled` (`unsigned short` in
`struct task_struct`).
---
### Synchronization Context
Both accesses occur on the same CPU (CPU 0). Task 5152 is executing in task
context within `bpf_flow_dissect` and invokes `migrate_enable()`, reading
`p->migration_disabled` without local interrupts disabled. A hardware interrupt
arrives on CPU 0, triggering softirq processing (`net_rx_action` -> `packet_rcv`
-> `run_filter`). The softirq invokes `bpf_prog_run_clear_cb` ->
`bpf_prog_run_pin_on_cpu` -> `migrate_disable()`, which modifies
`current->migration_disabled` of the interrupted task.
Mutual exclusion is absent because `__migrate_enable()` only acquires
`guard(preempt)()`, which disables preemption but leaves hardware interrupts
enabled.
---
### Final Classification: **BENIGN**
#### Reasoning
1. **Single-CPU Strict Serialization:** Both the interrupted task and the
interrupt/softirq handler run on the exact same CPU (CPU 0). An interrupt
suspends the task, executes to completion, and returns control to the task.
There is no cross-CPU concurrent access to `p->migration_disabled`.
2. **Strictly Balanced Reentrancy:** In the softirq handler, every invocation of
`migrate_disable()` inside BPF execution helpers (`bpf_prog_run_pin_on_cpu`,
`bpf_prog_run_save_cb`) is strictly paired with a corresponding
`migrate_enable()` before the softirq finishes.
- If the interrupted task had `p->migration_disabled == 1`, softirq
increments the counter to 2 and decrements it back to 1 before exiting.
- If the interrupted task had `p->migration_disabled == 0`, softirq
increments `rq->nr_pinned` and sets `migration_disabled = 1`, then restores
`migration_disabled = 0` and decrements `rq->nr_pinned` before returning.
- In all instruction interleavings, the net change to both
`p->migration_disabled` and `this_rq_pinned()` caused by the softirq
execution is zero.
3. **No Compiler/Ordering Hazards:** The variable is a naturally aligned 16-bit
integer (`unsigned short`), so loads and stores on x86-64 are atomic and cannot
tear. There are no loops or control flow paths that could be broken by compiler
hoisting or reloads.
4. **Historical Context:** Prior to commit `378b7708194f` ("sched: Make
migrate_{en,dis}able() inline"), `migrate_disable()` and `migrate_enable()` were
defined in `kernel/sched/core.c`, which is compiled with `KCSAN_SANITIZE := n`.
Inlining these functions into `include/linux/sched.h` exposed their memory
accesses to KCSAN instrumentation in external callers, triggering this harmless
interrupt reentrancy report.
---
### Recommended Annotations
To suppress the KCSAN report:
1. Mark the access to `p->migration_disabled` in `__migrate_enable()` and
`__migrate_disable()` with `data_race()`:
```c
if (data_race(p->migration_disabled) > 1) {
data_race(p->migration_disabled--);
return;
}
```
2. Alternatively, BPF filter helpers can omit
`migrate_disable()`/`migrate_enable()` when running in interrupt context
(`!in_task()`), since execution on the local CPU cannot migrate while running in
interrupt or bottom-half context.]
|