syzbot


possible deadlock in dpm_for_each_dev

Status: upstream: reported C repro on 2024/07/07 17:25
Subsystems: pm
[Documentation on labels]
Reported-by: syzbot+2a03726f1d4eff48b278@syzkaller.appspotmail.com
First crash: 691d, last: 143d
Cause bisection: failed (error log, bisect log)
  
Fix bisection: failed (error log, bisect log)
  
✨ AI Jobs (1)
ID Workflow Result Correct Bug Created Started Finished Revision Error
3ab44a4a-50c4-4220-8ffd-c8b68942ddc8 assessment-security 💥 possible deadlock in dpm_for_each_dev 2026/05/25 12:09 2026/05/25 12:09 2026/05/25 12:38 c69befb30ac10e158cc9d1557b508ee3f0eca1de failed to run ["make" "KERNELVERSION=syzkaller" "KERNELRELEASE=syzkaller" "LOCALVERSION=-syzkaller" "-j" "32" "ARCH=x86_64" "CC=ccache clang" "LD=ld.lld" "O=/app/workdir/cache/build/dd7b0b165947817100eea84b2e42d2ca873858c4" "-s" "bzImage" "compile_commands.json"]: exit status 2 Root cause: ld.lld: error: undefined symbol: wcslen * * Restart config... * * * General architecture-dependent options * Kprobes (KPROBES) [N/y/?] n Optimize very unlikely/likely branches (JUMP_LABEL) [Y/n/?] y Static key selftest (STATIC_KEYS_SELFTEST) [N/y/?] n Static call selftest (STATIC_CALL_SELFTEST) [N/y/?] n Enable seccomp to safely execute untrusted bytecode (SECCOMP) [Y/n/?] y Show seccomp filter cache status in /proc/pid/seccomp_cache (SECCOMP_CACHE_DEBUG) [N/y/?] n Link Time Optimization (LTO) > 1. None (LTO_NONE) choice[1]: 1 Use Clang's Control Flow Integrity (CFI) (CFI_CLANG) [N/y/?] (NEW) Error in reading or end of file. Number of bits to use for ASLR of mmap base address (ARCH_MMAP_RND_BITS) [28] 28 Number of bits to use for ASLR of mmap base address for compatible applications (ARCH_MMAP_RND_COMPAT_BITS) [8] 8 MMU page size > 1. 4KiB pages (PAGE_SIZE_4KB) choice[1]: 1 Provide system calls for 32-bit time_t (COMPAT_32BIT_TIME) [Y/n/?] y Use a virtually-mapped stack (VMAP_STACK) [Y/n/?] y Support for randomizing kernel stack offset on syscall entry (RANDOMIZE_KSTACK_OFFSET) [Y/n/?] y Default state of kernel stack offset randomization (RANDOMIZE_KSTACK_OFFSET_DEFAULT) [N/y/?] n Locking event counts collection (LOCK_EVENT_COUNTS) [N/y/?] n * * Memory initialization * Initialize kernel stack variables at function entry 1. no automatic stack variable initialization (weakest) (INIT_STACK_NONE) 2. pattern-init everything (strongest) (INIT_STACK_ALL_PATTERN) > 3. zero-init everything (strongest and safest) (INIT_STACK_ALL_ZERO) choice[1-3?]: 3 Enable heap memory zeroing on allocation by default (INIT_ON_ALLOC_DEFAULT_ON) [Y/n/?] y Enable heap memory zeroing on free by default (INIT_ON_FREE_DEFAULT_ON) [N/y/?] n Enable register zeroing on function exit (ZERO_CALL_USED_REGS) [N/y/?] (NEW) Error in reading or end of file. * * Kernel hardening options * Randomize layout of sensitive kernel structures > 1. Disable structure layout randomization (RANDSTRUCT_NONE) 2. Fully randomize structure layout (RANDSTRUCT_FULL) (NEW) choice[1-2?]: Error in reading or end of file. * * Compile-time checks and compiler options * Debug information 1. Disable debug information (DEBUG_INFO_NONE) 2. Rely on the toolchain's implicit default DWARF version (DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT) > 3. Generate DWARF Version 4 debuginfo (DEBUG_INFO_DWARF4) 4. Generate DWARF Version 5 debuginfo (DEBUG_INFO_DWARF5) choice[1-4?]: 3 Reduce debugging information (DEBUG_INFO_REDUCED) [N/y/?] n Compressed Debug information > 1. Don't compress debug information (DEBUG_INFO_COMPRESSED_NONE) 2. Compress debugging information with zlib (DEBUG_INFO_COMPRESSED_ZLIB) 3. Compress debugging information with zstd (DEBUG_INFO_COMPRESSED_ZSTD) (NEW) choice[1-3?]: Error in reading or end of file. Produce split debuginfo in .dwo files (DEBUG_INFO_SPLIT) [N/y/?] n Generate BTF type information (DEBUG_INFO_BTF) [N/y/?] n Provide GDB scripts for kernel debugging (GDB_SCRIPTS) [N/y/?] n Warn for stack frames larger than (FRAME_WARN) [2048] 2048 Strip assembler-generated symbols during link (STRIP_ASM_SYMS) [N/y/?] n Install uapi headers to usr/include (HEADERS_INSTALL) [N/y/?] n Make section mismatch errors non-fatal (SECTION_MISMATCH_WARN_ONLY) [Y/n/?] y Force all function address 64B aligned (DEBUG_FORCE_FUNCTION_ALIGN_64B) [N/y/?] n Generate vmlinux.map file when linking (VMLINUX_MAP) [N/y/?] n Force weak per-cpu definitions (DEBUG_FORCE_WEAK_PER_CPU) [N/y/?] n In file included from /app/workdir/cache/src/58bfcb47b1799155d5eea4364b0facc280c05473/net/socket.c:108: In file included from /app/workdir/cache/src/58bfcb47b1799155d5eea4364b0facc280c05473/include/net/busy_poll.h:18: /app/workdir/cache/src/58bfcb47b1799155d5eea4364b0facc280c05473/include/net/ip.h:472:14: w
Discussions (9)
Title Replies (including bot) Last reply
[syzbot] Monthly pm report (Oct 2025) 0 (1) 2025/10/07 07:53
[syzbot] Monthly pm report (Sep 2025) 0 (1) 2025/09/03 21:54
[syzbot] Monthly pm report (Jun 2025) 0 (1) 2025/06/17 07:53
[syzbot] Monthly pm report (Feb 2025) 0 (1) 2025/02/21 09:41
[syzbot] Monthly pm report (Jan 2025) 0 (1) 2025/01/17 15:00
[syzbot] Monthly pm report (Oct 2024) 0 (1) 2024/10/28 08:24
[syzbot] Monthly pm report (Sep 2024) 0 (1) 2024/09/26 12:43
[syzbot] Monthly pm report (Aug 2024) 0 (1) 2024/08/26 13:47
[syzbot] [pm?] [usb?] possible deadlock in dpm_for_each_dev 0 (1) 2024/07/07 17:25
Last patch testing requests (10)
Created Duration User Patch Repo Result
2026/05/16 19:25 25m retest repro upstream log
2026/04/11 23:31 58m retest repro upstream report log
2026/04/11 23:29 48m retest repro upstream report log
2026/03/07 17:50 17m retest repro upstream report log
2026/03/07 17:38 24m retest repro upstream report log
2025/12/27 14:19 25m retest repro upstream report log
2025/10/18 12:08 1h57m retest repro upstream report log
2025/10/10 10:48 29m retest repro upstream report log
2025/08/09 11:18 28m retest repro upstream report log
2025/05/10 15:36 24m retest repro upstream report log
Fix bisection attempts (14)
Created Duration User Patch Repo Result
2026/01/31 21:07 4m bisect fix upstream error job log
2026/01/01 15:50 3h53m bisect fix upstream OK (0) job log log
2025/12/02 10:41 5h09m bisect fix upstream OK (0) job log log
2025/11/01 00:28 4h55m bisect fix upstream OK (0) job log log
2025/06/16 18:47 3h21m bisect fix upstream OK (0) job log log
2025/05/17 05:43 1h54m bisect fix upstream OK (0) job log log
2025/04/16 11:40 2h29m bisect fix upstream OK (0) job log log
2025/03/17 09:26 2h13m bisect fix upstream OK (0) job log log
2025/02/15 03:04 3h14m bisect fix upstream OK (0) job log log
2025/01/15 23:17 2h38m bisect fix upstream OK (0) job log log
2024/12/05 03:19 2h23m bisect fix upstream OK (0) job log log
2024/10/21 13:43 3h25m bisect fix upstream OK (0) job log log
2024/09/20 19:11 1h51m bisect fix upstream OK (0) job log log
2024/08/19 04:48 3h12m bisect fix upstream OK (0) job log log

Sample crash report:
RBP: 00007fa6014b83d0 R08: 00007fa6013c3fa7 R09: 0000000000000038
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fa6013c4210
R13: 0000000000000001 R14: 00746f687370616e R15: 616e732f7665642f
 </TASK>
(NULL device *): loading /lib/firmware/dvb-usb-dib0700-1.20.fw failed with error -12
(NULL device *): Direct firmware load for dvb-usb-dib0700-1.20.fw failed with error -12
(NULL device *): Falling back to sysfs fallback for: dvb-usb-dib0700-1.20.fw
======================================================
WARNING: possible circular locking dependency detected
6.10.0-syzkaller-08280-g68b59730459e #0 Not tainted
------------------------------------------------------
syz-executor800/5113 is trying to acquire lock:
ffffffff8e1df830 (umhelper_sem){++++}-{3:3}, at: usermodehelper_read_trylock+0x140/0x300 kernel/umh.c:215

but task is already holding lock:
ffffffff8ec7df88 (dpm_list_mtx){+.+.}-{3:3}, at: device_pm_lock drivers/base/power/main.c:113 [inline]
ffffffff8ec7df88 (dpm_list_mtx){+.+.}-{3:3}, at: dpm_for_each_dev+0x2b/0xc0 drivers/base/power/main.c:1961

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (dpm_list_mtx){+.+.}-{3:3}:
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5753
       __mutex_lock_common kernel/locking/mutex.c:608 [inline]
       __mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
       device_pm_add+0x78/0x320 drivers/base/power/main.c:137
       device_add+0x5e8/0xbf0 drivers/base/core.c:3642
       fw_load_sysfs_fallback drivers/base/firmware_loader/fallback.c:86 [inline]
       fw_load_from_user_helper drivers/base/firmware_loader/fallback.c:162 [inline]
       firmware_fallback_sysfs+0x307/0x9e0 drivers/base/firmware_loader/fallback.c:238
       _request_firmware+0xcf5/0x12b0 drivers/base/firmware_loader/main.c:914
       request_firmware_work_func+0x12a/0x280 drivers/base/firmware_loader/main.c:1165
       process_one_work kernel/workqueue.c:3231 [inline]
       process_scheduled_works+0xa2c/0x1830 kernel/workqueue.c:3312
       worker_thread+0x86d/0xd40 kernel/workqueue.c:3390
       kthread+0x2f0/0x390 kernel/kthread.c:389
       ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
       ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244

-> #0 (umhelper_sem){++++}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3133 [inline]
       check_prevs_add kernel/locking/lockdep.c:3252 [inline]
       validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
       __lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5136
       lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5753
       down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
       usermodehelper_read_trylock+0x140/0x300 kernel/umh.c:215
       fw_load_from_user_helper drivers/base/firmware_loader/fallback.c:147 [inline]
       firmware_fallback_sysfs+0x184/0x9e0 drivers/base/firmware_loader/fallback.c:238
       _request_firmware+0xcf5/0x12b0 drivers/base/firmware_loader/main.c:914
       request_firmware drivers/base/firmware_loader/main.c:963 [inline]
       cache_firmware drivers/base/firmware_loader/main.c:1265 [inline]
       __async_dev_cache_fw_image+0xe7/0x320 drivers/base/firmware_loader/main.c:1379
       async_schedule_node_domain+0xdc/0x110 kernel/async.c:221
       async_schedule_domain include/linux/async.h:72 [inline]
       dev_cache_fw_image+0x36d/0x3e0 drivers/base/firmware_loader/main.c:1435
       dpm_for_each_dev+0x58/0xc0 drivers/base/power/main.c:1963
       device_cache_fw_images drivers/base/firmware_loader/main.c:1485 [inline]
       fw_pm_notify+0x24a/0x2f0 drivers/base/firmware_loader/main.c:1536
       notifier_call_chain+0x19f/0x3e0 kernel/notifier.c:93
       notifier_call_chain_robust kernel/notifier.c:128 [inline]
       blocking_notifier_call_chain_robust+0xe8/0x1e0 kernel/notifier.c:353
       pm_notifier_call_chain_robust+0x2c/0x60 kernel/power/main.c:102
       snapshot_open+0x1a1/0x280 kernel/power/user.c:77
       misc_open+0x313/0x390 drivers/char/misc.c:165
       chrdev_open+0x5b0/0x630 fs/char_dev.c:414
       do_dentry_open+0x970/0x1440 fs/open.c:959
       vfs_open+0x3e/0x330 fs/open.c:1089
       do_open fs/namei.c:3727 [inline]
       path_openat+0x2b3e/0x3470 fs/namei.c:3886
       do_filp_open+0x235/0x490 fs/namei.c:3913
       do_sys_openat2+0x13e/0x1d0 fs/open.c:1416
       do_sys_open fs/open.c:1431 [inline]
       __do_sys_openat fs/open.c:1447 [inline]
       __se_sys_openat fs/open.c:1442 [inline]
       __x64_sys_openat+0x247/0x2a0 fs/open.c:1442
       do_syscall_x64 arch/x86/entry/common.c:52 [inline]
       do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
       entry_SYSCALL_64_after_hwframe+0x77/0x7f

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(dpm_list_mtx);
                               lock(umhelper_sem);
                               lock(dpm_list_mtx);
  rlock(umhelper_sem);

 *** DEADLOCK ***

5 locks held by syz-executor800/5113:
 #0: ffffffff8eb397e8 (misc_mtx){+.+.}-{3:3}, at: misc_open+0x5c/0x390 drivers/char/misc.c:129
 #1: ffffffff8e1e9468 (system_transition_mutex){+.+.}-{3:3}, at: lock_system_sleep+0x60/0xa0 kernel/power/main.c:56
 #2: ffffffff8e209910 ((pm_chain_head).rwsem){++++}-{3:3}, at: blocking_notifier_call_chain_robust+0xac/0x1e0 kernel/notifier.c:352
 #3: ffffffff8ec82ee8 (fw_lock){+.+.}-{3:3}, at: device_cache_fw_images drivers/base/firmware_loader/main.c:1483 [inline]
 #3: ffffffff8ec82ee8 (fw_lock){+.+.}-{3:3}, at: fw_pm_notify+0x232/0x2f0 drivers/base/firmware_loader/main.c:1536
 #4: ffffffff8ec7df88 (dpm_list_mtx){+.+.}-{3:3}, at: device_pm_lock drivers/base/power/main.c:113 [inline]
 #4: ffffffff8ec7df88 (dpm_list_mtx){+.+.}-{3:3}, at: dpm_for_each_dev+0x2b/0xc0 drivers/base/power/main.c:1961

stack backtrace:
CPU: 0 PID: 5113 Comm: syz-executor800 Not tainted 6.10.0-syzkaller-08280-g68b59730459e #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 06/27/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x241/0x360 lib/dump_stack.c:114
 check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2186
 check_prev_add kernel/locking/lockdep.c:3133 [inline]
 check_prevs_add kernel/locking/lockdep.c:3252 [inline]
 validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
 __lock_acquire+0x1346/0x1fd0 kernel/locking/lockdep.c:5136
 lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5753
 down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
 usermodehelper_read_trylock+0x140/0x300 kernel/umh.c:215
 fw_load_from_user_helper drivers/base/firmware_loader/fallback.c:147 [inline]
 firmware_fallback_sysfs+0x184/0x9e0 drivers/base/firmware_loader/fallback.c:238
 _request_firmware+0xcf5/0x12b0 drivers/base/firmware_loader/main.c:914
 request_firmware drivers/base/firmware_loader/main.c:963 [inline]
 cache_firmware drivers/base/firmware_loader/main.c:1265 [inline]
 __async_dev_cache_fw_image+0xe7/0x320 drivers/base/firmware_loader/main.c:1379
 async_schedule_node_domain+0xdc/0x110 kernel/async.c:221
 async_schedule_domain include/linux/async.h:72 [inline]
 dev_cache_fw_image+0x36d/0x3e0 drivers/base/firmware_loader/main.c:1435
 dpm_for_each_dev+0x58/0xc0 drivers/base/power/main.c:1963
 device_cache_fw_images drivers/base/firmware_loader/main.c:1485 [inline]
 fw_pm_notify+0x24a/0x2f0 drivers/base/firmware_loader/main.c:1536
 notifier_call_chain+0x19f/0x3e0 kernel/notifier.c:93
 notifier_call_chain_robust kernel/notifier.c:128 [inline]
 blocking_notifier_call_chain_robust+0xe8/0x1e0 kernel/notifier.c:353
 pm_notifier_call_chain_robust+0x2c/0x60 kernel/power/main.c:102
 snapshot_open+0x1a1/0x280 kernel/power/user.c:77
 misc_open+0x313/0x390 drivers/char/misc.c:165
 chrdev_open+0x5b0/0x630 fs/char_dev.c:414
 do_dentry_open+0x970/0x1440 fs/open.c:959
 vfs_open+0x3e/0x330 fs/open.c:1089
 do_open fs/namei.c:3727 [inline]
 path_openat+0x2b3e/0x3470 fs/namei.c:3886
 do_filp_open+0x235/0x490 fs/namei.c:3913
 do_sys_openat2+0x13e/0x1d0 fs/open.c:1416
 do_sys_open fs/open.c:1431 [inline]
 __do_sys_openat fs/open.c:1447 [inline]
 __se_sys_openat fs/open.c:1442 [inline]
 __x64_sys_openat+0x247/0x2a0 fs/open.c:1442
 do_syscall_x64 arch/x86/entry/common.c:52 [inline]
 do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
 entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fa6014370c9
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 a1 1d 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fa6013c4208 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
RAX: ffffffffffffffda RBX: 00007fa6014b83d8 RCX: 00007fa6014370c9
RDX: 0000000000000000 RSI: 0000000020000080 RDI: ffffffffffffff9c
RBP: 00007fa6014b83d0 R08: 00007fa6013c3fa7 R09: 0000000000000038
R10: 0000000000000000 R11: 0000000000000246 R12: 00007fa6013c4210
R13: 0000000000000001 R14: 00746f687370616e R15: 616e732f7665642f
 </TASK>

Crashes (9):
Time Kernel Commit Syzkaller Config Log Report Syz repro C repro VM info Assets (help?) Manager Title
2024/07/19 10:43 upstream 68b59730459e ee4e11c8 .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce possible deadlock in dpm_for_each_dev
2024/07/03 17:17 upstream e9d22f7a6655 409d975c .config console log report syz / log C [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce possible deadlock in dpm_for_each_dev
2025/09/26 07:41 upstream 4ff71af020ae 0abd0691 .config console log report info [disk image] [vmlinux] [kernel image] ci-qemu-gce-upstream-auto possible deadlock in dpm_for_each_dev
2025/09/13 20:36 upstream 5cd64d4f9268 e2beed91 .config console log report info [disk image] [vmlinux] [kernel image] ci-qemu-gce-upstream-auto possible deadlock in dpm_for_each_dev
2025/08/31 01:40 upstream c8bc81a52d5a 807a3b61 .config console log report info [disk image] [vmlinux] [kernel image] ci-qemu-gce-upstream-auto possible deadlock in dpm_for_each_dev
2025/08/25 02:33 upstream 69fd6b99b8f8 bf27483f .config console log report info [disk image] [vmlinux] [kernel image] ci-qemu-gce-upstream-auto possible deadlock in dpm_for_each_dev
2025/08/24 12:02 upstream 8d245acc1e88 bf27483f .config console log report info [disk image] [vmlinux] [kernel image] ci-qemu-gce-upstream-auto possible deadlock in dpm_for_each_dev
2025/07/12 16:20 upstream 379f604cc3dc 3cda49cf .config console log report info [disk image] [vmlinux] [kernel image] ci-upstream-kasan-gce-smack-root possible deadlock in dpm_for_each_dev
2025/07/26 09:47 upstream 5f33ebd2018c fb8f743d .config console log report info [disk image (non-bootable)] [vmlinux] [kernel image] ci-qemu-upstream-386 possible deadlock in dpm_for_each_dev
* Struck through repros no longer work on HEAD.