syzbot
possible deadlock in __generic_file_fsync
Status:
upstream: reported syz repro on 2019/06/19 20:20
Subsystems:
ext4
fat
[Documentation on labels]
Reported-by: syzbot+86818ef71401e07e80a1@syzkaller.appspotmail.com
First crash: 2609d, last: 1272d
▶
▼
Fix bisection attempts (6)
Sample crash report:
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
Bluetooth: hci5 command 0x0419 tx timeout
IPv6: ADDRCONF(NETDEV_CHANGE): batadv_slave_1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1_to_batadv: link becomes ready
======================================================
WARNING: possible circular locking dependency detected
4.14.290-syzkaller #0 Not tainted
------------------------------------------------------
kworker/1:1/24 is trying to acquire lock:
(&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff818f942e>] inode_lock include/linux/fs.h:719 [inline]
(&sb->s_type->i_mutex_key#10){+.+.}, at: [<ffffffff818f942e>] __generic_file_fsync+0x9e/0x190 fs/libfs.c:989
but task is already holding lock:
((&dio->complete_work)){+.+.}, at: [<ffffffff81364ee6>] process_one_work+0x6e6/0x14a0 kernel/workqueue.c:2092
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #2 ((&dio->complete_work)){+.+.}:
process_one_work+0x736/0x14a0 kernel/workqueue.c:2093
worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
kthread+0x30d/0x420 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
-> #1 ("dio/%s"sb->s_id){+.+.}:
flush_workqueue+0xfa/0x1310 kernel/workqueue.c:2625
drain_workqueue+0x177/0x3e0 kernel/workqueue.c:2790
destroy_workqueue+0x71/0x710 kernel/workqueue.c:4116
sb_init_dio_done_wq+0x61/0x80 fs/direct-io.c:635
do_blockdev_direct_IO fs/direct-io.c:1287 [inline]
__blockdev_direct_IO+0x3df1/0xdcb0 fs/direct-io.c:1423
ext4_direct_IO_write fs/ext4/inode.c:3749 [inline]
ext4_direct_IO+0x888/0x1b80 fs/ext4/inode.c:3890
generic_file_direct_write+0x1df/0x420 mm/filemap.c:2958
__generic_file_write_iter+0x2a2/0x590 mm/filemap.c:3137
ext4_file_write_iter+0x276/0xd20 fs/ext4/file.c:270
call_write_iter include/linux/fs.h:1780 [inline]
aio_write+0x2ed/0x560 fs/aio.c:1553
io_submit_one fs/aio.c:1641 [inline]
do_io_submit+0x847/0x1570 fs/aio.c:1709
do_syscall_64+0x1d5/0x640 arch/x86/entry/common.c:292
entry_SYSCALL_64_after_hwframe+0x46/0xbb
-> #0 (&sb->s_type->i_mutex_key#10){+.+.}:
lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3998
down_write+0x34/0x90 kernel/locking/rwsem.c:54
inode_lock include/linux/fs.h:719 [inline]
__generic_file_fsync+0x9e/0x190 fs/libfs.c:989
ext4_sync_file+0x8ed/0x12c0 fs/ext4/fsync.c:118
vfs_fsync_range+0x103/0x260 fs/sync.c:196
generic_write_sync include/linux/fs.h:2684 [inline]
dio_complete+0x561/0x8d0 fs/direct-io.c:330
process_one_work+0x793/0x14a0 kernel/workqueue.c:2117
worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
kthread+0x30d/0x420 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
other info that might help us debug this:
Chain exists of:
&sb->s_type->i_mutex_key#10 --> "dio/%s"sb->s_id --> (&dio->complete_work)
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock((&dio->complete_work));
lock("dio/%s"sb->s_id);
lock((&dio->complete_work));
lock(&sb->s_type->i_mutex_key#10);
*** DEADLOCK ***
2 locks held by kworker/1:1/24:
#0: ("dio/%s"sb->s_id){+.+.}, at: [<ffffffff81364eb0>] process_one_work+0x6b0/0x14a0 kernel/workqueue.c:2088
#1: ((&dio->complete_work)){+.+.}, at: [<ffffffff81364ee6>] process_one_work+0x6e6/0x14a0 kernel/workqueue.c:2092
stack backtrace:
CPU: 1 PID: 24 Comm: kworker/1:1 Not tainted 4.14.290-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 07/22/2022
Workqueue: dio/sda1 dio_aio_complete_work
Call Trace:
__dump_stack lib/dump_stack.c:17 [inline]
dump_stack+0x1b2/0x281 lib/dump_stack.c:58
print_circular_bug.constprop.0.cold+0x2d7/0x41e kernel/locking/lockdep.c:1258
check_prev_add kernel/locking/lockdep.c:1905 [inline]
check_prevs_add kernel/locking/lockdep.c:2022 [inline]
validate_chain kernel/locking/lockdep.c:2464 [inline]
__lock_acquire+0x2e0e/0x3f20 kernel/locking/lockdep.c:3491
lock_acquire+0x170/0x3f0 kernel/locking/lockdep.c:3998
down_write+0x34/0x90 kernel/locking/rwsem.c:54
inode_lock include/linux/fs.h:719 [inline]
__generic_file_fsync+0x9e/0x190 fs/libfs.c:989
ext4_sync_file+0x8ed/0x12c0 fs/ext4/fsync.c:118
vfs_fsync_range+0x103/0x260 fs/sync.c:196
generic_write_sync include/linux/fs.h:2684 [inline]
dio_complete+0x561/0x8d0 fs/direct-io.c:330
process_one_work+0x793/0x14a0 kernel/workqueue.c:2117
worker_thread+0x5cc/0xff0 kernel/workqueue.c:2251
kthread+0x30d/0x420 kernel/kthread.c:232
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:404
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop4): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
FAT-fs (loop3): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop2): Unrecognized mount option "./bus" or missing value
FAT-fs (loop0): Unrecognized mount option "./bus" or missing value
FAT-fs (loop5): Unrecognized mount option "./bus" or missing value
FAT-fs (loop1): Unrecognized mount option "./bus" or missing value
Struck through repros no longer work on HEAD.