====================================================== WARNING: possible circular locking dependency detected syzkaller #0 Not tainted ------------------------------------------------------ kworker/u8:13/6181 is trying to acquire lock: ffffffff8ea5f200 (pcpu_alloc_mutex){+.+.}-{4:4}, at: pcpu_alloc_noprof+0x231/0x1900 mm/percpu.c:1788 but task is already holding lock: ffffffff8e7fa9c0 (wq_pool_mutex){+.+.}-{4:4}, at: apply_wqattrs_lock kernel/workqueue.c:5317 [inline] ffffffff8e7fa9c0 (wq_pool_mutex){+.+.}-{4:4}, at: __alloc_workqueue+0x9df/0x1f70 kernel/workqueue.c:5880 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #9 (wq_pool_mutex){+.+.}-{4:4}: __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x19d/0x1590 kernel/locking/mutex.c:820 apply_wqattrs_lock kernel/workqueue.c:5317 [inline] __alloc_workqueue+0x9df/0x1f70 kernel/workqueue.c:5880 alloc_workqueue_va kernel/workqueue.c:5931 [inline] alloc_workqueue_noprof+0xe3/0x210 kernel/workqueue.c:5947 padata_alloc+0xbe/0x360 kernel/padata.c:964 pcrypt_init_padata+0x27/0x100 crypto/pcrypt.c:331 pcrypt_init+0x60/0xc0 crypto/pcrypt.c:356 do_one_initcall+0x250/0x870 init/main.c:1392 do_initcall_level+0x10a/0x1a0 init/main.c:1454 do_initcalls+0x59/0xa0 init/main.c:1470 kernel_init_freeable+0x29d/0x3e0 init/main.c:1703 kernel_init+0x1d/0x1d0 init/main.c:1593 ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #8 (cpu_hotplug_lock){++++}-{0:0}: percpu_down_read_internal include/linux/percpu-rwsem.h:53 [inline] percpu_down_read include/linux/percpu-rwsem.h:77 [inline] cpus_read_lock+0x42/0x160 kernel/cpu.c:490 static_key_slow_inc+0x12/0x30 kernel/jump_label.c:190 udp_tunnel_encap_enable+0x86/0xb0 include/net/udp_tunnel.h:232 udp_lib_setsockopt+0x5b6/0x6b0 net/ipv4/udp.c:2945 udpv6_setsockopt+0x67/0xb0 net/ipv6/udp.c:1820 do_sock_setsockopt+0x17c/0x1b0 net/socket.c:2382 __sys_setsockopt net/socket.c:2407 [inline] __do_sys_setsockopt net/socket.c:2413 [inline] __se_sys_setsockopt net/socket.c:2410 [inline] __x64_sys_setsockopt+0x13d/0x1b0 net/socket.c:2410 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f -> #7 (sk_lock-AF_INET6){+.+.}-{0:0}: lock_sock_nested+0x41/0x100 net/core/sock.c:3792 lock_sock include/net/sock.h:1713 [inline] inet_shutdown+0x6a/0x390 net/ipv4/af_inet.c:915 nbd_mark_nsock_dead+0x2cb/0x550 drivers/block/nbd.c:318 recv_work+0x1cef/0x1e10 drivers/block/nbd.c:1021 process_one_work kernel/workqueue.c:3314 [inline] process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3397 worker_thread+0xa47/0xfb0 kernel/workqueue.c:3478 kthread+0x389/0x470 kernel/kthread.c:436 ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #6 (&nsock->tx_lock){+.+.}-{4:4}: __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x19d/0x1590 kernel/locking/mutex.c:820 nbd_handle_cmd drivers/block/nbd.c:1143 [inline] nbd_queue_rq+0x373/0x1150 drivers/block/nbd.c:1207 blk_mq_dispatch_rq_list+0x499/0x1990 block/blk-mq.c:2148 __blk_mq_do_dispatch_sched block/blk-mq-sched.c:168 [inline] blk_mq_do_dispatch_sched block/blk-mq-sched.c:182 [inline] __blk_mq_sched_dispatch_requests+0xd36/0x1580 block/blk-mq-sched.c:307 blk_mq_sched_dispatch_requests+0xd7/0x190 block/blk-mq-sched.c:329 blk_mq_run_hw_queue+0x348/0x4f0 block/blk-mq.c:2387 blk_mq_dispatch_list+0xd11/0xe10 include/linux/spinlock.h:-1 blk_mq_flush_plug_list+0x45f/0x540 block/blk-mq.c:2997 __blk_flush_plug+0x3ed/0x4d0 block/blk-core.c:1230 blk_finish_plug block/blk-core.c:1257 [inline] __submit_bio+0x465/0x560 block/blk-core.c:649 __submit_bio_noacct_mq block/blk-core.c:722 [inline] submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:753 submit_bh fs/buffer.c:2742 [inline] block_read_full_folio+0x599/0x830 fs/buffer.c:2344 filemap_read_folio+0x12c/0x3a0 mm/filemap.c:2502 do_read_cache_folio+0x354/0x590 mm/filemap.c:4107 read_mapping_folio include/linux/pagemap.h:1017 [inline] read_part_sector+0xb6/0x2b0 block/partitions/core.c:724 adfspart_check_ICS+0xb1/0x960 block/partitions/acorn.c:356 check_partition block/partitions/core.c:143 [inline] blk_add_partitions block/partitions/core.c:591 [inline] bdev_disk_changed+0x82b/0x1780 block/partitions/core.c:695 blkdev_get_whole+0x372/0x510 block/bdev.c:756 bdev_open+0x324/0xd70 block/bdev.c:965 blkdev_open+0x461/0x600 block/fops.c:697 do_dentry_open+0x816/0x1380 fs/open.c:947 vfs_open+0x3b/0x340 fs/open.c:1079 do_open fs/namei.c:4699 [inline] path_openat+0x2e3b/0x3890 fs/namei.c:4858 do_file_open+0x23e/0x4a0 fs/namei.c:4887 do_sys_openat2+0x113/0x200 fs/open.c:1364 do_sys_open fs/open.c:1370 [inline] __do_sys_openat fs/open.c:1386 [inline] __se_sys_openat fs/open.c:1381 [inline] __x64_sys_openat+0x138/0x170 fs/open.c:1381 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f -> #5 (&cmd->lock){+.+.}-{4:4}: __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x19d/0x1590 kernel/locking/mutex.c:820 nbd_queue_rq+0xc1/0x1150 drivers/block/nbd.c:1199 blk_mq_dispatch_rq_list+0x499/0x1990 block/blk-mq.c:2148 __blk_mq_do_dispatch_sched block/blk-mq-sched.c:168 [inline] blk_mq_do_dispatch_sched block/blk-mq-sched.c:182 [inline] __blk_mq_sched_dispatch_requests+0xd36/0x1580 block/blk-mq-sched.c:307 blk_mq_sched_dispatch_requests+0xd7/0x190 block/blk-mq-sched.c:329 blk_mq_run_hw_queue+0x348/0x4f0 block/blk-mq.c:2387 blk_mq_dispatch_list+0xd11/0xe10 include/linux/spinlock.h:-1 blk_mq_flush_plug_list+0x45f/0x540 block/blk-mq.c:2997 __blk_flush_plug+0x3ed/0x4d0 block/blk-core.c:1230 blk_finish_plug block/blk-core.c:1257 [inline] __submit_bio+0x465/0x560 block/blk-core.c:649 __submit_bio_noacct_mq block/blk-core.c:722 [inline] submit_bio_noacct_nocheck+0x2f4/0xa40 block/blk-core.c:753 submit_bh fs/buffer.c:2742 [inline] block_read_full_folio+0x599/0x830 fs/buffer.c:2344 filemap_read_folio+0x12c/0x3a0 mm/filemap.c:2502 do_read_cache_folio+0x354/0x590 mm/filemap.c:4107 read_mapping_folio include/linux/pagemap.h:1017 [inline] read_part_sector+0xb6/0x2b0 block/partitions/core.c:724 adfspart_check_ICS+0xb1/0x960 block/partitions/acorn.c:356 check_partition block/partitions/core.c:143 [inline] blk_add_partitions block/partitions/core.c:591 [inline] bdev_disk_changed+0x82b/0x1780 block/partitions/core.c:695 blkdev_get_whole+0x372/0x510 block/bdev.c:756 bdev_open+0x324/0xd70 block/bdev.c:965 blkdev_open+0x461/0x600 block/fops.c:697 do_dentry_open+0x816/0x1380 fs/open.c:947 vfs_open+0x3b/0x340 fs/open.c:1079 do_open fs/namei.c:4699 [inline] path_openat+0x2e3b/0x3890 fs/namei.c:4858 do_file_open+0x23e/0x4a0 fs/namei.c:4887 do_sys_openat2+0x113/0x200 fs/open.c:1364 do_sys_open fs/open.c:1370 [inline] __do_sys_openat fs/open.c:1386 [inline] __se_sys_openat fs/open.c:1381 [inline] __x64_sys_openat+0x138/0x170 fs/open.c:1381 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f -> #4 (set->srcu){.+.+}-{0:0}: srcu_lock_sync include/linux/srcu.h:199 [inline] __synchronize_srcu+0xc9/0x2f0 kernel/rcu/srcutree.c:1481 elevator_switch+0x1e8/0x7b0 block/elevator.c:576 elevator_change+0x2fa/0x480 block/elevator.c:681 elevator_set_default+0x375/0x440 block/elevator.c:754 blk_register_queue+0x3f3/0x4e0 block/blk-sysfs.c:987 __add_disk+0x6cb/0xe30 block/genhd.c:528 add_disk_fwnode+0xfb/0x4b0 block/genhd.c:597 add_disk include/linux/blkdev.h:794 [inline] nbd_dev_add+0x733/0xb60 drivers/block/nbd.c:1984 nbd_init+0x15f/0x1e0 drivers/block/nbd.c:2692 do_one_initcall+0x250/0x870 init/main.c:1392 do_initcall_level+0x10a/0x1a0 init/main.c:1454 do_initcalls+0x59/0xa0 init/main.c:1470 kernel_init_freeable+0x29d/0x3e0 init/main.c:1703 kernel_init+0x1d/0x1d0 init/main.c:1593 ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #3 (&q->elevator_lock){+.+.}-{4:4}: __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x19d/0x1590 kernel/locking/mutex.c:820 elevator_change+0x1af/0x480 block/elevator.c:679 elevator_set_none+0xb5/0x140 block/elevator.c:769 blk_mq_elv_switch_none block/blk-mq.c:5134 [inline] __blk_mq_update_nr_hw_queues block/blk-mq.c:5179 [inline] blk_mq_update_nr_hw_queues+0x5ef/0x19f0 block/blk-mq.c:5244 nbd_start_device+0x189/0xb30 drivers/block/nbd.c:1489 nbd_genl_connect+0x1597/0x1c10 drivers/block/nbd.c:2239 genl_family_rcv_msg_doit+0x233/0x340 net/netlink/genetlink.c:1114 genl_family_rcv_msg net/netlink/genetlink.c:1194 [inline] genl_rcv_msg+0x614/0x7a0 net/netlink/genetlink.c:1209 netlink_rcv_skb+0x226/0x4a0 net/netlink/af_netlink.c:2555 genl_rcv+0x28/0x40 net/netlink/genetlink.c:1218 netlink_unicast_kernel net/netlink/af_netlink.c:1318 [inline] netlink_unicast+0x7bb/0x940 net/netlink/af_netlink.c:1344 netlink_sendmsg+0x813/0xb40 net/netlink/af_netlink.c:1899 sock_sendmsg_nosec net/socket.c:787 [inline] __sock_sendmsg net/socket.c:802 [inline] ____sys_sendmsg+0x9b9/0xa20 net/socket.c:2699 ___sys_sendmsg+0x2a5/0x360 net/socket.c:2753 __sys_sendmsg net/socket.c:2785 [inline] __do_sys_sendmsg net/socket.c:2790 [inline] __se_sys_sendmsg net/socket.c:2788 [inline] __x64_sys_sendmsg+0x1bd/0x2a0 net/socket.c:2788 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f -> #2 (&q->q_usage_counter(io)#50){++++}-{0:0}: blk_alloc_queue+0x544/0x690 block/blk-core.c:461 blk_mq_alloc_queue block/blk-mq.c:4453 [inline] __blk_mq_alloc_disk+0x194/0x390 block/blk-mq.c:4500 nbd_dev_add+0x494/0xb60 drivers/block/nbd.c:1954 nbd_init+0x15f/0x1e0 drivers/block/nbd.c:2692 do_one_initcall+0x250/0x870 init/main.c:1392 do_initcall_level+0x10a/0x1a0 init/main.c:1454 do_initcalls+0x59/0xa0 init/main.c:1470 kernel_init_freeable+0x29d/0x3e0 init/main.c:1703 kernel_init+0x1d/0x1d0 init/main.c:1593 ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 -> #1 (fs_reclaim){+.+.}-{0:0}: __fs_reclaim_acquire mm/page_alloc.c:4320 [inline] fs_reclaim_acquire+0x71/0x100 mm/page_alloc.c:4334 might_alloc include/linux/sched/mm.h:317 [inline] prepare_alloc_pages+0x15b/0x650 mm/page_alloc.c:4990 __alloc_frozen_pages_noprof+0x12f/0x380 mm/page_alloc.c:5210 __alloc_pages_noprof+0x10/0x100 mm/page_alloc.c:5255 __alloc_pages_node_noprof include/linux/gfp.h:289 [inline] alloc_pages_node_noprof include/linux/gfp.h:316 [inline] pcpu_alloc_pages mm/percpu-vm.c:95 [inline] pcpu_populate_chunk+0x181/0xb30 mm/percpu-vm.c:285 pcpu_alloc_noprof+0xbca/0x1900 mm/percpu.c:1876 xt_percpu_counter_alloc+0x145/0x1e0 net/netfilter/x_tables.c:2137 find_check_entry net/ipv4/netfilter/ip_tables.c:526 [inline] translate_table+0x129a/0x1f80 net/ipv4/netfilter/ip_tables.c:716 ipt_register_table+0x112/0x310 net/ipv4/netfilter/ip_tables.c:1736 iptable_mangle_table_init+0x41/0x70 net/ipv4/netfilter/iptable_mangle.c:92 xt_find_table_lock+0x311/0x400 net/netfilter/x_tables.c:1353 xt_request_find_table_lock+0x26/0x100 net/netfilter/x_tables.c:1378 get_info net/ipv4/netfilter/ip_tables.c:961 [inline] do_ipt_get_ctl+0x719/0x1230 net/ipv4/netfilter/ip_tables.c:1656 nf_getsockopt+0x26e/0x290 net/netfilter/nf_sockopt.c:116 ip_getsockopt+0x19e/0x230 net/ipv4/ip_sockglue.c:1777 do_sock_getsockopt+0x51d/0x7e0 net/socket.c:2488 __sys_getsockopt net/socket.c:2519 [inline] __do_sys_getsockopt net/socket.c:2526 [inline] __se_sys_getsockopt net/socket.c:2523 [inline] __x64_sys_getsockopt+0x1a4/0x240 net/socket.c:2523 do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline] do_syscall_64+0x174/0x580 arch/x86/entry/syscall_64.c:94 entry_SYSCALL_64_after_hwframe+0x77/0x7f -> #0 (pcpu_alloc_mutex){+.+.}-{4:4}: check_prev_add kernel/locking/lockdep.c:3165 [inline] check_prevs_add kernel/locking/lockdep.c:3284 [inline] validate_chain kernel/locking/lockdep.c:3908 [inline] __lock_acquire+0x1520/0x2cd0 kernel/locking/lockdep.c:5237 lock_acquire+0x106/0x350 kernel/locking/lockdep.c:5868 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x19d/0x1590 kernel/locking/mutex.c:820 pcpu_alloc_noprof+0x231/0x1900 mm/percpu.c:1788 alloc_and_link_pwqs kernel/workqueue.c:5611 [inline] __alloc_workqueue+0xa6f/0x1f70 kernel/workqueue.c:5882 alloc_workqueue_va kernel/workqueue.c:5931 [inline] alloc_workqueue_noprof+0xe3/0x210 kernel/workqueue.c:5947 __rds_conn_create+0x130d/0x2250 net/rds/connection.c:274 rds_conn_create+0x43/0x60 net/rds/connection.c:367 rds_tcp_accept_one+0x44f/0xd40 net/rds/tcp_listen.c:226 rds_tcp_accept_worker+0x1d/0x70 net/rds/tcp.c:533 process_one_work kernel/workqueue.c:3314 [inline] process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3397 worker_thread+0xa47/0xfb0 kernel/workqueue.c:3478 kthread+0x389/0x470 kernel/kthread.c:436 ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245 other info that might help us debug this: Chain exists of: pcpu_alloc_mutex --> cpu_hotplug_lock --> wq_pool_mutex Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(wq_pool_mutex); lock(cpu_hotplug_lock); lock(wq_pool_mutex); lock(pcpu_alloc_mutex); *** DEADLOCK *** 4 locks held by kworker/u8:13/6181: #0: ffff8880327fe140 ((wq_completion)krdsd){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3289 [inline] #0: ffff8880327fe140 ((wq_completion)krdsd){+.+.}-{0:0}, at: process_scheduled_works+0xa20/0x14e0 kernel/workqueue.c:3397 #1: ffffc900055b7c40 ((work_completion)(&rtn->rds_tcp_accept_w)){+.+.}-{0:0}, at: process_one_work kernel/workqueue.c:3289 [inline] #1: ffffc900055b7c40 ((work_completion)(&rtn->rds_tcp_accept_w)){+.+.}-{0:0}, at: process_scheduled_works+0xa20/0x14e0 kernel/workqueue.c:3397 #2: ffff888031557c60 (&rtn->rds_tcp_accept_lock){+.+.}-{4:4}, at: rds_tcp_accept_one+0xb3/0xd40 net/rds/tcp_listen.c:172 #3: ffffffff8e7fa9c0 (wq_pool_mutex){+.+.}-{4:4}, at: apply_wqattrs_lock kernel/workqueue.c:5317 [inline] #3: ffffffff8e7fa9c0 (wq_pool_mutex){+.+.}-{4:4}, at: __alloc_workqueue+0x9df/0x1f70 kernel/workqueue.c:5880 stack backtrace: CPU: 0 UID: 0 PID: 6181 Comm: kworker/u8:13 Not tainted syzkaller #0 PREEMPT(full) Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 05/09/2026 Workqueue: krdsd rds_tcp_accept_worker Call Trace: dump_stack_lvl+0xe8/0x150 lib/dump_stack.c:120 print_circular_bug+0x2e1/0x300 kernel/locking/lockdep.c:2043 check_noncircular+0x12e/0x150 kernel/locking/lockdep.c:2175 check_prev_add kernel/locking/lockdep.c:3165 [inline] check_prevs_add kernel/locking/lockdep.c:3284 [inline] validate_chain kernel/locking/lockdep.c:3908 [inline] __lock_acquire+0x1520/0x2cd0 kernel/locking/lockdep.c:5237 lock_acquire+0x106/0x350 kernel/locking/lockdep.c:5868 __mutex_lock_common kernel/locking/mutex.c:646 [inline] __mutex_lock+0x19d/0x1590 kernel/locking/mutex.c:820 pcpu_alloc_noprof+0x231/0x1900 mm/percpu.c:1788 alloc_and_link_pwqs kernel/workqueue.c:5611 [inline] __alloc_workqueue+0xa6f/0x1f70 kernel/workqueue.c:5882 alloc_workqueue_va kernel/workqueue.c:5931 [inline] alloc_workqueue_noprof+0xe3/0x210 kernel/workqueue.c:5947 __rds_conn_create+0x130d/0x2250 net/rds/connection.c:274 rds_conn_create+0x43/0x60 net/rds/connection.c:367 rds_tcp_accept_one+0x44f/0xd40 net/rds/tcp_listen.c:226 rds_tcp_accept_worker+0x1d/0x70 net/rds/tcp.c:533 process_one_work kernel/workqueue.c:3314 [inline] process_scheduled_works+0xa8e/0x14e0 kernel/workqueue.c:3397 worker_thread+0xa47/0xfb0 kernel/workqueue.c:3478 kthread+0x389/0x470 kernel/kthread.c:436 ret_from_fork+0x514/0xb70 arch/x86/kernel/process.c:158 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245