BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 55s! Showing busy workqueues and worker pools: workqueue events: flags=0x100 pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3 in-flight: 9:netdev_rx_mode_work for 58s netdev_rx_mode_work pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=9 refcnt=10 in-flight: 856:nsim_fib_event_work for 58s nsim_fib_event_work ,24:request_firmware_work_func for 86s ,26:nsim_fib_event_work for 58s nsim_fib_event_work ,4756:nsim_fib_event_work for 58s nsim_fib_event_work ,1481:nsim_fib_event_work for 58s nsim_fib_event_work workqueue events_unbound: flags=0x2 pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2 in-flight: 3426:linkwatch_event for 57s workqueue ipv6_addrconf: flags=0x6000a pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=108 in-flight: 1048:addrconf_dad_work for 58s inactive: 104*addrconf_dad_work workqueue bat_events: flags=0x6000a pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=4 pending: batadv_mcast_mla_update workqueue udp_tunnel_nic: flags=0x20002 pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=5 in-flight: 40:udp_tunnel_nic_device_sync_work for 58s inactive: udp_tunnel_nic_device_sync_work workqueue wg-crypt-wg0: flags=0x128 pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2 pending: wg_packet_encrypt_worker workqueue wg-kex-wg1: flags=0x124 pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg1: flags=0x128 pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2 pending: wg_packet_encrypt_worker workqueue wg-kex-wg2: flags=0x124 pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg2: flags=0x128 pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3 pending: wg_packet_decrypt_worker, wg_packet_encrypt_worker pool 2: cpus=0 node=0 flags=0x0 nice=0 hung=0s workers=5 idle: 997 10 4770 4771 pool 6: cpus=1 node=0 flags=0x0 nice=0 hung=55s workers=9 idle: 4779 4792 4757 4772 pool 8: cpus=0-1 flags=0x4 nice=0 hung=0s workers=12 idle: 3126 39 14 1324 3874 3691 2925 12 3024 Showing backtraces of busy workers in stalled worker pools: pool 6: task:kworker/1:2 state:D stack:0 pid:856 tgid:856 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: events nsim_fib_event_work Call trace: __switch_to+0x2b0/0x6e0 arch/arm64/kernel/process.c:810 (T) context_switch kernel/sched/core.c:5387 [inline] __schedule+0x1b74/0x2d24 kernel/sched/core.c:7188 __schedule_loop kernel/sched/core.c:7267 [inline] schedule+0xa4/0x140 kernel/sched/core.c:7282 schedule_timeout+0x13c/0x294 kernel/time/sleep_timeout.c:99 schedule_timeout_uninterruptible+0x7c/0xbc kernel/time/sleep_timeout.c:158 msleep+0x3c/0x68 kernel/time/sleep_timeout.c:318 nsim_fib4_rt_add drivers/net/netdevsim/fib.c:369 [inline] nsim_fib4_rt_insert drivers/net/netdevsim/fib.c:432 [inline] nsim_fib4_event drivers/net/netdevsim/fib.c:464 [inline] nsim_fib_event+0x5070/0x5c5c drivers/net/netdevsim/fib.c:884 nsim_fib_event_work+0x1cc/0x30c drivers/net/netdevsim/fib.c:1493 process_one_work+0x78c/0x173c kernel/workqueue.c:3302 process_scheduled_works+0xdc/0x13c kernel/workqueue.c:3385 worker_thread+0x770/0xbd0 kernel/workqueue.c:3466 kthread+0x2f0/0x3c0 kernel/kthread.c:436 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:842 pool 6: task:kworker/1:0 state:D stack:0 pid:24 tgid:24 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: events request_firmware_work_func Call trace: __switch_to+0x2b0/0x6e0 arch/arm64/kernel/process.c:810 (T) context_switch kernel/sched/core.c:5387 [inline] __schedule+0x1b74/0x2d24 kernel/sched/core.c:7188 __schedule_loop kernel/sched/core.c:7267 [inline] schedule+0xa4/0x140 kernel/sched/core.c:7282 schedule_timeout+0x13c/0x294 kernel/time/sleep_timeout.c:99 do_wait_for_common+0x208/0x380 kernel/sched/completion.c:100 __wait_for_common kernel/sched/completion.c:121 [inline] wait_for_common+0x50/0x74 kernel/sched/completion.c:132 wait_for_completion_killable_timeout+0x14/0x20 kernel/sched/completion.c:293 __fw_state_wait_common drivers/base/firmware_loader/firmware.h:104 [inline] fw_sysfs_wait_timeout drivers/base/firmware_loader/fallback.c:44 [inline] fw_load_sysfs_fallback drivers/base/firmware_loader/fallback.c:110 [inline] fw_load_from_user_helper drivers/base/firmware_loader/fallback.c:162 [inline] firmware_fallback_sysfs+0x4d4/0x910 drivers/base/firmware_loader/fallback.c:238 _request_firmware+0xb04/0xf00 drivers/base/firmware_loader/main.c:898 request_firmware_work_func+0xa8/0x19c drivers/base/firmware_loader/main.c:1150 process_one_work+0x78c/0x173c kernel/workqueue.c:3302 process_scheduled_works+0xdc/0x13c kernel/workqueue.c:3385 worker_thread+0x770/0xbd0 kernel/workqueue.c:3466 kthread+0x2f0/0x3c0 kernel/kthread.c:436 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:842 pool 6: task:kworker/1:1 state:D stack:0 pid:26 tgid:26 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: events nsim_fib_event_work Call trace: __switch_to+0x2b0/0x6e0 arch/arm64/kernel/process.c:810 (T) context_switch kernel/sched/core.c:5387 [inline] __schedule+0x1b74/0x2d24 kernel/sched/core.c:7188 __schedule_loop kernel/sched/core.c:7267 [inline] schedule+0xa4/0x140 kernel/sched/core.c:7282 schedule_timeout+0x13c/0x294 kernel/time/sleep_timeout.c:99 schedule_timeout_uninterruptible+0x7c/0xbc kernel/time/sleep_timeout.c:158 msleep+0x3c/0x68 kernel/time/sleep_timeout.c:318 nsim_fib4_rt_add drivers/net/netdevsim/fib.c:369 [inline] nsim_fib4_rt_insert drivers/net/netdevsim/fib.c:432 [inline] nsim_fib4_event drivers/net/netdevsim/fib.c:464 [inline] nsim_fib_event+0x5070/0x5c5c drivers/net/netdevsim/fib.c:884 nsim_fib_event_work+0x1cc/0x30c drivers/net/netdevsim/fib.c:1493 process_one_work+0x78c/0x173c kernel/workqueue.c:3302 process_scheduled_works+0xdc/0x13c kernel/workqueue.c:3385 worker_thread+0x770/0xbd0 kernel/workqueue.c:3466 kthread+0x2f0/0x3c0 kernel/kthread.c:436 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:842 pool 6: task:kworker/1:4 state:D stack:0 pid:4756 tgid:4756 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: events nsim_fib_event_work Call trace: __switch_to+0x2b0/0x6e0 arch/arm64/kernel/process.c:810 (T) context_switch kernel/sched/core.c:5387 [inline] __schedule+0x1b74/0x2d24 kernel/sched/core.c:7188 __schedule_loop kernel/sched/core.c:7267 [inline] schedule+0xa4/0x140 kernel/sched/core.c:7282 schedule_timeout+0x13c/0x294 kernel/time/sleep_timeout.c:99 schedule_timeout_uninterruptible+0x7c/0xbc kernel/time/sleep_timeout.c:158 msleep+0x3c/0x68 kernel/time/sleep_timeout.c:318 nsim_fib6_rt_add drivers/net/netdevsim/fib.c:693 [inline] nsim_fib6_rt_insert drivers/net/netdevsim/fib.c:759 [inline] nsim_fib6_event drivers/net/netdevsim/fib.c:856 [inline] nsim_fib_event+0x5270/0x5c5c drivers/net/netdevsim/fib.c:889 nsim_fib_event_work+0x1cc/0x30c drivers/net/netdevsim/fib.c:1493 process_one_work+0x78c/0x173c kernel/workqueue.c:3302 process_scheduled_works+0xdc/0x13c kernel/workqueue.c:3385 worker_thread+0x770/0xbd0 kernel/workqueue.c:3466 kthread+0x2f0/0x3c0 kernel/kthread.c:436 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:842 pool 6: task:kworker/1:3 state:D stack:0 pid:1481 tgid:1481 ppid:2 task_flags:0x4208060 flags:0x00000010 Workqueue: events nsim_fib_event_work Call trace: __switch_to+0x2b0/0x6e0 arch/arm64/kernel/process.c:810 (T) context_switch kernel/sched/core.c:5387 [inline] __schedule+0x1b74/0x2d24 kernel/sched/core.c:7188 __schedule_loop kernel/sched/core.c:7267 [inline] schedule+0xa4/0x140 kernel/sched/core.c:7282 schedule_timeout+0x13c/0x294 kernel/time/sleep_timeout.c:99 schedule_timeout_uninterruptible+0x7c/0xbc kernel/time/sleep_timeout.c:158 msleep+0x3c/0x68 kernel/time/sleep_timeout.c:318 nsim_fib6_rt_add drivers/net/netdevsim/fib.c:693 [inline] nsim_fib6_rt_insert drivers/net/netdevsim/fib.c:759 [inline] nsim_fib6_event drivers/net/netdevsim/fib.c:856 [inline] nsim_fib_event+0x5270/0x5c5c drivers/net/netdevsim/fib.c:889 nsim_fib_event_work+0x1cc/0x30c drivers/net/netdevsim/fib.c:1493 process_one_work+0x78c/0x173c kernel/workqueue.c:3302 process_scheduled_works+0xdc/0x13c kernel/workqueue.c:3385 worker_thread+0x770/0xbd0 kernel/workqueue.c:3466 kthread+0x2f0/0x3c0 kernel/kthread.c:436 ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:842