BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 171s! BUG: workqueue lockup - pool cpus=0-1 flags=0x5 nice=0 stuck for 171s! Showing busy workqueues and worker pools: workqueue events: flags=0x0 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 pending: psi_avgs_work, free_work pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=7/256 refcnt=8 in-flight: 6384:bpf_map_free_deferred pending: vmstat_shepherd, free_work, drain_local_stock, destroy_list_workfn, kfree_rcu_monitor, rht_deferred_worker workqueue events_long: flags=0x0 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: br_fdb_cleanup pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 pending: br_multicast_gc_work, br_fdb_cleanup workqueue events_unbound: flags=0x2 pwq 4: cpus=0-1 flags=0x5 nice=0 active=1/512 refcnt=3 pending: toggle_allocation_gate workqueue events_power_efficient: flags=0x80 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 pending: gc_worker, check_lifetime pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: neigh_periodic_work, wg_ratelimiter_gc_entries, neigh_periodic_work workqueue rcu_gp: flags=0x8 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wait_rcu_exp_gp workqueue mm_percpu_wq: flags=0x8 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: vmstat_update workqueue netns: flags=0xe000a pwq 4: cpus=0-1 flags=0x5 nice=0 active=1/1 refcnt=3 in-flight: 20757:cleanup_net workqueue writeback: flags=0x4a pwq 4: cpus=0-1 flags=0x5 nice=0 active=1/256 refcnt=3 in-flight: 7:wb_workfn workqueue dm_bufio_cache: flags=0x8 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: work_fn workqueue ipv6_addrconf: flags=0x40008 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/1 refcnt=2 in-flight: 29731:addrconf_verify_work workqueue wg-kex-wg0: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=2/256 refcnt=4 pending: wg_packet_handshake_send_worker, wg_packet_handshake_send_worker workqueue wg-kex-wg1: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=2/256 refcnt=4 pending: wg_packet_handshake_send_worker, wg_packet_handshake_send_worker workqueue wg-kex-wg2: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=2/256 refcnt=4 pending: wg_packet_handshake_send_worker, wg_packet_handshake_send_worker workqueue wg-kex-wg0: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=4/256 refcnt=6 in-flight: 19054:wg_packet_handshake_send_worker pending: wg_queued_expired_zero_key_material, wg_queued_expired_zero_key_material, wg_packet_handshake_send_worker workqueue wg-kex-wg1: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=4/256 refcnt=6 pending: wg_packet_handshake_send_worker, wg_queued_expired_zero_key_material, wg_queued_expired_zero_key_material, wg_packet_handshake_send_worker workqueue wg-kex-wg2: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=2/256 refcnt=4 pending: wg_packet_handshake_send_worker, wg_packet_handshake_send_worker workqueue wg-kex-wg0: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=1/256 refcnt=3 in-flight: 22898:wg_packet_handshake_send_worker workqueue wg-crypt-wg0: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg1: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=3/256 refcnt=5 in-flight: 313:wg_packet_handshake_send_worker wg_packet_handshake_send_worker pending: wg_packet_handshake_send_worker workqueue wg-kex-wg2: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=2/256 refcnt=4 pending: wg_packet_handshake_send_worker, wg_packet_handshake_send_worker workqueue wg-kex-wg0: flags=0x24 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 in-flight: 29720:wg_packet_handshake_receive_worker wg_packet_handshake_receive_worker pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg0: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker workqueue wg-kex-wg1: flags=0x24 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-kex-wg1: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=1/256 refcnt=3 pending: wg_packet_handshake_send_worker workqueue wg-crypt-wg1: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: wg_packet_decrypt_worker, wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-crypt-wg2: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg0: flags=0x24 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_encrypt_worker pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker workqueue wg-kex-wg1: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=1/256 refcnt=3 pending: wg_packet_handshake_send_worker workqueue wg-crypt-wg1: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: wg_packet_decrypt_worker, wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg2: flags=0x24 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg2: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 pending: wg_packet_encrypt_worker, wg_packet_tx_worker workqueue wg-kex-wg0: flags=0x24 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 in-flight: 29750:wg_packet_handshake_receive_worker wg_packet_handshake_receive_worker pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-crypt-wg0: flags=0x28 pwq 2: cpus=1 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_encrypt_worker pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker workqueue wg-kex-wg1: flags=0x24 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-kex-wg1: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=2/256 refcnt=4 pending: wg_packet_handshake_send_worker, wg_packet_handshake_send_worker workqueue wg-crypt-wg1: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=3/256 refcnt=4 pending: wg_packet_tx_worker, wg_packet_encrypt_worker, wg_packet_decrypt_worker workqueue wg-kex-wg2: flags=0x24 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256 refcnt=2 pending: wg_packet_handshake_receive_worker workqueue wg-kex-wg2: flags=0x6 pwq 4: cpus=0-1 flags=0x5 nice=0 active=1/256 refcnt=3 pending: wg_packet_handshake_send_worker workqueue wg-crypt-wg2: flags=0x28 pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=2/256 refcnt=3 pending: wg_packet_tx_worker, wg_packet_encrypt_worker pool 0: cpus=0 node=0 flags=0x0 nice=0 hung=304s workers=3 idle: 6405 6340 pool 2: cpus=1 node=0 flags=0x0 nice=0 hung=172s workers=68 idle: 29769 29751 4277 4278 4270 29739 29763 29754 29762 29725 29770 4265 29701 29742 4290 29758 29774 29765 29761 740 1919 29764 4256 29734 29766 4266 4229 4281 29721 29706 29694 1925 29728 4259 29753 29715 29695 29755 29768 29713 29741 4280 4255 26597 29738 4251 29714 4284 460 4220 29759 29746 29767 4289 29703 29747 29718 29735 29737 29757 29745 29699 29772 4250 29696 pool 4: cpus=0-1 flags=0x5 nice=0 hung=172s workers=8 manager: 253 idle: 27749 314 rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: rcu: Tasks blocked on level-0 rcu_node (CPUs 0-1): P26/1:b..l (detected by 0, t=10003 jiffies, g=229901, q=62) task:khungtaskd state:R running task stack: 0 pid: 26 ppid: 2 flags:0x00004000 Call Trace: context_switch kernel/sched/core.c:4048 [inline] __schedule+0xc03/0x13e0 kernel/sched/core.c:4806 preempt_schedule_common+0x9b/0xf0 kernel/sched/core.c:4965 preempt_schedule+0xbc/0xd0 kernel/sched/core.c:4990 preempt_schedule_thunk+0x16/0x18 arch/x86/entry/thunk_64.S:39 vprintk_emit+0x32e/0x340 kernel/printk/printk.c:2070 vprintk_default+0x26/0x30 kernel/printk/printk.c:2086 vprintk_func+0x1a3/0x1e0 kernel/printk/printk_safe.c:401 printk+0xda/0x130 kernel/printk/printk.c:2117 check_hung_task kernel/hung_task.c:133 [inline] check_hung_uninterruptible_tasks kernel/hung_task.c:205 [inline] watchdog+0xa2c/0xf60 kernel/hung_task.c:300 kthread+0x324/0x3b0 kernel/kthread.c:313 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298 rcu: rcu_preempt kthread starved for 754 jiffies! g229901 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0 rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior. rcu: RCU grace-period kthread stack dump: task:rcu_preempt state:R running task stack: 0 pid: 13 ppid: 2 flags:0x00004000 Call Trace: context_switch kernel/sched/core.c:4048 [inline] __schedule+0xc03/0x13e0 kernel/sched/core.c:4806 schedule+0x13c/0x1d0 kernel/sched/core.c:4884 schedule_timeout+0x159/0x330 kernel/time/timer.c:1913 rcu_gp_fqs_loop kernel/rcu/tree.c:1943 [inline] rcu_gp_kthread+0x109a/0x2810 kernel/rcu/tree.c:2117 kthread+0x324/0x3b0 kernel/kthread.c:313 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:298