BUG: workqueue lockup - pool cpus=0 node=0 flags=0x0 nice=0 stuck for 179s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x100
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=16 refcnt=17
    pending: 2*nsim_dev_hwstats_traffic_work, 2*psi_avgs_work, vmstat_shepherd, 2*psi_avgs_work, 3*ovs_dp_masks_rebalance, xfrm_state_gc_task, free_obj_work, delayed_vfree_work, ima_keys_handler, switchdev_deferred_process_work, rht_deferred_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=14 refcnt=15
    in-flight: 796:ovs_dp_masks_rebalance for 10s
    pending: 2*ovs_dp_masks_rebalance, delayed_vfree_work, 3*rht_deferred_worker, 5*nsim_dev_hwstats_traffic_work, 2*psi_avgs_work
workqueue events_long: flags=0x100
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=4 refcnt=5
    pending: 2*defense_work_handler, br_multicast_gc_work, br_fdb_cleanup
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=10 refcnt=11
    pending: 2*br_multicast_gc_work, 2*br_fdb_cleanup, br_multicast_gc_work, 5*defense_work_handler
workqueue events_unbound: flags=0x2
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    in-flight: 61:call_usermodehelper_exec_work for 181s
workqueue events_unbound: flags=0x2
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=5 refcnt=6
    in-flight: 8235:idle_cull_fn for 19s ,8246:cfg80211_wiphy_work for 173s cfg80211_wiphy_work ,8245:cfg80211_wiphy_work for 173s cfg80211_wiphy_work
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=12 refcnt=13
    in-flight: 8238:nsim_dev_trap_report_work for 2s ,8249:nsim_dev_trap_report_work for 4s ,7828:nsim_dev_trap_report_work for 9s ,8233:nsim_dev_trap_report_work for 95s ,8244:nsim_dev_trap_report_work for 80s ,8250:cfg80211_wiphy_work for 180s cfg80211_wiphy_work ,8251:cfg80211_wiphy_work for 174s cfg80211_wiphy_work ,7826:toggle_allocation_gate for 79s
    pending: 2*nsim_dev_trap_report_work
workqueue events_freezable: flags=0x104
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: update_balloon_stats_func
workqueue events_power_efficient: flags=0x180
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=5 refcnt=6
    pending: gc_worker, reg_check_chans_work, 3*check_lifetime
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=6 refcnt=7
    in-flight: 24:neigh_periodic_work for 191s ,5927:wg_ratelimiter_gc_entries for 102s
    pending: check_lifetime, do_cache_clean, 2*neigh_managed_work
workqueue rcu_gp: flags=0x108
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
    in-flight: 5875:process_srcu for 181s
workqueue mm_percpu_wq: flags=0x108
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: vmstat_update
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: vmstat_update
workqueue writeback: flags=0x4a
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    in-flight: 8237:wb_workfn for 163s
workqueue ipv6_addrconf: flags=0x6000a
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=9
    in-flight: 8234:addrconf_verify_work for 148s
    inactive: 5*addrconf_verify_work
workqueue bat_events: flags=0x6000a
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=43
    in-flight: 58:batadv_iv_send_outstanding_bat_ogm_packet for 4s
    inactive: batadv_iv_send_outstanding_bat_ogm_packet, batadv_dat_purge, batadv_bla_periodic_work, 5*batadv_tt_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, batadv_dat_purge, batadv_bla_periodic_work, 2*batadv_dat_purge, batadv_bla_periodic_work, batadv_purge_orig, 7*batadv_iv_send_outstanding_bat_ogm_packet, 5*batadv_mcast_mla_update, 2*batadv_iv_send_outstanding_bat_ogm_packet, batadv_purge_orig, 3*batadv_iv_send_outstanding_bat_ogm_packet, 2*batadv_purge_orig, batadv_iv_send_outstanding_bat_ogm_packet, batadv_purge_orig
workqueue wg-kex-wg0: flags=0x124
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_decrypt_worker, wg_packet_tx_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_decrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg1: flags=0x6
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=2 refcnt=3
    in-flight: 8232:wg_packet_handshake_send_worker for 17s
    pending: wg_packet_handshake_send_worker
workqueue wg-crypt-wg1: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_decrypt_worker, wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-crypt-wg2: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x124
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_encrypt_worker
workqueue wg-crypt-wg1: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg2: flags=0x6
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    in-flight: 8243:wg_packet_handshake_send_worker for 1s
workqueue wg-crypt-wg2: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_encrypt_worker
workqueue wg-kex-wg0: flags=0x124
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_receive_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    in-flight: 42:wg_packet_handshake_receive_worker for 130s
    pending: wg_packet_handshake_receive_worker
workqueue wg-kex-wg0: flags=0x6
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_send_worker
workqueue wg-crypt-wg0: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_decrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg1: flags=0x124
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_receive_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    in-flight: 5966:wg_packet_handshake_receive_worker for 18s
    pending: wg_packet_handshake_receive_worker
workqueue wg-kex-wg1: flags=0x6
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=2 refcnt=3
    in-flight: 8242:wg_packet_handshake_send_worker for 11s ,6612:wg_packet_handshake_send_worker for 94s
workqueue wg-crypt-wg1: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_decrypt_worker, wg_packet_encrypt_worker, wg_packet_tx_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_tx_worker, wg_packet_encrypt_worker
workqueue wg-kex-wg2: flags=0x124
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-kex-wg2: flags=0x6
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    in-flight: 7839:wg_packet_handshake_send_worker for 10s
workqueue wg-crypt-wg2: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_tx_worker
workqueue wg-kex-wg0: flags=0x124
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_receive_worker
workqueue wg-crypt-wg0: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
  pwq 6: cpus=1 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_decrypt_worker, wg_packet_tx_worker
workqueue wg-kex-wg1: flags=0x6
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    in-flight: 8247:wg_packet_handshake_send_worker for 20s
workqueue wg-crypt-wg1: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=3 refcnt=4
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker, wg_packet_decrypt_worker
workqueue wg-kex-wg2: flags=0x6
  pwq 8: cpus=0-1 flags=0x4 nice=0 active=1 refcnt=2
    pending: wg_packet_handshake_send_worker
workqueue wg-crypt-wg2: flags=0x128
  pwq 2: cpus=0 node=0 flags=0x0 nice=0 active=2 refcnt=3
    pending: wg_packet_encrypt_worker, wg_packet_tx_worker
pool 2: cpus=0 node=0 flags=0x0 nice=0 hung=180s workers=7 idle: 5899 2156 5913 5892 5191 9
pool 6: cpus=1 node=0 flags=0x0 nice=0 hung=11s workers=9 idle: 5929 5940 5932 5962
pool 8: cpus=0-1 flags=0x4 nice=0 hung=1s workers=41 idle: 6610 34 1108 7833 7831 13 6611 12 1113 7823 142 7836 1167 1153 49 8252 8241 8259 8257 8255
Showing backtraces of busy workers in stalled worker pools:
pool 2:
task:kworker/0:4     state:R  running task     stack:23624 pid:5875  tgid:5875  ppid:2      task_flags:0x4208060 flags:0x00080000
Workqueue: rcu_gp process_srcu
Call Trace:
 <TASK>
 context_switch kernel/sched/core.c:5298 [inline]
 __schedule+0x15dd/0x52d0 kernel/sched/core.c:6911
 __schedule_loop kernel/sched/core.c:6993 [inline]
 schedule+0x164/0x360 kernel/sched/core.c:7008
 synchronize_rcu_expedited+0x584/0x770 kernel/rcu/tree_exp.h:976
 srcu_readers_active_idx_check kernel/rcu/srcutree.c:567 [inline]
 try_check_zero+0x294/0x470 kernel/rcu/srcutree.c:1180
 srcu_advance_state kernel/rcu/srcutree.c:1888 [inline]
 process_srcu+0x2d3/0x1220 kernel/rcu/srcutree.c:1997
 process_one_work kernel/workqueue.c:3276 [inline]
 process_scheduled_works+0xb6e/0x18c0 kernel/workqueue.c:3359
 worker_thread+0xa53/0xfc0 kernel/workqueue.c:3440
 kthread+0x388/0x470 kernel/kthread.c:436
 ret_from_fork+0x51e/0xb90 arch/x86/kernel/process.c:158
 ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:245
 </TASK>