Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AR9271 random freezes in Ad-hoc mode when distant client disconnect #145

Open
dagf2101 opened this issue Jul 12, 2018 · 14 comments
Open

AR9271 random freezes in Ad-hoc mode when distant client disconnect #145

dagf2101 opened this issue Jul 12, 2018 · 14 comments

Comments

@dagf2101
Copy link

This is similar than issue #137 but since it is Ad-hoc related and the problem happen when a remote client disconnect(instead of connect), I have decided to repost it in its own thread.

I'm also providing new logs with all ath debugs enabled.

Issue:
On AWUS036NHA in Ad-Hoc mode on kernel 4.15 / 4.16. We are using a network of around 8 devices connected to NUC PCs in our tests. Every time a device disconnect, other devices have a chance to crash. I reproduce it by rebooting one of the PC and eventually other devices start crashing the kernel thread freeze on those crashed nodes.

First the log show a working sta entry remove, and then later it show one that crashed the card.

Here is the kernel log:
7,9361,911527345,-;ath: phy2: Choose slot: 0, tsf: 528240644, tsftu: 515860, intval: 100
7,9362,911629831,-;ath: phy2: Choose slot: 0, tsf: 528343045, tsftu: 515960, intval: 100
7,9363,911732122,-;ath: phy2: Choose slot: 0, tsf: 528445445, tsftu: 516060, intval: 100
7,9364,911744161,-;ath: phy2: Set HW Key
4,9365,911745132,-;FDB: ath9k_htc_sta_remove before cancel_work_sync
4,9366,911745139,-;FDB: ath9k_htc_sta_remove after cancel_work_sync
7,9367,911745618,-;ath: phy2: Removed a station entry for: 00:c0:ca:97:3a:cc (idx: 4)
7,9368,911834579,-;ath: phy2: Choose slot: 0, tsf: 528547842, tsftu: 516160, intval: 100
7,9369,911936951,-;ath: phy2: Choose slot: 0, tsf: 528650245, tsftu: 516260, intval: 100
7,9370,912039331,-;ath: phy2: Choose slot: 0, tsf: 528752643, tsftu: 516360, intval: 100

...

7,9439,919102079,-;ath: phy2: Choose slot: 0, tsf: 535818244, tsftu: 523260, intval: 100
7,9440,919204380,-;ath: phy2: Choose slot: 0, tsf: 535920644, tsftu: 523360, intval: 100
7,9441,919306830,-;ath: phy2: Choose slot: 0, tsf: 536023045, tsftu: 523460, intval: 100
4,9442,919392253,-;FDB: ath9k_htc_sta_remove before cancel_work_sync
7,9443,919408963,-;ath: phy2: Choose slot: 0, tsf: 536125445, tsftu: 523560, intval: 100
7,9444,919511254,-;ath: phy2: Choose slot: 0, tsf: 536227843, tsftu: 523660, intval: 100
7,9445,919613631,-;ath: phy2: Choose slot: 0, tsf: 536330244, tsftu: 523760, intval: 100

...

7,9572,932615043,-;ath: phy2: Choose slot: 0, tsf: 549335042, tsftu: 536460, intval: 100
7,9573,932717546,-;ath: phy2: Choose slot: 0, tsf: 549437443, tsftu: 536560, intval: 100
7,9574,932819792,-;ath: phy2: Choose slot: 0, tsf: 549539845, tsftu: 536660, intval: 100
4,9575,932832054,-;asynchronous wait on fence i915:gnome-shell[781]/1:2963 timed out
7,9576,932922171,-;ath: phy2: Choose slot: 0, tsf: 549642243, tsftu: 536760, intval: 100
7,9577,933024543,-;ath: phy2: Choose slot: 0, tsf: 549744645, tsftu: 536860, intval: 100
7,9578,933126926,-;ath: phy2: Choose slot: 0, tsf: 549847043, tsftu: 536960, intval: 100
7,9579,933229292,-;ath: phy2: Choose slot: 0, tsf: 549949442, tsftu: 537060, intval: 100
7,9580,933434044,-;ath: phy2: Resuming beacon xmit after 1 misses
7,9581,933434045,-;ath: phy2: Choose slot: 0, tsf: 550154242, tsftu: 537260, intval: 100
7,9582,933536543,-;ath: phy2: Choose slot: 0, tsf: 550256644, tsftu: 537360, intval: 100
7,9583,933638920,-;ath: phy2: Choose slot: 0, tsf: 550359046, tsftu: 537460, intval: 100

...

7,9647,940190669,-;ath: phy2: Choose slot: 0, tsf: 556912645, tsftu: 543860, intval: 100
7,9648,940293051,-;ath: phy2: Choose slot: 0, tsf: 557015044, tsftu: 543960, intval: 100
7,9649,940395542,-;ath: phy2: Choose slot: 0, tsf: 557117443, tsftu: 544060, intval: 100
3,9650,940404014,-;INFO: rcu_sched self-detected stall on CPU
3,9651,940404019,-;\x091-....: (5249 ticks this GP) idle=cca/1/4611686018427387906 softirq=115167/115167 fqs=2625
3,9652,940404020,-;\x09 (t=5250 jiffies g=75919 c=75918 q=1429)
4,9653,940404022,-;NMI backtrace for cpu 1
4,9654,940404024,-;CPU: 1 PID: 131 Comm: kworker/u8:2 Tainted: G W 4.16.16-custom #1
4,9655,940404025,-;Hardware name: Intel Corporation NUC7i5DNB/NUC7i5DNB, BIOS DNKBLi5v.86A.0040.2018.0315.1451 03/15/2018
4,9656,940404039,-;Workqueue: phy2 ieee80211_iface_work [mac80211]
4,9657,940404040,-;Call Trace:
4,9658,940404043,-;
4,9659,940404046,-; dump_stack+0x5c/0x85
4,9660,940404048,-; nmi_cpu_backtrace+0xbf/0xd0
4,9661,940404050,-; ? lapic_can_unplug_cpu+0xa0/0xa0
4,9662,940404051,-; nmi_trigger_cpumask_backtrace+0xf5/0x130
4,9663,940404053,-; rcu_dump_cpu_stacks+0x9e/0xd7
4,9664,940404055,-; rcu_check_callbacks+0x6c8/0x910
4,9665,940404057,-; ? update_wall_time+0x474/0x6f0
4,9666,940404059,-; ? tick_sched_do_timer+0x40/0x40
4,9667,940404060,-; update_process_times+0x28/0x50
4,9668,940404061,-; tick_sched_handle+0x22/0x70
4,9669,940404062,-; tick_sched_timer+0x34/0x70
4,9670,940404064,-; __hrtimer_run_queues+0x108/0x290
4,9671,940404065,-; hrtimer_interrupt+0xe5/0x240
4,9672,940404067,-; smp_apic_timer_interrupt+0x62/0x120
4,9673,940404068,-; apic_timer_interrupt+0xf/0x20
4,9674,940404069,-;
4,9675,940404071,-;RIP: 0010:try_to_grab_pending+0x118/0x150
4,9676,940404072,-;RSP: 0018:ffffb8ab41ee3cc0 EFLAGS: 00000287 ORIG_RAX: ffffffffffffff12
4,9677,940404073,-;RAX: 00000000fffffffe RBX: ffff8e3ac9c05928 RCX: 0000000000000000
4,9678,940404074,-;RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000286
4,9679,940404074,-;RBP: ffffb8ab41ee3ce8 R08: ffff8e3b20002fe8 R09: ffff8e3b30c211c0
4,9680,940404075,-;R10: 0000000000000000 R11: 0000000000000040 R12: ffff8e3b30c211c0
4,9681,940404075,-;R13: ffffb8ab41ee3d08 R14: ffffffffb4890810 R15: ffff8e3ac9c05000
4,9682,940404077,-; ? get_work_pool+0x40/0x40
4,9683,940404079,-; __cancel_work_timer+0x42/0x1b0
4,9684,940404083,-; ath9k_htc_sta_remove+0x3b/0xa0 [ath9k_htc]
4,9685,940404089,-; drv_sta_state+0x25b/0x3f0 [mac80211]
4,9686,940404096,-; sta_info_move_state+0x181/0x260 [mac80211]
4,9687,940404102,-; __sta_info_destroy_part2+0x54/0x110 [mac80211]
4,9688,940404107,-; __sta_info_destroy+0x27/0x40 [mac80211]
4,9689,940404113,-; ieee80211_ibss_work+0x1be/0x580 [mac80211]
4,9690,940404115,-; ? kmem_cache_free+0x19c/0x1d0
4,9691,940404117,-; ? skb_dequeue+0x52/0x60
4,9692,940404124,-; ? ieee80211_iface_work+0xbe/0x340 [mac80211]
4,9693,940404125,-; process_one_work+0x17b/0x360
4,9694,940404127,-; worker_thread+0x2e/0x390
4,9695,940404129,-; ? process_one_work+0x360/0x360
4,9696,940404130,-; kthread+0x113/0x130
4,9697,940404131,-; ? kthread_create_worker_on_cpu+0x70/0x70
4,9698,940404133,-; ret_from_fork+0x35/0x40
7,9699,940497752,-;ath: phy2: Choose slot: 0, tsf: 557219844, tsftu: 544160, intval: 100

@dagf2101
Copy link
Author

Here is a other log, slightly different:

7,7330,1070405992,-;ath: phy2: Choose slot: 0, tsf: 7654670338, tsftu: 7475264, intval: 100
7,7331,1070508219,-;ath: phy2: Choose slot: 0, tsf: 7654772739, tsftu: 7475364, intval: 100
7,7332,1070551375,-;ath: phy2: Added a station entry for: 00:c0:ca:97:3a:a9 (idx: 5)
7,7333,1070552401,-;ath: phy2: Updated target sta: 00:c0:ca:97:3a:a9, rate caps: 0x0
4,7334,1070553174,-;FDB: ath9k_htc_sta_rc_update_work before lock
4,7335,1070553179,-;FDB: ath9k_htc_sta_rc_update_work after lock
4,7336,1070553185,-;FDB: ath9k_htc_sta_rc_update_work:1302
4,7337,1070553191,-;FDB: ath9k_htc_sta_rc_update_work:1306
7,7338,1070554109,-;ath: phy2: Supported rates for sta: 00:c0:ca:97:3a:a9 updated, rate caps: 0xC
4,7339,1070554114,-;FDB: ath9k_htc_sta_rc_update_work:1316
4,7340,1070554118,-;FDB: ath9k_htc_sta_rc_update_work:1318
4,7341,1070554120,-;FDB: ath9k_htc_sta_rc_update_work DONE
7,7342,1070610610,-;ath: phy2: Choose slot: 0, tsf: 7654875138, tsftu: 7475464, intval: 100
7,7343,1070663599,-;ath: phy2: Set HW Key
7,7344,1070712789,-;ath: phy2: Choose slot: 0, tsf: 7654977540, tsftu: 7475564, intval: 100
7,7345,1070815237,-;ath: phy2: Choose slot: 0, tsf: 7655079941, tsftu: 7475664, intval: 100
...
7,7424,1078898491,-;ath: phy2: Choose slot: 0, tsf: 7663169541, tsftu: 7483564, intval: 100
7,7425,1079000780,-;ath: phy2: Choose slot: 0, tsf: 7663271939, tsftu: 7483664, intval: 100
7,7426,1079095560,-;ath: phy2: Added a station entry for: 00:c0:ca:97:3a:99 (idx: 6)
7,7427,1079096227,-;ath: phy2: Updated target sta: 00:c0:ca:97:3a:99, rate caps: 0x0
7,7428,1079103237,-;ath: phy2: Choose slot: 0, tsf: 7663374339, tsftu: 7483764, intval: 100
4,7429,1079166020,-;FDB: ath9k_htc_sta_rc_update_work before lock
4,7430,1079166026,-;FDB: ath9k_htc_sta_rc_update_work after lock
4,7431,1079166032,-;FDB: ath9k_htc_sta_rc_update_work:1302
4,7432,1079166038,-;FDB: ath9k_htc_sta_rc_update_work:1306
7,7433,1079166984,-;ath: phy2: Supported rates for sta: 00:c0:ca:97:3a:99 updated, rate caps: 0xC
4,7434,1079166991,-;FDB: ath9k_htc_sta_rc_update_work:1316
4,7435,1079166995,-;FDB: ath9k_htc_sta_rc_update_work:1318
4,7436,1079166998,-;FDB: ath9k_htc_sta_rc_update_work DONE
7,7437,1079205324,-;ath: phy2: Choose slot: 0, tsf: 7663476740, tsftu: 7483864, intval: 100
7,7438,1079239409,-;ath: phy2: Set HW Key
7,7439,1079307868,-;ath: phy2: Choose slot: 0, tsf: 7663579140, tsftu: 7483964, intval: 100
7,7440,1079410113,-;ath: phy2: Choose slot: 0, tsf: 7663681538, tsftu: 7484064, intval: 100
...
7,7459,1081353991,-;ath: phy2: Choose slot: 0, tsf: 7665627139, tsftu: 7485964, intval: 100
7,7460,1081456233,-;ath: phy2: Choose slot: 0, tsf: 7665729540, tsftu: 7486064, intval: 100
7,7461,1081546056,-;ath: phy2: Added a station entry for: 00:c0:ca:97:3a:b6 (idx: 7)
7,7462,1081546931,-;ath: phy2: Updated target sta: 00:c0:ca:97:3a:b6, rate caps: 0x0
4,7463,1081547026,-;FDB: ath9k_htc_sta_rc_update_work before lock
4,7464,1081547029,-;FDB: ath9k_htc_sta_rc_update_work after lock
4,7465,1081547035,-;FDB: ath9k_htc_sta_rc_update_work:1302
4,7466,1081547041,-;FDB: ath9k_htc_sta_rc_update_work:1306
7,7467,1081548104,-;ath: phy2: Supported rates for sta: 00:c0:ca:97:3a:b6 updated, rate caps: 0xC
4,7468,1081548128,-;FDB: ath9k_htc_sta_rc_update_work:1316
4,7469,1081548154,-;FDB: ath9k_htc_sta_rc_update_work:1318
4,7470,1081548176,-;FDB: ath9k_htc_sta_rc_update_work DONE
7,7471,1081558612,-;ath: phy2: Choose slot: 0, tsf: 7665831940, tsftu: 7486164, intval: 100
7,7472,1081660862,-;ath: phy2: Choose slot: 0, tsf: 7665934341, tsftu: 7486264, intval: 100
7,7473,1081704893,-;ath: phy2: Set HW Key
7,7474,1081763241,-;ath: phy2: Choose slot: 0, tsf: 7666036739, tsftu: 7486364, intval: 100
7,7475,1081865510,-;ath: phy2: Choose slot: 0, tsf: 7666139138, tsftu: 7486464, intval: 100
...
7,7576,1092196986,-;ath: phy2: Choose slot: 0, tsf: 7676481539, tsftu: 7496564, intval: 100
7,7577,1092299243,-;ath: phy2: Choose slot: 0, tsf: 7676583939, tsftu: 7496664, intval: 100
6,7578,1092391330,-;manet: failed to move IBSS STA 00:c0:ca:97:3a:a6 to state 3 (-105) - keeping it anyway
7,7579,1092401489,-;ath: phy2: Choose slot: 0, tsf: 7676686340, tsftu: 7496764, intval: 100
4,7580,1092444881,-;WARNING: CPU: 0 PID: 2095 at kernel/workqueue.c:1444 __queue_work+0x34b/0x3f0
4,7581,1092444886,-;Modules linked in: ath9k_htc ath9k_common ath9k_hw ath ctr ccm tun fuse snd_hda_codec_hdmi binfmt_misc arc4 btusb btrtl btbcm btintel snd_soc_skl bluetooth snd_soc_skl_ipc iwlmvm snd_hda_ext_core snd_soc_sst_dsp snd_soc_sst_ipc snd_soc_acpi mac80211 hid_generic wmi_bmof evdev i2c_designware_platform jitterentropy_rng i2c_designware_core snd_soc_core snd_compress intel_rapl x86_pkg_temp_thermal iwlwifi intel_powerclamp snd_hda_intel kvm_intel snd_hda_codec drbg snd_hda_core kvm irqbypass i915 intel_cstate snd_hwdep intel_uncore ansi_cprng idma64 intel_rapl_perf snd_pcm pcspkr drm_kms_helper snd_timer mei_me cfg80211 snd ecdh_generic iTCO_wdt intel_lpss_pci drm rfkill iTCO_vendor_support sg soundcore shpchp intel_lpss mei mfd_core i2c_algo_bit intel_pch_thermal video usbhid hid wmi acpi_pad
4,7582,1092445037,c; button coretemp ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic fscrypto ecb sd_mod crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd ahci libahci xhci_pci e1000e libata xhci_hcd i2c_i801 scsi_mod usbcore usb_common fan thermal [last unloaded: ath]
4,7583,1092445115,-;CPU: 0 PID: 2095 Comm: kworker/u8:2 Not tainted 4.16.16-custom #1
4,7584,1092445119,-;Hardware name: Intel Corporation NUC7i5DNB/NUC7i5DNB, BIOS DNKBLi5v.86A.0040.2018.0315.1451 03/15/2018
4,7585,1092445185,-;Workqueue: phy2 ieee80211_iface_work [mac80211]
4,7586,1092445199,-;RIP: 0010:__queue_work+0x34b/0x3f0
4,7587,1092445204,-;RSP: 0018:ffffaf3103467a20 EFLAGS: 00010086
4,7588,1092445210,-;RAX: 0000000000000000 RBX: ffffffffae0a4700 RCX: 0000000000000000
4,7589,1092445214,-;RDX: ffff93afd78c1930 RSI: 0000000000000000 RDI: ffff93aff0c211c0
4,7590,1092445218,-;RBP: ffff93afe0406c00 R08: ffff93afe0001b60 R09: ffff93aff0c211c0
4,7591,1092445222,-;R10: 0000000000000000 R11: 0000000000000040 R12: 0000000000000000
4,7592,1092445226,-;R13: 0000000000000200 R14: ffff93afd78c1928 R15: ffff93aff0c25100
4,7593,1092445233,-;FS: 0000000000000000(0000) GS:ffff93aff0c00000(0000) knlGS:0000000000000000
4,7594,1092445238,-;CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
4,7595,1092445242,-;CR2: 000055f7103c6b58 CR3: 00000002e640a001 CR4: 00000000003606f0
4,7596,1092445246,-;DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
4,7597,1092445250,-;DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
4,7598,1092445252,-;Call Trace:
4,7599,1092445271,-; queue_work_on+0x34/0x40
4,7600,1092445316,-; drv_sta_rc_update+0x85/0x150 [mac80211]
4,7601,1092445369,-; ieee80211_rx_mgmt_probe_beacon+0x460/0x700 [mac80211]
4,7602,1092445425,-; ieee80211_ibss_rx_queued_mgmt+0x18a/0x420 [mac80211]
4,7603,1092445438,-; ? load_balance+0x161/0x9e0
4,7604,1092445451,-; ? __switch_to_asm+0x40/0x70
4,7605,1092445460,-; ? __switch_to_asm+0x34/0x70
4,7606,1092445468,-; ? __switch_to_asm+0x40/0x70
4,7607,1092445476,-; ? __switch_to_asm+0x34/0x70
4,7608,1092445485,-; ? __switch_to_asm+0x40/0x70
4,7609,1092445493,-; ? __switch_to_asm+0x34/0x70
4,7610,1092445501,-; ? __switch_to_asm+0x40/0x70
4,7611,1092445510,-; ? __switch_to_asm+0x34/0x70
4,7612,1092445518,-; ? __switch_to_asm+0x40/0x70
4,7613,1092445526,-; ? __switch_to_asm+0x34/0x70
4,7614,1092445535,-; ? __switch_to_asm+0x40/0x70
4,7615,1092445543,-; ? __switch_to_asm+0x34/0x70
4,7616,1092445551,-; ? __switch_to_asm+0x40/0x70
4,7617,1092445560,-; ? __switch_to_asm+0x34/0x70
4,7618,1092445568,-; ? __switch_to_asm+0x40/0x70
4,7619,1092445576,-; ? __switch_to_asm+0x34/0x70
4,7620,1092445585,-; ? __switch_to_asm+0x40/0x70
4,7621,1092445593,-; ? __switch_to_asm+0x34/0x70
4,7622,1092445601,-; ? __switch_to_asm+0x40/0x70
4,7623,1092445610,-; ? __switch_to_asm+0x34/0x70
4,7624,1092445618,-; ? __switch_to_asm+0x40/0x70
4,7625,1092445626,-; ? __switch_to_asm+0x34/0x70
4,7626,1092445634,-; ? __switch_to_asm+0x40/0x70
4,7627,1092445643,-; ? __switch_to_asm+0x34/0x70
4,7628,1092445651,-; ? __switch_to_asm+0x40/0x70
4,7629,1092445659,-; ? __switch_to_asm+0x34/0x70
4,7630,1092445669,-; ? __switch_to+0xa2/0x450
4,7631,1092445711,-; ieee80211_iface_work+0x22c/0x340 [mac80211]
4,7632,1092445725,-; process_one_work+0x17b/0x360
4,7633,1092445736,-; worker_thread+0x2e/0x390
4,7634,1092445746,-; ? process_one_work+0x360/0x360
4,7635,1092445754,-; kthread+0x113/0x130
4,7636,1092445764,-; ? kthread_create_worker_on_cpu+0x70/0x70
4,7637,1092445775,-; ret_from_fork+0x35/0x40
4,7638,1092445781,-;Code: 3c 25 00 5c 01 00 f6 47 24 20 74 18 e8 8f 88 00 00 48 85 c0 74 0e 48 8b 40 20 48 3b 68 08 0f 84 e8 fc ff ff 0f 0b e9 c3 fd ff ff <0f> 0b 49 8b 3f c6 07 00 0f 1f 40 00 e9 b2 fd ff ff 83 cd 02 49
4,7639,1092445911,-;---[ end trace ae5347402132e49d ]---
7,7640,1092503742,-;ath: phy2: Choose slot: 0, tsf: 7676788739, tsftu: 7496864, intval: 100
7,7641,1092605928,-;ath: phy2: Choose slot: 0, tsf: 7676891138, tsftu: 7496964, intval: 100
..
7,7796,1108455614,-;ath: phy2: Choose slot: 0, tsf: 7692763138, tsftu: 7512464, intval: 100
7,7797,1108557868,-;ath: phy2: Choose slot: 0, tsf: 7692865539, tsftu: 7512564, intval: 100
7,7798,1108864733,-;ath: phy2: Resuming beacon xmit after 2 misses
7,7799,1108864774,-;ath: phy2: Choose slot: 0, tsf: 7693172740, tsftu: 7512864, intval: 100
7,7800,1108967119,-;ath: phy2: Choose slot: 0, tsf: 7693275141, tsftu: 7512964, intval: 100
..
7,8171,1146908156,-;ath: phy2: Choose slot: 0, tsf: 7731265538, tsftu: 7550064, intval: 100
7,8172,1147010364,-;ath: phy2: Choose slot: 0, tsf: 7731367939, tsftu: 7550164, intval: 100
7,8173,1147214991,-;ath: phy2: Resuming beacon xmit after 1 misses
7,8174,1147215002,-;ath: phy2: Choose slot: 0, tsf: 7731572740, tsftu: 7550364, intval: 100
7,8175,1147317235,-;ath: phy2: Choose slot: 0, tsf: 7731675140, tsftu: 7550464, intval: 100
..
7,8809,1212167992,-;ath: phy2: Choose slot: 0, tsf: 7796596740, tsftu: 7613864, intval: 100
7,8810,1212270244,-;ath: phy2: Choose slot: 0, tsf: 7796699137, tsftu: 7613964, intval: 100
7,8811,1212280248,-;ath: phy2: Set HW Key
4,8812,1212281395,-;FDB: ath9k_htc_sta_remove before cancel_work_sync
4,8813,1212281403,-;FDB: ath9k_htc_sta_remove after cancel_work_sync
7,8814,1212281856,-;ath: phy2: Removed a station entry for: 00:c0:ca:97:3a:cc (idx: 2)
7,8815,1212336513,-;ath: phy2: Added a station entry for: 00:c0:ca:97:3a:cc (idx: 2)
7,8816,1212337325,-;ath: phy2: Updated target sta: 00:c0:ca:97:3a:cc, rate caps: 0x0
7,8817,1212372575,-;ath: phy2: Choose slot: 0, tsf: 7796801538, tsftu: 7614064, intval: 100
7,8818,1212474878,-;ath: phy2: Choose slot: 0, tsf: 7796903938, tsftu: 7614164, intval: 100
7,8819,1212577124,-;ath: phy2: Choose slot: 0, tsf: 7797006339, tsftu: 7614264, intval: 100
7,8820,1212679615,-;ath: phy2: Choose slot: 0, tsf: 7797108741, tsftu: 7614364, intval: 100
7,8821,1212760246,-;ath: phy2: Set HW Key
4,8822,1212761442,-;FDB: ath9k_htc_sta_remove before cancel_work_sync
4,8823,1212761451,-;FDB: ath9k_htc_sta_remove after cancel_work_sync
7,8824,1212761923,-;ath: phy2: Removed a station entry for: 00:c0:ca:97:3a:b9 (idx: 1)
7,8825,1212781638,-;ath: phy2: Choose slot: 0, tsf: 7797211138, tsftu: 7614464, intval: 100
7,8826,1212884125,-;ath: phy2: Choose slot: 0, tsf: 7797313541, tsftu: 7614564, intval: 100
7,8827,1212946450,-;ath: phy2: Added a station entry for: 00:c0:ca:97:3a:b9 (idx: 1)
7,8828,1212947309,-;ath: phy2: Updated target sta: 00:c0:ca:97:3a:b9, rate caps: 0x0
4,8829,1212947404,-;FDB: ath9k_htc_sta_rc_update_work before lock
4,8830,1212947407,-;FDB: ath9k_htc_sta_rc_update_work after lock
4,8831,1212947413,-;FDB: ath9k_htc_sta_rc_update_work:1302
4,8832,1212947419,-;FDB: ath9k_htc_sta_rc_update_work:1306
7,8833,1212948341,-;ath: phy2: Supported rates for sta: 00:c0:ca:97:3a:b9 updated, rate caps: 0xC
4,8834,1212948348,-;FDB: ath9k_htc_sta_rc_update_work:1316
4,8835,1212948353,-;FDB: ath9k_htc_sta_rc_update_work:1318
4,8836,1212948356,-;FDB: ath9k_htc_sta_rc_update_work DONE
7,8837,1212979276,-;ath: phy2: Set HW Key
7,8838,1212986269,-;ath: phy2: Choose slot: 0, tsf: 7797415939, tsftu: 7614664, intval: 100
7,8839,1213088494,-;ath: phy2: Choose slot: 0, tsf: 7797518340, tsftu: 7614764, intval: 100
..
7,8840,1213190878,-;ath: phy2: Choose slot: 0, tsf: 7797620740, tsftu: 7614864, intval: 100
7,8841,1213293109,-;ath: phy2: Choose slot: 0, tsf: 7797723140, tsftu: 7614964, intval: 100
7,8842,1213352169,-;ath: phy2: Set HW Key
7,8843,1213395263,-;ath: phy2: Choose slot: 0, tsf: 7797825541, tsftu: 7615064, intval: 100
7,8844,1213497612,-;ath: phy2: Choose slot: 0, tsf: 7797927939, tsftu: 7615164, intval: 100
..
7,8984,1227818862,-;ath: phy2: Choose slot: 0, tsf: 7812263940, tsftu: 7629164, intval: 100
7,8985,1227921111,-;ath: phy2: Choose slot: 0, tsf: 7812366339, tsftu: 7629264, intval: 100
7,8986,1227980099,-;ath: phy2: Set HW Key
4,8987,1227981011,-;FDB: ath9k_htc_sta_remove before cancel_work_sync
4,8988,1227981019,-;FDB: ath9k_htc_sta_remove after cancel_work_sync
7,8989,1227981497,-;ath: phy2: Removed a station entry for: 00:c0:ca:97:3a:b9 (idx: 1)
7,8990,1227982236,-;ath: phy2: Added a station entry for: 00:c0:ca:97:3a:b9 (idx: 1)
7,8991,1227983047,-;ath: phy2: Updated target sta: 00:c0:ca:97:3a:b9, rate caps: 0x0
4,8992,1227983155,-;FDB: ath9k_htc_sta_rc_update_work before lock
4,8993,1227983158,-;FDB: ath9k_htc_sta_rc_update_work after lock
4,8994,1227983165,-;FDB: ath9k_htc_sta_rc_update_work:1302
4,8995,1227983170,-;FDB: ath9k_htc_sta_rc_update_work:1306
7,8996,1227984171,-;ath: phy2: Supported rates for sta: 00:c0:ca:97:3a:b9 updated, rate caps: 0xC
4,8997,1227984181,-;FDB: ath9k_htc_sta_rc_update_work:1316
4,8998,1227984186,-;FDB: ath9k_htc_sta_rc_update_work:1318
4,8999,1227984193,-;FDB: ath9k_htc_sta_rc_update_work DONE
7,9000,1227992550,-;ath: phy2: Set HW Key
7,9001,1228023407,-;ath: phy2: Choose slot: 0, tsf: 7812468739, tsftu: 7629364, intval: 100
7,9002,1228125735,-;ath: phy2: Choose slot: 0, tsf: 7812571140, tsftu: 7629464, intval: 100
..
7,9374,1266189988,-;ath: phy2: Choose slot: 0, tsf: 7850663938, tsftu: 7666664, intval: 100
7,9375,1266292164,-;ath: phy2: Choose slot: 0, tsf: 7850766339, tsftu: 7666764, intval: 100
7,9376,1266496983,-;ath: phy2: Resuming beacon xmit after 1 misses
7,9377,1266497025,-;ath: phy2: Choose slot: 0, tsf: 7850971139, tsftu: 7666964, intval: 100
7,9378,1266599265,-;ath: phy2: Choose slot: 0, tsf: 7851073541, tsftu: 7667064, intval: 100
..
7,9637,1293098617,-;ath: phy2: Choose slot: 0, tsf: 7877595140, tsftu: 7692964, intval: 100
7,9638,1293200984,-;ath: phy2: Choose slot: 0, tsf: 7877697541, tsftu: 7693064, intval: 100
4,9639,1293300247,-;FDB: ath9k_htc_sta_remove before cancel_work_sync
7,9640,1293303264,-;ath: phy2: Choose slot: 0, tsf: 7877799937, tsftu: 7693164, intval: 100
7,9641,1293405587,-;ath: phy2: Choose slot: 0, tsf: 7877902340, tsftu: 7693264, intval: 100
..
7,9844,1314179206,-;ath: phy2: Choose slot: 0, tsf: 7898689539, tsftu: 7713564, intval: 100
7,9845,1314281330,-;ath: phy2: Choose slot: 0, tsf: 7898791941, tsftu: 7713664, intval: 100
3,9846,1314312022,-;INFO: rcu_sched self-detected stall on CPU
3,9847,1314312029,-;\x093-....: (5249 ticks this GP) idle=14e/1/4611686018427387906 softirq=137394/137394 fqs=2618
3,9848,1314312030,-;\x09 (t=5250 jiffies g=100671 c=100670 q=3305)
4,9849,1314312032,-;NMI backtrace for cpu 3
4,9850,1314312034,-;CPU: 3 PID: 1992 Comm: kworker/u8:0 Tainted: G W 4.16.16-custom #1
4,9851,1314312034,-;Hardware name: Intel Corporation NUC7i5DNB/NUC7i5DNB, BIOS DNKBLi5v.86A.0040.2018.0315.1451 03/15/2018
4,9852,1314312050,-;Workqueue: phy2 ieee80211_iface_work [mac80211]
4,9853,1314312051,-;Call Trace:
4,9854,1314312054,-;
4,9855,1314312058,-; dump_stack+0x5c/0x85
4,9856,1314312060,-; nmi_cpu_backtrace+0xbf/0xd0
4,9857,1314312062,-; ? lapic_can_unplug_cpu+0xa0/0xa0
4,9858,1314312064,-; nmi_trigger_cpumask_backtrace+0xf5/0x130
4,9859,1314312066,-; rcu_dump_cpu_stacks+0x9e/0xd7
4,9860,1314312068,-; rcu_check_callbacks+0x6c8/0x910
4,9861,1314312070,-; ? update_wall_time+0x474/0x6f0
4,9862,1314312071,-; ? tick_sched_do_timer+0x40/0x40
4,9863,1314312072,-; update_process_times+0x28/0x50
4,9864,1314312073,-; tick_sched_handle+0x22/0x70
4,9865,1314312074,-; tick_sched_timer+0x34/0x70
4,9866,1314312076,-; __hrtimer_run_queues+0x108/0x290
4,9867,1314312077,-; hrtimer_interrupt+0xe5/0x240
4,9868,1314312079,-; smp_apic_timer_interrupt+0x62/0x120
4,9869,1314312081,-; apic_timer_interrupt+0xf/0x20
4,9870,1314312081,-;
4,9871,1314312084,-;RIP: 0010:try_to_grab_pending+0xfc/0x150
4,9872,1314312084,-;RSP: 0018:ffffaf3104187cc0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff12
4,9873,1314312085,-;RAX: 0000000000000000 RBX: ffff93afd78c1928 RCX: 0000000000000000
4,9874,1314312086,-;RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000286
4,9875,1314312086,-;RBP: ffffaf3104187ce8 R08: ffff93afe0001b60 R09: ffff93aff0c211c0
4,9876,1314312087,-;R10: 0000000000000000 R11: 0000000000000040 R12: ffff93aff0c211c0
4,9877,1314312087,-;R13: ffffaf3104187d08 R14: ffffffffad290810 R15: ffff93afd78c1000
4,9878,1314312089,-; ? get_work_pool+0x40/0x40
4,9879,1314312091,-; __cancel_work_timer+0x42/0x1b0
4,9880,1314312095,-; ath9k_htc_sta_remove+0x3b/0xa0 [ath9k_htc]
4,9881,1314312102,-; drv_sta_state+0x25b/0x3f0 [mac80211]
4,9882,1314312108,-; sta_info_move_state+0x181/0x260 [mac80211]
4,9883,1314312113,-; __sta_info_destroy_part2+0x54/0x110 [mac80211]
4,9884,1314312119,-; __sta_info_destroy+0x27/0x40 [mac80211]
4,9885,1314312125,-; ieee80211_ibss_work+0x1be/0x580 [mac80211]
4,9886,1314312128,-; ? kmem_cache_free+0x19c/0x1d0
4,9887,1314312129,-; ? skb_dequeue+0x52/0x60
4,9888,1314312135,-; ? ieee80211_iface_work+0xbe/0x340 [mac80211]
4,9889,1314312137,-; process_one_work+0x17b/0x360
4,9890,1314312139,-; worker_thread+0x2e/0x390
4,9891,1314312140,-; ? process_one_work+0x360/0x360
4,9892,1314312141,-; kthread+0x113/0x130
4,9893,1314312143,-; ? kthread_create_worker_on_cpu+0x70/0x70
4,9894,1314312144,-; ? SyS_exit+0x13/0x20
4,9895,1314312146,-; ret_from_fork+0x35/0x40
7,9896,1314383705,-;ath: phy2: Choose slot: 0, tsf: 7898894341, tsftu: 7713764, intval: 100
7,9897,1314486079,-;ath: phy2: Choose slot: 0, tsf: 7898996740, tsftu: 7713864, intval: 100
...
7,10145,1339867459,-;ath: phy2: Choose slot: 0, tsf: 7924391939, tsftu: 7738664, intval: 100
7,10146,1339969829,-;ath: phy2: Choose slot: 0, tsf: 7924494340, tsftu: 7738764, intval: 100
0,10147,1340040022,-;watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u8:0:1992]
4,10148,1340040025,-;Modules linked in: ath9k_htc ath9k_common ath9k_hw ath ctr ccm tun fuse snd_hda_codec_hdmi binfmt_misc arc4 btusb btrtl btbcm btintel snd_soc_skl bluetooth snd_soc_skl_ipc iwlmvm snd_hda_ext_core snd_soc_sst_dsp snd_soc_sst_ipc snd_soc_acpi mac80211 hid_generic wmi_bmof evdev i2c_designware_platform jitterentropy_rng i2c_designware_core snd_soc_core snd_compress intel_rapl x86_pkg_temp_thermal iwlwifi intel_powerclamp snd_hda_intel kvm_intel snd_hda_codec drbg snd_hda_core kvm irqbypass i915 intel_cstate snd_hwdep intel_uncore ansi_cprng idma64 intel_rapl_perf snd_pcm pcspkr drm_kms_helper snd_timer mei_me cfg80211 snd ecdh_generic iTCO_wdt intel_lpss_pci drm rfkill iTCO_vendor_support sg soundcore shpchp intel_lpss mei mfd_core i2c_algo_bit intel_pch_thermal video usbhid hid wmi acpi_pad
4,10149,1340040054,c; button coretemp ip_tables x_tables autofs4 ext4 crc16 mbcache jbd2 crc32c_generic fscrypto ecb sd_mod crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd ahci libahci xhci_pci e1000e libata xhci_hcd i2c_i801 scsi_mod usbcore usb_common fan thermal [last unloaded: ath]
4,10150,1340040068,-;CPU: 3 PID: 1992 Comm: kworker/u8:0 Tainted: G W 4.16.16-custom #1
4,10151,1340040069,-;Hardware name: Intel Corporation NUC7i5DNB/NUC7i5DNB, BIOS DNKBLi5v.86A.0040.2018.0315.1451 03/15/2018
4,10152,1340040083,-;Workqueue: phy2 ieee80211_iface_work [mac80211]
4,10153,1340040086,-;RIP: 0010:try_to_grab_pending+0xfc/0x150
4,10154,1340040087,-;RSP: 0018:ffffaf3104187cc0 EFLAGS: 00000286 ORIG_RAX: ffffffffffffff12
4,10155,1340040088,-;RAX: 0000000000000000 RBX: ffff93afd78c1928 RCX: 0000000000000000
4,10156,1340040089,-;RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000286
4,10157,1340040090,-;RBP: ffffaf3104187ce8 R08: ffff93afe0001b60 R09: ffff93aff0c211c0
4,10158,1340040090,-;R10: 0000000000000000 R11: 0000000000000040 R12: ffff93aff0c211c0
4,10159,1340040091,-;R13: ffffaf3104187d08 R14: ffffffffad290810 R15: ffff93afd78c1000
4,10160,1340040092,-;FS: 0000000000000000(0000) GS:ffff93aff0d80000(0000) knlGS:0000000000000000
4,10161,1340040092,-;CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
4,10162,1340040093,-;CR2: 00007f9feb105000 CR3: 00000002e640a001 CR4: 00000000003606e0
4,10163,1340040094,-;DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
4,10164,1340040094,-;DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
4,10165,1340040095,-;Call Trace:
4,10166,1340040099,-; __cancel_work_timer+0x42/0x1b0
4,10167,1340040102,-; ath9k_htc_sta_remove+0x3b/0xa0 [ath9k_htc]
4,10168,1340040108,-; drv_sta_state+0x25b/0x3f0 [mac80211]
4,10169,1340040115,-; sta_info_move_state+0x181/0x260 [mac80211]
4,10170,1340040120,-; __sta_info_destroy_part2+0x54/0x110 [mac80211]
4,10171,1340040125,-; __sta_info_destroy+0x27/0x40 [mac80211]
4,10172,1340040132,-; ieee80211_ibss_work+0x1be/0x580 [mac80211]
4,10173,1340040135,-; ? kmem_cache_free+0x19c/0x1d0
4,10174,1340040137,-; ? skb_dequeue+0x52/0x60
4,10175,1340040143,-; ? ieee80211_iface_work+0xbe/0x340 [mac80211]
4,10176,1340040145,-; process_one_work+0x17b/0x360
4,10177,1340040146,-; worker_thread+0x2e/0x390
4,10178,1340040148,-; ? process_one_work+0x360/0x360
4,10179,1340040149,-; kthread+0x113/0x130
4,10180,1340040151,-; ? kthread_create_worker_on_cpu+0x70/0x70
4,10181,1340040152,-; ? SyS_exit+0x13/0x20
4,10182,1340040154,-; ret_from_fork+0x35/0x40
4,10183,1340040156,-;Code: 48 83 c8 01 4c 89 e7 48 89 03 c6 07 00 0f 1f 40 00 b8 01 00 00 00 e9 4d ff ff ff 4c 89 e7 c6 07 00 0f 1f 40 00 48 8b 7d 00 57 9d <0f> 1f 44 00 00 48 8b 13 b8 fe ff ff ff 83 e2 14 48 83 fa 10 0f

@olerem
Copy link
Contributor

olerem commented Jul 12, 2018

Hi, i know there is some thing fishy with ad-hoc mode. For some time I made this script to crash the system just by spoofing ad-hock interface:
https://pastebin.com/KhXgQ15V
But i really don't have time to do it now. If you can investigate it, it will be really good.

@dagf2101
Copy link
Author

Just tried your script, looks like I can use it to reproduce our bug.
Instead of connecting to the adhoc using iw dev ibss I am using the wpa_supplicant with the following configuration: https://pastebin.com/hpLJ0LBW

Simply ifconfig down; macchanger; ifconfig up can trigger the same bug.

Do you think it is a firmware or driver related problem ?

@erikarn
Copy link
Collaborator

erikarn commented Jul 12, 2018 via email

@dagf2101
Copy link
Author

I thought it was freezing because the card was dead and not responding anymore. I will try to get a cable to connect to the UART and log the firmware output...

Anything else I can try or enable in the kernel that could help ?

@erikarn
Copy link
Collaborator

erikarn commented Jul 12, 2018 via email

@olerem
Copy link
Contributor

olerem commented Jul 13, 2018

My last research on this topic was in 20 Oct 2014:
https://www.spinics.net/lists/linux-wireless/msg128248.html
may be it can help you.

@dagf2101
Copy link
Author

Thanks @olerem for this information.

It looks like I would need to increase the max station support into the firmware.
(probably from 8 to something around 16)

The Idea proposed here is to remove some station information from the firmware and keep it in the kernel driver instead.

I am pretty lost looking at the firmware code, can someone point me where to start looking or where this hard limit is enforced in the firmware ?

Or if you guys have a idea where we can easily save resources.

@olerem
Copy link
Contributor

olerem commented Jul 25, 2018

@dagf2101 , on you place I would attach UART and add some prints in to the firmware. If will allow you to see how it works. Suddenly we have no available open source solution to JTAG this chip.

@dagf2101
Copy link
Author

Yeah I will do but still haven't received the cable... damn bureaucracy.

@olerem
Copy link
Contributor

olerem commented Jul 25, 2018

I thought each SW/HW developer has a bucket full of cheap uart adapters...

@dagf2101
Copy link
Author

Looks like the bucket is empty here.

@dagf2101
Copy link
Author

Ok I received this cable from digikey but cant make it work. Soldered the rx, tx and ground as shown here: https://photos.app.goo.gl/PVC8acPcVmnzPTnq8
(I also tried to invert rx and tx)

Trying to connect to the USB serial device (19200) but I dont see anything...
I'm also getting weird driver errors into ieee80211_ibss_rx_queued_mgmt.

Maybe I have the wrong cable or I'm missing something ?

@olerem
Copy link
Contributor

olerem commented Jul 25, 2018

I already had one issue with exact this cable. Purchased 3v3 version and got 5V version. Check this with scope or logic analyzer if you have. If not, make sure the WiFi is still working after desoldering of this cable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants