No WiFi device found on Ubuntu - ath10k has crashed during init

[filipleple]

Device

Protectli VP4650

Dasharo version

protectli_vault_cml_v1.1.0_vp46xx (coreboot 4.21 rebase)

Affected component(s) or functionality

WiFi scanning

Brief summary

nmcli device wifi rescan returns Error: No Wi-Fi device found.. The network controller is however listed in lspci (Qualcomm Atheros)

How reproducible

100%

How to reproduce

Boot Ubuntu and log in via SSH
Execute nmcli device wifi rescan

Expected behavior

It should do a WiFi scan and return found networks

Actual behavior

Error: No Wi-Fi device found.

Screenshots

root@3mdeb:/home/ubuntu# nmcli device wifi rescan
Error: No Wi-Fi device found.

root@3mdeb:/home/ubuntu# lspci | grep "Network controller:"
07:00.0 Network controller: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter (rev 30)

root@3mdeb:/home/ubuntu# dmesg | grep ath
[    3.496439] usbcore: registered new interface driver ath3k
[    3.573298] ath10k_pci 0000:07:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[    3.847564] ath10k_pci 0000:07:00.0: qca6174 hw3.0 target 0x05020000 chip_id 0x003409ff sub 168c:3361
[    3.847568] ath10k_pci 0000:07:00.0: kconfig debug 0 debugfs 1 tracing 1 dfs 0 testmode 0
[    3.847937] ath10k_pci 0000:07:00.0: firmware ver WLAN.RM.4.4.1-00157-QCARMSWPZ-1 api 6 features wowlan,ignore-otp,mfp crc32 90eebefb
[    3.917880] ath10k_pci 0000:07:00.0: board_file api 2 bmi_id N/A crc32 706c395e
[    5.980960] ath10k_pci 0000:07:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[    5.980962] ath10k_pci 0000:07:00.0:   device [168c:003e] error status/mask=00000001/00006000
[    5.980965] ath10k_pci 0000:07:00.0:    [ 0] RxErr                 
[   11.098547] ath10k_pci 0000:07:00.0: wmi unified ready event not received
[   11.148567] ath10k_pci 0000:07:00.0: device has crashed during init
[   11.176535] ath10k_pci 0000:07:00.0: device has crashed during init
[   11.176539] ath10k_pci 0000:07:00.0: failed to wait for target init: -70
[   11.177684] ath10k_pci 0000:07:00.0: could not init core (-110)
[   11.177711] ath10k_pci 0000:07:00.0: could not probe fw (-110)

Additional context

No response

Solutions you've tried

No response

[Psotas]

Same for v1210 with Dasharo v0.9.2:

ubuntu@3mdeb:~$ sudo nmcli radio wifi on                                                              
ubuntu@3mdeb:~$ sudo nmcli device wifi rescan                                                         
Error: No Wi-Fi device found.                                                                         
ubuntu@3mdeb:~$ lspci | grep "Network controller:"                                                    
04:00.0 Network controller: Qualcomm Atheros QCA6174 802.11ac Wireless Network Adapter (rev 30)                                               
ubuntu@3mdeb:~$ sudo dmesg | grep ath                                                                 
[    2.788040] usbcore: registered new interface driver ath3k                                         
[    2.806149] ath10k_pci 0000:04:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0           
[    3.079365] ath10k_pci 0000:04:00.0: qca6174 hw3.0 target 0x05020000 chip_id 0x003409ff sub 168c:31
[    3.079375] ath10k_pci 0000:04:00.0: kconfig debug 0 debugfs 1 tracing 1 dfs 0 testmode 0          
[    3.079813] ath10k_pci 0000:04:00.0: firmware ver WLAN.RM.4.4.1-00157-QCARMSWPZ-1 api 6 features wb
[    3.153368] ath10k_pci 0000:04:00.0: board_file api 2 bmi_id N/A crc32 706c395e                    
[   10.522795] ath10k_pci 0000:04:00.0: wmi unified ready event not received                          
[   10.576917] ath10k_pci 0000:04:00.0: device has crashed during init                                
[   10.604941] ath10k_pci 0000:04:00.0: device has crashed during init                                
[   10.604950] ath10k_pci 0000:04:00.0: failed to wait for target init: -70                           
[   10.606727] ath10k_pci 0000:04:00.0: could not init core (-110)                                    
[   10.613151] ath10k_pci 0000:04:00.0: could not probe fw (-110)                                     
ubuntu@3mdeb:~$ 

[filipleple]

also occurs on VP4670, same binary:

ubuntu@3mdeb:~$ sudo dmesg | grep ath
[sudo] password for ubuntu: 
[    4.615047] usbcore: registered new interface driver ath3k
[    4.634742] ath10k_pci 0000:07:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[    4.917748] ath10k_pci 0000:07:00.0: qca6174 hw3.0 target 0x05020000 chip_id 0x003409ff sub 168c:3361
[    4.917753] ath10k_pci 0000:07:00.0: kconfig debug 0 debugfs 1 tracing 1 dfs 0 testmode 0
[    4.918089] ath10k_pci 0000:07:00.0: firmware ver WLAN.RM.4.4.1-00157-QCARMSWPZ-1 api 6 features wowlan,ignore-otp,mfp crc32 90eebefb
[    4.989288] ath10k_pci 0000:07:00.0: board_file api 2 bmi_id N/A crc32 706c395e
[   12.090718] ath10k_pci 0000:07:00.0: wmi unified ready event not received
[   12.140748] ath10k_pci 0000:07:00.0: device has crashed during init
[   12.168778] ath10k_pci 0000:07:00.0: device has crashed during init
[   12.168789] ath10k_pci 0000:07:00.0: failed to wait for target init: -70
[   12.171673] ath10k_pci 0000:07:00.0: could not init core (-110)
[   12.171722] ath10k_pci 0000:07:00.0: could not probe fw (-110)

[mkopec]

[    5.980960] ath10k_pci 0000:07:00.0: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[    5.980962] ath10k_pci 0000:07:00.0:   device [168c:003e] error status/mask=00000001/00006000
[    5.980965] ath10k_pci 0000:07:00.0:    [ 0] RxErr   

Could be that AER for that slot has to be disabled. I also saw similar issues on M900 Tiny, had to disable PCIe ASPM substates to get the wifi card to work: Dasharo/coreboot#402 (comment)

[miczyg1]

Okay, so buggy cards...

[macpijan]

Okay, so buggy cards...

@miczyg1 @mkopec Do you mean this specific unit to be buggy, or the card model / chip in general? Can replacing to another unit help?

I am afraid we need this card to work, it comes with the kit sold by Protectli: https://eu.protectli.com/product/m2-wifi/

Could be that AER for that slot has to be disabled. I also saw similar issues on M900 Tiny, had to disable PCIe ASPM substates to get the wifi card to work

@mkopec can you point to the specific change needed to test out this hypothesis? Ideally as PR, so we can use binaries from CI for testing.

[macpijan]

@filipleple

protectli_vault_cml_v1.1.0_vp46xx (coreboot 4.21 rebase)

Have you checked if it worked before rebase, so on the previous release?

[mkopec]

Do you mean this specific unit to be buggy, or the card model / chip in general? Can replacing to another unit help?

The card model most likely

an you point to the specific change needed to test out this hypothesis? Ideally as PR, so we can use binaries from CI for testing.

Dasharo/coreboot#469 CI should produce binaries for both changes, so they can be both tested

@macpijan

Although I think only Dasharo/coreboot@4f4bfb2 is actually needed

[macpijan]

Thanks, let's await tests results from @pkubaj

[pkubaj]

I first tried to compile myself with only one of the commits applied at the time - no change in behaviour.
Then I tried the CI-built image, also no change.

[pkubaj]

Downgrading the firmware as described in https://kb.protectli.com/kb/wifi-on-the-vault/ causes the Wifi card to work.

[macpijan]

At a very least we need to document it in dasharo docs to avoid in the future. Similar steps for other platforms have been documented like: https://docs.dasharo.com/unified/clevo/post-install/#linux

[miczyg1]

I would check if the FW is not broken: https://kb.protectli.com/kb/wifi-on-the-vault/
The page says that QCA6174-based WiFI cards need a firmware fix/workaround

[miczyg1]

There is also some thread related to the broken FW: https://lore.kernel.org/all/87zg7m8mt2.fsf@kernel.org/#t

[macpijan]

@pkubaj can you please document it as suggested here: #595 (comment) ? Then we can close this one.

[pkubaj]

Please see Dasharo/docs#782

[macpijan]

Thanks, @pkubaj left one comment there.