No Hotspot after Stretch install, detailed debug log with preliminary solution

[MiloMindbender]

Subject of the issue

Had a lot of problems getting Raspap runningon a Stretch distro I have been using for awhile. Decided to debug using a fresh install of Stretch. Think I have a solution but it points to some kind of timing issue which may be why so many people see this.

Your environment

• Pi 3 Model B
• Latest Stretch, fresh install, all updates applied
• RaspAP Quick Install
• Only the defaults for stretch.

Steps to reproduce

Install stretch on SD card, do all updates, set wifi to US. During install connected to my office WiFi AP as well as being on a wired ethernet. Did quick install of RaspAP

Expected behaviour

Should have an up-and-running hotspot on reboot.

Actual behaviour

On reboot the Raspap dashboard claims everything is ok but no AP is seen on iPhone or PC.
Starting and stopping the AP in Raspap causes AP to appear but it can't be joined.

DETAILED DEBUG LOG

1. Did clean install of stretch, performed all updates
2. During stretch setup, connected to my Office WiFi access point as well as a hardwired ethernet
3. Set wifi country code to US, shutdown, power off and boot
4. Did quick install of raspap
5. On reboot, Raspap web page says everything is running but no wifi network is showing up. Dashboard shows interface up on 10.3.141.1 and wireless shows connected to my local AP but PC and iPhone show no ssid.
6. Stop the AP and restart it using the Raspap control panel. Sometimes this makes the AP show up on my iPhone but I can't join the network (bad password) also my PC does not detect the network at all.
7. Apply the fix suggested in issue 272 rebooted
8. The AP is now visible on my PC and iPhone, the iPhone now joins but never fully connects and never gets an IP assigned. The PC appears to do the same thing. In the dashboard the IPv4 address says "not found"
9. Removed fix from issue 272 rebooted
10. System back to previous state. AP is visible to iPhone but I can't join
11. Stopped hotspot, manually did killall wpa_supplicant as suggested in issue 200, restarted hotspot. AP seems to be working now. iPhone is able to connect, put phone in airplaine mode with wifi active and was able to surf web. added the killall to the /etc/init.d/hostapd file and rebooted.
12. AP did NOT come up on reboot. Interface shows down, hostapd not running.
13. Try "start wlan0" on dashboard, says interface is up, press refresh, still says up, click to network page come back to dashboard, now says interface down.
14. Try "start hotspot" and hostapd does not start. Big difference from manually doing killall of wpa_supplicant after startup, suspect timing issue.
15. Put a "sleep 10" before and after killall in /etc/init.d/hostapd and reboot
16. AP now seems to come up and is working, able to surf web from phone. During startup I see the wifi icon in the upper right of the desktop briefly indicate a connection, then it switches back to the wired-only icon.

CONCLUSION
It appears that there is a timing issue during startup, possibly wpa_supplicant needs to come up for a brief period to initialize something, but this is not conclusive. It could easily be something else in the startup. Based on other issues I've read, the issues vary with type of PI which further suggests some kind of timing issue.

It appears that sometimes when you do a "refresh" on a Raspap config page, it doesn't show you the latest state of things (see #13 above), it just repeats the previously seen page.

FURTHER TESTS
It's late now, tomorrow I will try a few more things.

1. Try removing the killall, leaving only the sleep
2. Try only sleeping before killall
3. Try only sleeping after the killall
4. Try shorter sleep lengths
5. Will try the same SD card in a Raspberry Pi B+
6. Apply this fix to the SD card with my development config on it.

COMMENTS
What I REALLY wanted was the ability to run my PI as EITHER an AP or a client, but not both at the same time. I was really hoping there would be something in Raspap where I could push a few buttons to bring down the AP and bring up client mode. Doesn't seem like Raspap has this. Trying to work out a procedure for switching between modes but really wish you could add this.

I did read almost all the related issues on this subject. It seems like everyone's solution (and problems) were a little different and not every solution worked for everyone. Most had some element that would change the timing of things. When I was testing I also tried turning on the -dd debugging log of hostapd, even this drastically changed how things behaved, but it didn't completely fix the problem. This further points to a timing issue of some kind. It would make sense that differences in people's OS setup of services and startup tasks would cause timing differences that would result in different bugs (and different fixes).

I was having even weirder problems with the PI Stretch installation I'd been using for development, that's why I tried doing this with a clean install. Not sure why, maybe I messed something up trying fixes? Or maybe like I said, the timing of startup was different. But a lot of the problems persisted in the clean install test above.

[MiloMindbender]

Ok, performed the follow-up tests.

1. removing killall leaving only the sleep DOES NOT work
2. sleeping before killall but NOT after DOES NOT work
3. killall followed by sleep DOES work
4. Was able to reduce sleep to 1 second on a Pi B DOES work

Sometimes, with only the killall and no sleeps it will work but it is not reliable, sometimes works, sometimes hotspot comes up but can't be joined, sometimes no hotspot at all.

So the change that is working for me so far is to do killall and sleep 1 in /etc/init.d/hostapd, below is a fragment of the file to show where I put it.

case "$1" in
  start)
	log_daemon_msg "Starting $DESC" "$NAME"
        killall wpa_supplicant
        sleep 1 
	start-stop-daemon --start --oknodo --quiet --exec "$DAEMON_SBIN" \
		--pidfile "$PIDFILE" -- $DAEMON_OPTS >/dev/null
	log_end_msg "$?"
	;;
  stop)

Hope this is helpful, I'll test on some other PI configurations a bit later.

[1ubuntuuser]

@MiloMindbender Thanks for posting. Were you able to easily switch between AP mode and Client mode only using the GUI?

[MiloMindbender]

@danielwilson654 as near as I can tell there is no way to switch between AP and Client mode in the GUI. I think wpa_supplicant needs to be running to do client and it needs to be killed for AP to work. However I think if you just comment out the "killall" in my post above and reboot it will come up in client mode.

Also, I tested the fix above on a Rasperry Pi 3B+ and it worked there too.

[1ubuntuuser]

RIghtio... so it doesn't really work as the GUI suggests. So this is a bug or a new feature that's needed?

[MiloMindbender]

@danielwilson654 The Raspap package DOES work as the GUI suggests, because (at least as far as I can see) have a way to switch between AP and Client mode.

It would be pretty nice feature to have though.

The one thing that IS A BUG is the problem I detailed at the beginning of the issue about needing to kill wpa_supplicant. If they could fix this bug it would be a one-command-install which is great!

[billz]

@MiloMindbender thanks for taking the time to provide detailed debug info. For clarity and reproducibility, I'll detail my environment and the steps I used to setup my AP, leaving aside the WiFi client mode for the moment:

Environment

Model: Raspberry Pi 3B+
OS: Raspbian Stretch Lite (clean install on SD card)
Network: RPi connected via Ethernet cable

Initial setup

1. Update Raspbian, including kernel and firmware, followed by a reboot:

sudo apt-get update
sudo apt-get dist-upgrade
sudo reboot

2. Set Wi-Fi country in raspi-config's Localisation Options: sudo raspi-config
3. Run the RaspAP Quick Installer: wget -q https://git.io/voEUQ -O /tmp/raspap && bash /tmp/raspap
4. At the final prompt, allow Quick Installer to reboot the RPi.

The reported hostapd version in the above environment is v2.4.

Observed behavior

On reboot, the default SSID 'raspi-webgui' appears in the list of available WiFi networks on PC, iOS + Android devices. The raspi-webgui WiFi network is joined successfully with the default password. Confirm internet browsing via the AP works. In the default 802.11g mode, reported bandwidth is capped at 54 Mb/s.

Open the RaspAP administration interface in a browser. On the Configure hotspot > Basic tab, change the SSID and set Wireless Mode to 'n'. On the Security tab, set Encryption Type to 'CCMP'. Save settings and stop/start the hotspot from the UI. Result: the new SSID appears in clients with 802.11n mode, able to be joined with a PC and several handheld devices (iOS + Android). Verify that internet browsing works.

Given the above, RaspAP does indeed provide a single-command AP setup 🙂

There may indeed be differences with other RPi hardware and/or different kernels and package versions. Probably there's an unpleasant test grid with all the various permutations that I haven't fully addressed, although I've tried to make this project compatible with the most common hardware and the latest available firmware/packages, etc.

I'll take up the WiFi client switching in a separate issue. Again, thanks.

[cleanev]

Environment

Model: Raspberry Pi 3B+
OS: Raspbian Stretch Lite - new installation using etcher
Network: connected to local LAN by Ethernet
Uname: 4.14.98-v7+
Hostapd version: 2.6

Initial setup

Image Stretch on to new sad card using etcher
Use following commands as per readme.md
sudo apt-get update
sudo apt-get dist-upgrade
sudo reboot
Set Wi-Fi country to US
Run Installer per readme.md - wget -q https://git.io/voEUQ -O /tmp/raspap && bash /tmp/raspap
There are 4 times where I selected ‘y’ to continue
Final request to reboot - y

Observed behavior
AP does not get created

If I run sudo hostapd /etc/hostapd/hostapd.conf then I get AP started!

Please help to make this run on boot.

[billz]

Hostapd was just updated from v2.4 to 2.6. Probably there are some new changes that require testing.

[billz]

RaspAP starts the hostapd daemon via the service, not by excuting the binary directly. From the shell, try sudo systemctl start hostapd. If you see the message, “Failed to start hostapd.service: Unit hostapd.service is masked.”, unmask the service and try it again:

sudo systemctl unmask hostapd.service

[cleanev]

@billz
sudo systemctl start hostapd - error as “Failed to start hostapd.service: Unit hostapd.service is masked”
sudo systemctl unmask hostapd.service - there is no output
Nothing happens right away
sudo reboot
still no AP showing on laptop/iPhone/iPad
Run sudo systemctl start hostapd manually - AP is created and cannot be joined using ‘ChangeMe’ password. AP goes away after few minutes and comes on only after reboot and running the systemctl command manually.
No matter what cannot connect to AP with default password

[billz]

@clean-ev I've pushed a hotfix to the installer that resolves this

[cleanev]

@billz - Thanks for the update.
AP did not come up after using apt update/apt dist-upgrade.
I tried without any updates and lo and behold it comes up right away. Though I am not hardcore Linux admin, I do know much to be tinkering. So I started reading and wpasupplicant has had some issues that are making some things to break. I read more and found apt-mark command to put wpasupplicant on hold using this command

sudo apt-mark hold wpasupplicant
sudo apt dist-upgrade
sudo reboot

Now AP raspi-webui comes up the first time and every time I reboot.

I can open a new request for suggestion around channel numbers but will continue here for now instead of forgetting what I have done

Once logged in to AP GUI, I can see a lot of changes that can be exercised. The challenge is that if I use band ‘a’ then the channel gets stuck between 1-11 for US or 1-13 for SG. If I make a mistake by leaving it at channel 1with band ‘a’ then there is no AP.

Also when I change the dhcp scope to a new scope AP gets activated, but there is no internet connection. As usual in past few days read more and found that dhcpcd.conf file kept original settings of 10.3.141.1 while dnsmasq.conf will be showing new dhcp scope. Is this something that can be changed in your script to make changes to both dnsmasq and dhcpcd files?

I will continue to test as much as I can to provide with any errors or issues I encounter.

Thank you,

[MiloMindbender]

Sorry for the delay getting back, I just tried this with @billz approach, doesn't work. Did not get a hotspot after the quick installer or spending some time messing with (turning interfaces on and off...etc.)
Tried the unmask thing above, did not fix the problem
I do not get a hotspot at all....This is with

Raspberry pi 3b+
stretch FULL
pi connected to ethernet cable.

Not sure but I suspect part of the problem may be that when stretch comes up it searches for and tries to join a local WiFi?

Webgui works fine, suspect wpasupplicant is still interfering with hostapd starting up because after boot the raspap GUI says hostapd is running, but it is not.

[MiloMindbender]

By the way I tested on a blank SD card, fresh install.

Was thinking it might be stretch full vs stretch lite, but @clean-ev tested with lite and it didn't work either.

Only thing I can think of is it maybe not letting the stretch installer connect to wifi when you are installing? I'll give that a try if I have time.

[cleanev]

@clean-ev I've pushed a hotfix to the installer that resolves this

Hello - I can confirm that the new hotfix takes care of creating an AP at first reboot.

[lla4u]

Hello,

I played around with fresh stretch install and update then RaspAP.

No SSID visible after reboot.

wlan0 is missing IPv4:
pi@raspberrypi:/var/log $ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.10.21 netmask 255.255.255.0 broadcast 192.168.10.255
inet6 fe80::4a47:e874:410d:ee1 prefixlen 64 scopeid 0x20
ether b8:27:eb:a4:7b:79 txqueuelen 1000 (Ethernet)
RX packets 4927 bytes 499578 (487.8 KiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 3565 bytes 1323983 (1.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 1000 (Boucle locale)
RX packets 4 bytes 444 (444.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4 bytes 444 (444.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

wlan0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether b8:27:eb:f1:2e:2c txqueuelen 1000 (Ethernet)
RX packets 60 bytes 6732 (6.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 405 bytes 59093 (57.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

looks like hostapd running:
pi@raspberrypi:/var/log $ sudo systemctl status hostapd.service
● hostapd.service - Advanced IEEE 802.11 AP and IEEE 802.1X/WPA/WPA2/EAP Authenticator
Loaded: loaded (/lib/systemd/system/hostapd.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-03-26 18:16:44 CET; 10min ago
Process: 1336 ExecStart=/usr/sbin/hostapd -P /run/hostapd.pid -B $DAEMON_OPTS ${DAEMON_CONF} (code=exited, status=0/SUCCESS)
Main PID: 1338 (hostapd)
CGroup: /system.slice/hostapd.service
└─1338 /usr/sbin/hostapd -P /run/hostapd.pid -B /etc/hostapd/hostapd.conf

mars 26 18:16:44 raspberrypi systemd[1]: Starting Advanced IEEE 802.11 AP and IEEE 802.1X/WPA/WPA2/EAP Authenticator...
mars 26 18:16:44 raspberrypi hostapd[1336]: Configuration file: /etc/hostapd/hostapd.conf
mars 26 18:16:44 raspberrypi hostapd[1336]: Failed to create interface mon.wlan0: -95 (Operation not supported)
mars 26 18:16:44 raspberrypi hostapd[1336]: wlan0: interface state UNINITIALIZED->COUNTRY_UPDATE
mars 26 18:16:44 raspberrypi hostapd[1336]: wlan0: Could not connect to kernel driver
mars 26 18:16:44 raspberrypi hostapd[1336]: Using interface wlan0 with hwaddr b8:27:eb:f1:2e:2c and ssid "FBOX"
mars 26 18:16:44 raspberrypi hostapd[1336]: wlan0: interface state COUNTRY_UPDATE->ENABLED
mars 26 18:16:44 raspberrypi hostapd[1336]: wlan0: AP-ENABLED
mars 26 18:16:44 raspberrypi systemd[1]: Started Advanced IEEE 802.11 AP and IEEE 802.1X/WPA/WPA2/EAP Authenticator.

Stop/starting Hostapd from UI make SSID visible and i can connect but this fail after very few time.
wlan0 IP is set at this time and vanish

wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.3.141.1 netmask 255.255.255.0 broadcast 10.3.141.255
inet6 fe80::fabd:ad20:7394:effd prefixlen 64 scopeid 0x20
ether b8:27:eb:f1:2e:2c txqueuelen 1000 (Ethernet)
RX packets 1175 bytes 244903 (239.1 KiB)
RX errors 0 dropped 1 overruns 0 frame 0
TX packets 1478 bytes 809841 (790.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
pi@raspberrypi:/var/log $ sudo systemctl status hostapd.service
hostapd.service - Advanced IEEE 802.11 AP and IEEE 802.1X/WPA/WPA2/EAP Authenticator
Loaded: loaded (/lib/systemd/system/hostapd.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2019-03-26 18:28:01 CET; 3min 25s ago
Process: 1710 ExecStart=/usr/sbin/hostapd -P /run/hostapd.pid -B $DAEMON_OPTS ${DAEMON_CONF} (code=exited, status=0/SUCCESS)
Main PID: 1712 (hostapd)
CGroup: /system.slice/hostapd.service
└─1712 /usr/sbin/hostapd -P /run/hostapd.pid -B /etc/hostapd/hostapd.conf

mars 26 18:28:00 raspberrypi hostapd[1710]: Using interface wlan0 with hwaddr b8:27:eb:f1:2e:2c and ssid "FBOX"
mars 26 18:28:01 raspberrypi hostapd[1710]: wlan0: interface state COUNTRY_UPDATE->ENABLED
mars 26 18:28:01 raspberrypi hostapd[1710]: wlan0: AP-ENABLED
mars 26 18:28:01 raspberrypi systemd[1]: Started Advanced IEEE 802.11 AP and IEEE 802.1X/WPA/WPA2/EAP Authenticator.
mars 26 18:28:07 raspberrypi hostapd[1712]: wlan0: STA a0:99:9b:0f:0d:b9 IEEE 802.11: associated
mars 26 18:28:07 raspberrypi hostapd[1712]: wlan0: STA a0:99:9b:0f:0d:b9 RADIUS: starting accounting session D0AE6441C30E3C48
mars 26 18:28:07 raspberrypi hostapd[1712]: wlan0: STA a0:99:9b:0f:0d:b9 WPA: pairwise key handshake completed (WPA)
mars 26 18:28:07 raspberrypi hostapd[1712]: wlan0: STA a0:99:9b:0f:0d:b9 WPA: group key handshake completed (WPA)
mars 26 18:30:03 raspberrypi hostapd[1712]: wlan0: STA a0:99:9b:0f:0d:b9 IEEE 802.11: disassociated
mars 26 18:30:03 raspberrypi hostapd[1712]: wlan0: STA e1:01:e1:01:e0:01 IEEE 802.11: disassociated

Any idea for fixing?
Laurent

[BoLtWD]

I am sure this bug is also affecting me also, I have a RaspAp set up using a Wifi USB RTL8192cu Chipset and Stretch.
I can see a Wifi in stretch and in RaspAp after changing the settings for the hotspot and start hotspot I can see the Wifi also in stretch the Wifi symbol has changed to Lan. At this point the hotspot is working untill I restart the PI. After restart I cannot see the hotspot Wifi, in Stretch the Wifi symbol is back on and in RaspAp under dashboard and hotspot all reported as running but no Wifi hotspot can be found again.
Looks like some kind of conflict...

[billz]

@MiloMindbender if you're feeling adventurous, try out the wificlient-ap branch. I posted instructions here. This branch includes a new service start script that brings up network services in a specific order and timing. Alternatively, you could also just download the file to your RPi, make it executable and run it directly:

wget https://raw.githubusercontent.com/billz/raspap-webgui/wificlient-ap/installers/servicestart.sh
sudo chmod +x servicestart.sh
./servicestart.sh --seconds 5

I'm not aware of conflicts Raspbian desktop might create with wpa_supplicant. Anecdotally, several users have demonstrated setting up this project in a Stretch desktop environment without issues. Having said that, I do think this project is better suited for headless RPi servers.

[billz]

Resolved with #300