[WiFi] Fix issue where WiFi could not scan to reconnect

[TD-er]

As reported by a number of users since the latest mega-20210503 build.

• Reconnect after reboot may take forever
• Reconnect when WiFi connection lost may take forever.
• Cannot perform WiFi scan

[ghtester]

With the latest Vagrant custom build I have encountered a reconnect issue - one node after firmware upgrade could not reconnect and it was necessary to power off / on, then it conencted quickly.
Another node with the same firmware did not reconnect after crash due to exception, WiFi status LED is quickly flashing... :-(

Build:⋄	20114 - Mega
System Libraries:⋄	ESP82xx Core 2843a5ac, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2 PUYA support
Git Build:⋄	My Build: Jul 4 2021 08:52:34
Plugin Count:⋄	35
Build Origin:	Vagrant
Build Time:⋄	Jul 4 2021 08:51:36
Binary Filename:⋄	ESP_Easy_mega_20210704_custom_IR_ESP8266_4M1M
Build Platform:⋄	Linux-4.15.0-51-generic-x86_64-with-glibc2.27
Git HEAD:⋄	mega_74901b1

I have just found an option "Use Last Connected AP from RTC:".
Checked and will test...

[TD-er]

That RTC option used to be the default option, to speed-up reconnects.
But it did result in several issues for some, so I added the selection for it and by default disabled it.
So checking it is the default as it was before.

[TD-er]

Another node with the same firmware did not reconnect after crash due to exception, WiFi status LED is quickly flashing... :-(

Can you connect it to the serial to get some idea of why it is crashing?

[ghtester]

Unfortunately not, both nodes have no serial port attached.
I suppose it's a combination of bad WiFi reception + MQTT Controller used. Both nodes have a backup AP configured, also with a weak signal.
Currently one of nodes is trying to reconnect for a long time, WiFi status LED just quickly flashing (the opposite node than before, which I had to turn off/on to reconnect).

BTW. the primary AP has BSSID broadcast disabled. When I enabled it (the AP rebooted then), the node trying to reconnect unsuccesfully (even after power off/on) finally connected (don't know if it was due to AP reboot or BSSID broadcast change). Both nodes of course have enabled the option "Include Hidden SSID:"

[TD-er]

Are those APs set to only allow 802.11n ?
There is a problem when the ESP has started its AP as it cannot connect to known APs as it cannot connect to "N-only" APs if its own AP is also active.

Can you set the number of WiFi scans to 2?

[ghtester]

No, the Primary AP is b/g, the Backup AP is in mixed mode 802.11 b,g,n.
It would be really nice to have an option to permanently disable AP mode on ESP Easy. I have asked for this a couple years ago but I am perhaps only one who would appreciate it.
What I need when the WiFi connectivity is lost is to quickly reconnect to the same AP (Primary or Backup) and if not possible, switch to opposite (Backup or Primary) and so on.
Currently both nodes are connected properly to Primary AP, with disabled BSSID broadcast.
Do you mean to change "Extra WiFi scan loops" option which is currently 0?

[TD-er]

There is a checkbox for it to disable AP mode.

Do you mean to change "Extra WiFi scan loops" option which is currently 0?

Yep, this does perform multiple scans increasing the chance it will find the best AP.
And if you need to reconnect to the last used one as first retry, check this new RTC option mentioned before.

[ghtester]

There is a checkbox for it to disable AP mode.

Ahh, really, "Do Not Start AP:"
I am sorry I have overlooked it, don't know when it appeared there... :-) Thanks a lot!
I'll give it a try, going to change it to 2.
RTC option already checked, hopefully it works also for hidden APs... We'll see when a next crash happen...

[TD-er]

Yep it does also help on hidden SSIDs.
What it does is:

• Scan for APs
• Filter out known SSIDs and BSSID of hidden SSIDs and store those in a list of candidates
• Sort list of candidates based on RSSI + known first, hidden next. (as can be seen in the WiFi scan page)
• Try to connect to each candidate in sorted order. (hidden ssid's will be tried with all combinations of known passwords)
• If a candidate fails to connect (or connection is lost), it is removed from the list.
• If list of candidates is empty, new scan is performed

As an extra, if a connection is considered stable, the list of candidates is cleared and the active one is left as the only one in the list.
Reconnect from RTC does make sure that after a reboot/crash (without power loss) the last stable connection is added to the list (which will then be the only one in the list)

[ghtester]

Thanks for the explanation, although I don't fully understand this complexity (from my perspective), I believe it's the best alghoritm, except trying hidden SSID's with all password combinations. Is it really necessary? I thought connecting hidden AP is the same process as with visible APs.
I think in most cases the simplified process (for instance try to connect Primary AP twice, then Backup AP once and repeat) would work much better, especially when only 1 AP is configured and Do Not Start AP option is enabled.

[TD-er]

For a hidden SSID you only know the BSSID and channel.
So you don't know anything else.

If I try to use the standard WiFi.begin function, I have no guarantee which AP I will connect to.
It can be the worst one (which actually did happen quite often).
As a matter of fact, it did happen so often I started writing this procedure as it kept popping up in all kinds of issues.

[ghtester]

OK, thanks again!
It's interesting that both nodes crashed soon after firmware upgrade and no crash encountered afterwards (uptime 18,5 and 23 hours).

[TD-er]

"soon" or "immediately" ?

[ghtester]

Not immediately but as far as I can remember, in several minutes, then crash repeated, when the nodes could not reconnect, I turned them off and on. Still experiencing a connectivity issue so I enabled the BSSID broadcast on AP and after succesfull connection disabled it. So currently the nodes are after cold boot and connected to hidden primary AP.

[TD-er]

OK, will perform some tests here with hidden SSID to see how it behaves.
Still ESP8266?

[ghtester]

Yes, ESP8266. Please note in my case the AP signal is quite low which can also make troubles, hardly reproducible when the signal is strong (then ESP nodes are quite stable). Currently I have RSSI -91 dBm and -87dBm and both ESP nodes are working fine, the web GUI is very quick.

[ghtester]

So far one node (with more plugins) crashed twice, I suppose it was due to exhausted RAM for some reason:

Uptime:	0 days 9 hours 55 minutes
Load:	21.23% (LC=894)
CPU Eco Mode:	false
Boot:	Exception (2)
Reset Reason:	Exception
Last Action before Reboot:	3 timer, id: 7
SW WD count:	0
Memory
Free RAM:	5064
Heap Max Free Block:	872
Heap Fragmentation:	71%
Free Stack:	3664

Second node is still running without crash:

Uptime:	2 days 10 hours 53 minutes
Load:	9.61% (LC=1696)
CPU Eco Mode:	false
Boot:	Cold Boot (0)
Reset Reason:	External System
Last Action before Reboot:	Background Task
SW WD count:	0
Memory
Free RAM:	7024
Heap Max Free Block:	4560
Heap Fragmentation:	34%
Free Stack:	3664

[TD-er]

Can you test that specific node with a build from this PR: #3680
N.B. you must then run the "beta" build and only build the beta build.
The reason it was not yet merged is that it sometimes resulted in a build which crashes immediately, but cleaning it and rebuilding is running stable.
It will give you some headroom in memory usage.

[ghtester]

I am not sure. I can build from PR but in general I need the 'custom_IR_ESP8266_4M1M' binary which I have in the pio_envlist.txt file. So the question is how to specify the beta build exactly (perhaps put custom_beta_ESP8266_4M1M
there instead?) and if the all plugins I have specified in the 'Custom.h' file will be there (including IRRX and IRTX).

Edit - tried to compile with 'custom_beta_ESP8266_4M1M' in pio_envlist.txt but it did not create the .bin file:
default: Compiling .pio/build/custom_beta_ESP8266_4M1M/src/ESPEasy.ino.cpp.o
default: /home/vagrant/GitHub/letscontrolit/ESPEasy/src/ESPEasy.ino:18:5: warning: #warning "**** Using Settings from Custom.h File " [-Wcpp]
default: 18 | # warning "* Using Settings from Custom.h File ***"
default: | ^~~~~~~
default: /home/vagrant/GitHub/letscontrolit/ESPEasy/src/_P016_IR.ino:22:10: fatal error: IRremoteESP8266.h: No such file or directory
default:
default: *************************************************************************
default: * Looking for IRremoteESP8266.h dependency? Check our library registry!
default: *
default: * CLI > platformio lib search "header:IRremoteESP8266.h"
default: * Web > https://platformio.org/lib/search?query=header:IRremoteESP8266.h
default: *
default: *************************************************************************
default: 22 | #include <IRremoteESP8266.h>
default: | ^~~~~~~~~~~~~~~~~~~
default: compilation terminated.
default: *** [.pio/build/custom_beta_ESP8266_4M1M/src/ESPEasy.ino.cpp.o] Error 1
default: ========================= [FAILED] Took 267.30 seconds =========================
default: Environment Status Duration
default: ------------------------ -------- ------------
default: custom_beta_ESP8266_4M1M FAILED 00:04:27.300
default: ==================== 1 failed, 0 succeeded in 00:04:27.300 ====================
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument

[TD-er]

I was first trying to describe what you needed to change and then I realized it is way easier to just add another PIO env: custom_beta_IR_ESP8266_4M1M
So if you fetch the PR #3680 you will have this extra build env.

[ghtester]

Thank you, I did not find the proper string 'custom_beta_IR_ESP8266_4M1M' anywhere. It would be great if the pio_envlist.txt.sample file could contain all valid build names.

Compiled and one node upgraded, I'll put a feedback here couple days later.
BTW. after upgrade & cold boot the ESP node did not connect to hidden AP (even though the 'Include Hidden SSID:' was enabled there). After enabling the BSSID broadcast on AP the node connected. Then I enabled the option 'Do Not Start AP:', saved config by Submit button, disabled the BSSID broadcast on AP again and ESP node connected properly then. After another cold boot the WiFi status LED is flashing quickly and node does not connect. Strange... Maybe it's because of hidden AP is configured as Fallback SSID and the primary SSID is inaccessible?

[TD-er]

Nope, I think it has to do with the "broadcast SSID".
I guess I really need to store the BSSID also in the settings or else you keep getting these issues.
And then the connection procedure also needs to have some kind of fallback to connect as it does now if connecting fails.
Only problem remains.... how to detect the correct channel as I do need to give a channel and BSSID or none of them.
Maybe it also works with channel set to "0", but that needs to be tested.

[TD-er]

By the way, the unit now does show a "2nd heap" in the system overview?

[ghtester]

The node crashed again after 114 minutes and does not reconnect to hidden AP. So unfortunately it's less stable than earlier firmware in my configuraton.

[ghtester]

The current status (the node is now reconnecting to hidden AP succesfully):

Uptime:	0 days 0 hours 42 minutes
Load:	41.10% (LC=281)
CPU Eco Mode:	false
Boot:	External Watchdog (13)
Reset Reason:	Hardware Watchdog
Last Action before Reboot:	Background Task
SW WD count:	0
Memory
Free RAM:	5976
Heap Max Free Block:	4584
Heap Fragmentation:	23%
Free 2nd Heap:	16120
Free Stack:	3632

The second upgraded node with only DS18b20 and TSOP4838 plugins and MQTT controller:

Uptime:	0 days 19 hours 20 minutes
Load:	19.45% (LC=559)
CPU Eco Mode:	false
Boot:	Cold Boot (0)
Reset Reason:	External System
Last Action before Reboot:	Background Task
SW WD count:	0
Memory
Free RAM:	5680
Heap Max Free Block:	4064
Heap Fragmentation:	28%
Free 2nd Heap:	16080
Free Stack:	3632

[TD-er]

Hmm watchdog timer is still happening so it seems.
That's probably related to a connection issue in the HTTP controller as that one needs to make a connection every time.
The MQTT controllers are much more reliable.

[ghtester]

Yeah it could be the reason... Even with previous firmware release this node sometimes crashed when I rebooted AP.
I am using the HTTP Controller to send 2 fields at the same time to ThingSpeak server (which is not supported by ThingSpeak Controller) by rule. I am also sending the data to another server with HTTP Controller and it's not easy to change that.

But it looks in the beta firmware the 2nd Heap is not used in fact.

[TD-er]

The 2nd heap is mainly used for 'temporary' stuff.
So if the queue can be flushed in due time, it will not stay there.

What you can do to see it is working, is sending messages to some controller with 1 sec interval and set the controller to only allow to send messages with a minimum interval > 1 sec.
This does cause the queue to be filled.
You can also set it to a longer queue to really see the difference.

[ghtester]

OK, I understand. I have minimized the queues to save memory in past so I can try to increase them again.
But what it's not clear to me why the HTTP controller needs a connection every time otherwise may crash? There are the same queue and timeout parameters configurable as in other Controllers so I would expect that temporary disconnection should not be so critical?

[TD-er]

An HTTP connection is not kept open.
So with SendToHTTP and any HTTP controller, each time a message is sent, it will create a new connection to the server and close it.

The reason why this may lead to crashes is not 100% clear to me, but it seems like the set timeout is not used in the core libraries when either doing a DNS resolve (if needed) or during the connection phase.
So then it is a blocking call which may take longer than the watchdog allows.

[ghtester]

Well, after last crash the node does not reconnect to hiden WiFi AP again. This is the worst case which can happen and this is unacceptable for me so I'll have to downgrade this node to earlier FW...
The reconnect issue was confirmed on another 'beta' node, after AP reboot it also did not reconnect. It looks it's a bit random issue.

After downgrade to custom FW built on 20210531 there's no reconnect issue and it's reconnecting much more quickly and reliable.

[TD-er]

OK, I found some issue related to WiFi reconnect.
Can you comment out this code:

ESPEasy/src/src/ESPEasyCore/ESPEasyWifi.cpp

Lines 876 to 896 in 2c6a742

	{
	static bool FIRST_SCAN = true;
	struct scan_config config;
	memset(&config, 0, sizeof(config));
	config.ssid = nullptr;
	config.bssid = nullptr;
	config.channel = channel;
	config.show_hidden = 1;
	config.scan_type = WIFI_SCAN_TYPE_ACTIVE;
	if (FIRST_SCAN) {
	config.scan_time.active.min = 100;
	config.scan_time.active.max = 200;
	} else {
	config.scan_time.active.min = 400;
	config.scan_time.active.max = 500;
	}
	FIRST_SCAN = false;
	wifi_station_scan(&config, &onWiFiScanDone);
	}

[TD-er]

N.B. I made a new PR for it: #3702

This is not yet included in the "core 3.0.0" PR, but just to fix the WiFi connection issues since they are not specific to core 3.0.0.

So you can also try to build one for that PR to test if it solves your connection issues.

[ghtester]

Thanks a lot but as there's a stability issue as well, not just a WiFi reconnect, I am not going to upgrade the primary ESP node yet.
I can compile PR3702 and upgrade the another ESP node (currently running the beta firmware, after one crash due to Hardware Watchdog).
So if I understood well, I should compile 'custom_IR_ESP8266_4M1M' (not beta) with PR 3702. I can try.

[TD-er]

Yep, there is no "beta" for that custom build ... I think... outside the "core 3.0.0" PR.
I have seen some very strange situations where I could not reconnect to WiFi here, so I truly hope you also now can reconnect to WiFi without issues with this fix.

[ghtester]

Thanks but unfortunately the compilation stopped with error:
...
...
Compiling .pio/build/custom_IR_ESP8266_4M1M/src/src/ESPEasyCore/ESPEasyWifi_ProcessEvent.cpp.o
src/src/ESPEasyCore/ESPEasyWifi_ProcessEvent.cpp: In function 'void processScanDone()':
src/src/ESPEasyCore/ESPEasyWifi_ProcessEvent.cpp:536:26: error: 'struct WiFi_AP_CandidatesList' has no member named 'addedKnownCandidate'
if (WiFi_AP_Candidates.addedKnownCandidate()) {
^
*** [.pio/build/custom_IR_ESP8266_4M1M/src/src/ESPEasyCore/ESPEasyWifi_ProcessEvent.cpp.o] Error 1
=================================================================================== [FAILED] Took 361.50 seconds ===================================================================================

Environment Status Duration

---

custom_IR_ESP8266_4M1M FAILED 00:06:01.495
============================================================================== 1 failed, 0 succeeded in 00:06:01.495 ==============================================================================

[TD-er]

Ah, yep, that's my fault as I developed the commits first on another branch and then cherry picked it to this PR branch.

Will fix it.

[TD-er]

Done...

[ghtester]

A great job, thank you! I'll give it a try again.
Edit - compiled and upgraded ESP node (from beta, this is the node with only DS18b20 and TSOP4838 plugins and MQTT controller). WiFi connected succesfully to primary SSID (with weak signal) but then I couldn't open ESP node web GUI. After entering username & password the WiFi status LED started shining permanently strong and the webpage did not open - just a blank page, perhaps waiting for data. The same behavior after cold boot, I was waiting quite long but maybe not so long to discover if the node recovers from the stalled state or not.
After another cold boot the ESP node connected to Fallback SSID with strong signal and the web GUI opened without issue. I'll keep testing of WiFi connectivity & stability.

Uptime:	0 days 0 hours 13 minutes
Load:	7.78% (LC=1570)
CPU Eco Mode:	false
Boot:	Cold Boot (0)
Reset Reason:	External System
Last Action before Reboot:	Background Task
SW WD count:	0
Memory
Free RAM:	8536
Heap Max Free Block:	5144
Heap Fragmentation:	38%
Free Stack:	3664

Firmware

Build:⋄ | 20114 - Mega
System Libraries:⋄ | ESP82xx Core 2843a5ac, NONOS SDK 2.2.2-dev(38a443e), LWIP: 2.1.2 PUYA support
Git Build:⋄ | My Build: Jul 12 2021 23:46:20
Plugin Count:⋄ | 35
Build Origin: | Self built
Build Time:⋄ | Jul 12 2021 23:45:21
Binary Filename:⋄ | ESP_Easy_mega_20210712_custom_IR_ESP8266_4M1M
Build Platform:⋄ | Linux-4.15.0-51-generic-x86_64-with-glibc2.27
Git HEAD:⋄ | HEAD_aa05b84

[ghtester]

BTW. Today upgraded another ESP node with the firmware build created 3 days ago with PR 3702. So far this is the status:

Uptime:	0 days 4 hours 21 minutes
Load:	9.31% (LC=1591)
CPU Eco Mode:	false
Boot:	Exception (3)
Reset Reason:	Exception
Last Action before Reboot:	Background Task
SW WD count:	0

The good news is that the node is reconnecting OK, unfortunately it's not so stable as needed... The configuration is a bit different, plugins & controllers used:  

[ghtester]

FYI yesterday in late evening I have recompiled the custom firmware again from latest sources where PR3702 is already merged. Upgraded 3 ESP nodes, it's early (uptime 9 - 10 hours) but so far no crash and no WiFi connection issue.

[TD-er]

Great!
Sounds almost like we can make another nightly again, which has been way too long since the last one.

[ghtester]

Well, the primary node crashed but at least it reconnected succesfully (it took some time - about 4 minutes - but the WiFi signal level is quite low there).
It looks the crash was just after the WiFi connection (and therefore MQTT connection) was temporarily lost and reconnected.

Uptime:	0 days 0 hours 7 minutes
Load:	19.82% (LC=920)
CPU Eco Mode:	false
Boot:	Exception (1)
Reset Reason:	Exception
Last Action before Reboot:	3 timer, id: 7
SW WD count:	0
Memory
Free RAM:	4184
Heap Max Free Block:	1376
Heap Fragmentation:	56%
Free Stack:	3664

[TD-er]

That setup should benefit from the 2nd heap in the core 3.0.0 PR.

You really have low free memory.

[TD-er]

I just looked through your posted config.
Do you actually use the Homie controller?
I know that controller really needs to be taken care of as it is a great collection of copy/paste code with lots of things that can be improved.

[ghtester]

Yeah I'll try to compile the beta firmware again a couple days later and give it a try again. The current FW needs to be tested on another nodes as well but I see an improvement.
This node is connected to OpenHAB where the MQTT broker (external, from OS) is running.
Another nodes are connected to Home Assistant's MQTT (which would be a preferred config).

[ghtester]

Just tried to compile the beta firmware but it did not work:
default: ================
default: Library Manager
default: ===============
default: Library Storage: /home/vagrant/.platformio/lib
default: grep:
default: /home/vagrant/.platformio/packages/framework-arduinoespressif8266//cores/esp8266/Esp.cpp
default: : No such file or directory
default: Patching /home/vagrant/.platformio/packages/framework-arduinoespressif8266/
default: patch: **** Can't change to directory /home/vagrant/.platformio/packages/framework-arduinoespressif8266/
default: : No such file or directory
default: grep:
default: /home/vagrant/.platformio/packages/framework-arduinoespressif8266*//cores/esp8266/Esp.cpp
default: : No such file or directory
default: Patching /home/vagrant/.platformio/packages/framework-arduinoespressif8266*/
default: patch: **** Can't change to directory '/home/vagrant/.platformio/packages/framework-arduinoespressif8266*/'
default: : No such file or directory
default: Copy /vagrant/Custom.h > /home/vagrant/GitHub/letscontrolit/ESPEasy/src/Custom.h
default: Error: Unknown environment names 'custom_beta_IR_ESP8266_4M1M'. Valid names are 'custom_ESP8266_1M, custom_beta_ESP8266_4M1M, test_C_ESP32_4M316k_lolin_d32_pro, normal_alt_wifi_ESP8266_1M, max_ESP32_16M8M_LittleFS, test_A_ESP32_4M316k, test_C_beta_ESP8266_16M_LittleFS, minimal_core_274_sdk3_ESP8285_1M_OTA_Domoticz, custom_alt_wifi_ESP8266_1M, test_D_ESP8266_4M1M, max_ESP32_16M2M_LittleFS, hard_LCtech_relay_x2_1M, normal_WROOM02_2M256, test_D_beta_ESP8266_4M1M, test_D_beta_ESP8266_16M_LittleFS, test_C_ESP32-wrover-kit_4M316k, custom_alt_wifi_ESP8266_4M1M, custom_IR_ESP8266_4M1M, normal_alt_wifi_ESP8266_1M_VCC, test_B_ESP32-wrover-kit_4M316k_ETH, max_ESP32_16M1M, test_C_ESP32_4M316k, normal_ESP8266_1M, test_B_ESP8266_4M1M_VCC, spec_memanalyze_ESP8266, hard_Ventus_W266, normal_ESP8285_1M, minimal_core_274_sdk3_ESP8266_1M_OTA_Domoticz, energy_ESP32_4M316k, minimal_core_274_ESP8285_1M_OTA_Domoticz, test_B_ESP32_4M316k_ETH, test_C_alt_wifi_ESP8266_4M1M_VCC, test_D_ESP32-wrover-kit_4M316k_ETH, hard_Shelly_1_2M256, test_A_ESP32_4M316k_ETH, spec_debug_custom_ESP32_4M316k, test_A_ESP8266_4M1M_VCC, test_A_beta_ESP8266_16M_LittleFS, spec_debug_beta_custom_ESP8266_4M1M, normal_ESP8266_4M1M, normal_ESP8266_4M1M_VCC, hard_other_POW_ESP8285_1M, spec_debug_custom_ESP8266_4M1M, test_A_ESP32-wrover-kit_4M316k_ETH, custom_sdk3_ESP8266_4M1M, minimal_core_274_ESP8266_1M_OTA_FHEM_HA, normal_beta_ESP8266_16M_LittleFS, test_C_ESP8266_4M1M_VCC, energy_ESP8266_4M1M, normal_alt_wifi_ESP8266_4M1M, max_ESP32_16M2M_LittleFS_ETH, test_A_ESP32-wrover-kit_4M316k, test_D_ESP32_IRExt_4M316k, test_A_beta_ESP8266_4M1M, custom_IR_ESP32_4M316k, minimal_core_274_sdk3_ESP8266_1M_OTA_FHEM_HA, max_ESP32_16M8M_LittleFS_ETH, test_A_alt_wifi_ESP8266_4M1M_VCC, minimal_core_274_sdk3_ESP8285_1M_OTA_FHEM_HA, test_B_alt_wifi_ESP8266_4M1M_VCC, minimal_core_274_ESP8285_1M_OTA_FHEM_HA, test_D_ESP32_4M316k, test_A_ESP32_4M316k_lolin_d32_pro, test_D_ESP32_4M316k_ETH, display_ESP8266_4M1M, custom_ESP8266_4M2M, custom_ESP8266_4M1M, test_B_ESP32_IRExt_4M316k, normal_sdk3_ESP8266_1M, normal_WROOM02_2M, test_D_ESP32-wrover-kit_4M316k, test_B_beta_ESP8266_16M_LittleFS, test_C_beta_ESP8266_4M1M, test_B_ESP32_4M316k, test_B_ESP32_4M316k_lolin_d32_pro, test_C_ESP32-wrover-kit_4M316k_ETH, custom_beta_ESP8266_1M, minimal_IRext_ESP8266_1M, test_C_ESP32_IRExt_4M316k, test_D_alt_wifi_ESP8266_4M1M_VCC, normal_ESP32_4M316k_ETH, test_C_ESP32_4M316k_ETH, minimal_IRext_ESP8266_4M2M, test_B_ESP32-wrover-kit_4M316k, normal_ESP8266_1M_VCC, display_ESP32_4M316k, test_C_ESP8266_4M1M, custom_ESP32_4M316k_ETH, normal_IRext_no_rx_ESP8266_4M2M, test_A_ESP8266_4M1M, test_D_ESP32_4M316k_lolin_d32_pro, test_B_beta_ESP8266_4M1M, custom_ESP32_4M316k, test_A_ESP32_IRExt_4M316k, spec_debug_custom_IR_ESP8266_4M1M, custom_ESP8266_4M2M_LittleFS, minimal_IRext_ESP8266_4M1M, test_D_ESP8266_4M1M_VCC, max_ESP32_16M1M_ETH, normal_ESP32_4M316k, hard_SONOFF_POW_4M1M, minimal_core_274_ESP8266_1M_OTA_Domoticz, test_B_ESP8266_4M1M, hard_Shelly_PLUG_S_2M256'
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Created /tmp/tmp.zAeeLuITgA
default: Created /tmp/tmp.zAeeLuITgA/ESPEasy_collect_dist
default: ### Creating zip archives
default:
default: zip error: Nothing to do! (/home/vagrant/GitHub/letscontrolit/ESPEasy/ESPEasy_ELF_files_20210720_vagrant.zip)
default: ### Created ESPEasy_ELF_files_20210720_vagrant.zip
default: cp:
default: cannot stat 'build_output/bin/*': No such file or directory
default: ### Created ESPEasy_ESP82xx_20210720_vagrant.zip
default: cp:
default: cannot stat 'build_output/bin/ESP32'
default: : No such file or directory
root@Ubuntu-rk:/ESP_Easy/ESPEasy/tools/vagrant#
root@Ubuntu-rk:/ESP_Easy/ESPEasy/tools/vagrant# find / -name ESPEasy_ELF_files_20210720_vagrant.zip
find: ‘/run/user/1000/gvfs’: Permission denied
root@Ubuntu-rk:/ESP_Easy/ESPEasy/tools/vagrant# cd /ESP_Easy/

[TD-er]

The custom_beta_IR_ESP8266_4M1M you try to build is not known in the current mega branch, only in the core 3.0.0 PR.
I can add it to the mega branch to ease testing for you and also since you plan on using it ;)

[ghtester]

Unfortunately now even custom_IR_ESP8266_4M1M can't be compiled:
default: Compiling .pio/build/custom_IR_ESP8266_4M1M/src/src/PluginStructs_tmp/__tmpfile.cpp.o
default: Compiling .pio/build/custom_IR_ESP8266_4M1M/src/src/Static/WebStaticData.cpp.o
default: Compiling .pio/build/custom_IR_ESP8266_4M1M/src/src/WebServer_tmp/__tmpfile.cpp.o
default: In file included from src/src/WebServer_tmp/__tmpfile.cpp:196:0:
default: src/src/WebServer_tmp/../WebServer/AdvancedConfigPage.h:16:100: error: 'ExtTimeSource_e' has not been declared
default: void addFormExtTimeSourceSelect(const __FlashStringHelper * label, const __FlashStringHelper * id, ExtTimeSource_e choice);
default: ^
default: src/src/WebServer_tmp/__tmpfile.cpp: In function 'void handle_advanced()':
default: src/src/WebServer_tmp/__tmpfile.cpp:343:101: error: cannot convert 'ExtTimeSource_e' to 'int' for argument '3' to 'void addFormExtTimeSourceSelect(const __FlashStringHelper*, const __FlashStringHelper*, int)'
default: addFormExtTimeSourceSelect(F("External Time Source"), F("exttimesource"), Settings.ExtTimeSource());
default: ^
default: *** [.pio/build/custom_IR_ESP8266_4M1M/src/src/WebServer_tmp/__tmpfile.cpp.o] Error 1
default: ========================= [FAILED] Took 583.70 seconds =========================
default: Environment Status Duration
default: ---------------------- -------- ------------
default: custom_IR_ESP8266_4M1M FAILED 00:09:43.702
default: ==================== 1 failed, 0 succeeded in 00:09:43.702 ====================
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Error: -e option requires an argument
default: Created /tmp/tmp.JAIpx8Ad0z
default: Created /tmp/tmp.JAIpx8Ad0z/ESPEasy_collect_dist
default: ### Creating zip archives
default:
default: zip error: Nothing to do! (/home/vagrant/GitHub/letscontrolit/ESPEasy/ESPEasy_ELF_files_20210722_vagrant.zip)
default: ### Created ESPEasy_ELF_files_20210722_vagrant.zip
default: cp:
default: cannot stat 'build_output/bin/*'
default: : No such file or directory
default: ### Created ESPEasy_ESP82xx_20210722_vagrant.zip
default: cp:
default: cannot stat 'build_output/bin/ESP32'
default: : No such file or directory
root@Ubuntu-rk:/ESP_Easy/ESPEasy/tools/vagrant#

[TD-er]

Ah looks like an issue introduced by the last PR I merged.

[ghtester]

No problem, please just let me know when fixed so I can try compiling again. It's not urgent / important at all.

[TD-er]

OK, can you try again to build it?

[ghtester]

Yes, compiled without issue and updated 2 nodes. Thanks a lot.