Replacing the main application fails

[spuke113]

Hello @hn,
I'm trying to flash esphome image to the Solis S3 WiFi Data Logging Stick (3rd gen). I followed your instructions, but the very last step fails with just a generic error:

user@linux-pc:~/rtltool/libretiny-esphome$ python3 -m esphome upload solis-inv-esphome.yaml --device /dev/ttyUSB0
INFO Reading configuration solis-inv-esphome.yaml...
Processing solis-inv (board: generic-rtl8710bx-4mb-980k; framework: arduino; platform: libretiny)

Configuring upload protocol...
AVAILABLE: uart
CURRENT: upload_protocol = uart
Looking for upload port...
Using manually specified: /dev/ttyUSB0
Uploading .pioenvs/solis-inv/firmware.uf2
|-- Detected file type: UF2 - esphome 2023.5.0-dev
|-- Connecting to 'Realtek AmebaZ' on /dev/ttyUSB0 @ 1500000
|   |-- Success! Chip info: Realtek RTL87xxB
|-- Writing '.pioenvs/solis-inv/firmware.uf2'
|   |-- esphome 2023.5.0-dev @ 2023-04-29 22:50:42 -> generic-rtl8710bx-4mb-980k
|-- ValueError: Failed to write to 0x800B000
|   |-- File "/home/user/.local/lib/python3.10/site-packages/ltchiptool/soc/ambz/flash.py", line 147, in flash_write_raw
*** [upload] Error 1

The stick is in bricked state after flashing fails. I managed to restore the stock firmware by running (MCU is in UART boot mode):

user@linux-pc:~/rtltool$ python3 rtltool.py -p /dev/ttyUSB0 wf 0x8000000 solis-s3-firmware-1012f.bin 
Connecting...
Write Flash data 0x08000000 to 0x08800000 from file: solis-s3-firmware-1012f.bin ...
Traceback (most recent call last):
  File "/home/user/rtltool/rtltool.py", line 523, in <module>
    main(*sys.argv[1:])
  File "/home/user/rtltool/rtltool.py", line 388, in main
    for _ in gen:
TypeError: 'bool' object is not iterable

Even though I get this error, the stick seems to work fine with the stock firmware.

Any ideas what I might be doing wrong?

[hn]

Well, things are a little bit shaky at the moment. @kuba2k2 does a really great job with LibreTiny, but it's not easy to have stable support for those cloned chips like the EMW3080.

The good thing is: it's doable -- I've got a stable WiFi S3 stick running with HA for some weeks.

With rtltool3.py I also experienced some errors, I reverted back to using rtltool2.py.

Try to flash the app binary alone:

python2 ./rtltool2.py -p /dev/ttyUSB0 wf 0xb000 ./.esphome/build/solis-inv/.pioenvs/solis-inv/image_0x00B000.ota1.bin

If this works and LibreTiny comes up (watch boot log, e.g. screen /dev/ttyUSB0 115200) you might experience WiFi problems with LibreTiny 1.0.1. Then you have to downgrade LibreTiny and apply some patches. Please report back, I'll help you with that.

[spuke113]

Thanks for quick response! Rtltool python2 version worked flawlessly.

As you expected, now I'm experiencing exactly the same WiFi issue you have. Any downgrade tips for downgrading to older LibreTiny would be much appreciated :)

[kuba2k2]

@hn could you post the rtltool2 and rtltool3 versions that you're using? ltchiptool is based on rtltool (an old version that worked on Python 2 I think) which makes it odd that it doesn't work. I could compare the working tool to the current code and maybe fix ltchiptool.

[hn]

Install LibreTiny v0.13.0 (may you have to remove the current version before, I dont know):

platformio platform install -f https://github.com/kuba2k2/libretiny#v0.13.0

This will probably install LibreTiny under its old name LibreTuya and you will have to adjust paths and/or names. Please report back so others can benefit from your learnings.

Apply this diff to make the UART work:

--- ./.platformio/platforms/libretuya/arduino/realtek-ambz/cores/arduino/SerialClass.cpp	2023-03-22 15:50:18.620881017 +0100
+++ ./.platformio/platforms/libretuya/arduino/realtek-ambz/cores/arduino/SerialClass.cpp	2023-03-22 16:50:18.620881017 +0100
@@ -27,9 +27,10 @@
 	data->uart->DLH_INTCR = 0;
 
 	uint8_t c;
-	UART_CharGet(data->uart, &c);
-	if (c)
-		data->buf->store_char(c);
+	while (UART_Readable(data->uart)) {
+		UART_CharGet(data->uart, &c);
+ 		data->buf->store_char(c);
+	}
 
 	data->uart->DLH_INTCR = intcr;
 	return 0;

Compile and upload the binary to the WiFi stick as before. Subsequent updates can be carried out via OTA.

If you experience ModBus errors after some hours (e.g. no new values show up in HA), you may have to apply this "makes-no-sense-but-helps" fix as well: libretiny-eu/libretiny#91 (comment)

[spuke113]

I'm running into a lot of compiling errors. So far, I made the changes you suggested and git checked out latest commit from libretiny-esphome before name change (commit SHA 7d37eea0f731a55274659ca08662dd96386e38dd). I also changed libretiny->tuya in solis-inv-esphome.yaml. I'm currently trying to sort out this error:

Compiling .pioenvs/solis-inv/src/esphome/components/libretuya/preferences.cpp.o
src/esphome/components/libretuya/core.cpp: In function 'void esphome::arch_init()':
src/esphome/components/libretuya/core.cpp:22:3: error: 'lt_wdt_enable' was not declared in this scope
   22 |   lt_wdt_enable(10000L);
      |   ^~~~~~~~~~~~~
src/esphome/components/libretuya/core.cpp:24:3: error: 'lt_gpio_recover' was not declared in this scope
   24 |   lt_gpio_recover();
      |   ^~~~~~~~~~~~~~~
src/esphome/components/libretuya/core.cpp: In function 'void esphome::arch_restart()':
src/esphome/components/libretuya/core.cpp:29:3: error: 'lt_reboot' was not declared in this scope
   29 |   lt_reboot();
      |   ^~~~~~~~~
src/esphome/components/libretuya/core.cpp: In function 'void esphome::arch_feed_wdt()':
src/esphome/components/libretuya/core.cpp:33:38: error: 'lt_wdt_feed' was not declared in this scope
   33 | void IRAM_ATTR HOT arch_feed_wdt() { lt_wdt_feed(); }
      |                                      ^~~~~~~~~~~
src/esphome/components/libretuya/core.cpp: In function 'uint32_t esphome::arch_get_cpu_cycle_count()':
src/esphome/components/libretuya/core.cpp:34:46: error: 'lt_cpu_get_cycle_count' was not declared in this scope
   34 | uint32_t arch_get_cpu_cycle_count() { return lt_cpu_get_cycle_count(); }
      |                                              ^~~~~~~~~~~~~~~~~~~~~~
src/esphome/components/libretuya/core.cpp: In function 'uint32_t esphome::arch_get_cpu_freq_hz()':
src/esphome/components/libretuya/core.cpp:35:42: error: 'lt_cpu_get_freq' was not declared in this scope
   35 | uint32_t arch_get_cpu_freq_hz() { return lt_cpu_get_freq(); }
      |                                          ^~~~~~~~~~~~~~~
Compiling .pioenvs/solis-inv/src/esphome/components/logger/logger.cpp.o
*** [.pioenvs/solis-inv/src/esphome/components/libretuya/core.cpp.o] Error 1
In file included from src/esphome/components/sensor/sensor.h:3,
                 from src/esphome/components/libretuya/lt_component.h:9,
                 from src/esphome/components/libretuya/lt_component.cpp:1:
src/esphome/components/libretuya/lt_component.cpp: In member function 'virtual void esphome::libretuya::LTComponent::dump_config()':
src/esphome/components/libretuya/lt_component.cpp:14:39: error: 'lt_get_version' was not declared in this scope
   14 |   ESP_LOGCONFIG(TAG, "  Version: %s", lt_get_version());
      |                                       ^~~~~~~~~~~~~~
src/esphome/core/log.h:97:90: note: in definition of macro 'esph_log_config'
   97 |   esp_log_printf_(ESPHOME_LOG_LEVEL_CONFIG, tag, __LINE__, ESPHOME_LOG_FORMAT(format), ##__VA_ARGS__)
      |                                                                                          ^~~~~~~~~~~
src/esphome/components/libretuya/lt_component.cpp:14:3: note: in expansion of macro 'ESP_LOGCONFIG'
   14 |   ESP_LOGCONFIG(TAG, "  Version: %s", lt_get_version());
      |   ^~~~~~~~~~~~~
src/esphome/components/libretuya/lt_component.cpp:19:35: error: 'LT_BANNER_STR' was not declared in this scope; did you mean 'LT_BANNER'?
   19 |     this->version_->publish_state(LT_BANNER_STR);
      |                                   ^~~~~~~~~~~~~
      |                                   LT_BANNER
*** [.pioenvs/solis-inv/src/esphome/components/libretuya/lt_component.cpp.o] Error 1

I'm very new to all of this, but I'll report back as soon as I figure this out.

[kuba2k2]

You'd have to check out the latest commit before the refactor in esphome. The name change commit was a bit after the refactor commit.

[hn]

@spuke113 I used libretiny-esphome from mid-march kuba2k2/libretiny-esphome@d076454

[spuke113]

It works! At least for now.
kuba2k2/libretiny-esphome@d076454 commit compiled just fine. I did not uninstall current platformio libretiny - I just installed older version on top of the current one. I made the suggested change in SerialClass.cpp. No other changes were needed, except in solis-inv-esphome.yaml libretiny was renamed back to libretuya. I did not apply that modbus "fix", at least not yet since the stick has been working flawlessly for the past 8 hours.

I don't have UART-USB device connected to the stick at the moment, since it's kinda tricky to access the UART pins while the stick is attached to the inverter (without drilling holes to the case, warranty void bla bla...). Therefore I don't have access to debug logs to monitor status of the modbus, but I suppose the problem shows itself as generic instability in HA as well.

Thanks for useful tips! I'll report back in a couple of days.

[kuba2k2]

@hn we've fixed a malloc() issue just yesterday, which I haven't noticed - it is possible that it was the source of your WiFi problems. Could you check the new version when you get the chance?

[hn]

@spuke113 : Congratulations, you are registered user # 2 for wifi-stick-s3-open-source :-)

For connecting the UART (without voiding warranty -- a good joke if you re-flashed the firmware some minutes ago), I completely removed the PCB vom the case and just plugged in the (black) cap into the inverter. Then I pushed the PCB into the cap, it it sits there quite firmly, even without the case. Warning: do not confuse the orientation of the PCB!

@kuba2k2 : Great, just give me 1-2 days.

[hn]

@kuba2k2 : Just a quick test with 1.0.1+sha.86924d8: WiFi connect works, even WiFi Captive Portal works for the very first time with the EMW3080, hooray :) Did not test ModBus yet, but looks promising.

[kuba2k2]

Note to self: don't use linker aliases.

Let me know if it's stable in a few days.

[spuke113]

@hn The stick worked fine 3 days in a row. Yesterday updates stopped flowing to HA and sure shit modbus was throwing crc errors. I applied that Serial2.println("foo"); "fix" today morning, but only a couple of hours later crc errors were back again. Booo :(

I have a logic analyzer, I'll try to sniff the traffic between the stick and the inverter tomorrow. Deep rabbit hole, here we come!

[hn]

@spuke113 I think the ModBus traffic is ok on wire, for some reason it gets de-synced (all the data is there, but incorrectly shifted into the individual telegrams).

I've opened a new issue to have one place to track this.