Slow Webinterface on Hardware rev. 3 -> Most likely hardware issue #2298
Comments
|
Sounds like a similar problem to the one I am having. |
|
There is somebody also who has performance issues on a rev3 hardware, see #2301 Maybe this is related... |
|
I improved stability, at least temporarly. It took me over 4 hours to configure everything, as the interface was stupidly slow and crashed regularly. Found the option to push the clock frequency to 240 which helped tremendously with performance ... until I rebooted once. Had almost 50% packetloss before and latency was always >250ms with spikes up to 10s. After switching clock frequency it ran at around 150ms on average with spikes going to 400ms. Now, after reboot, it got subjectively worse again. Sadly I do not have any more time to check today. |
|
Unfortunately, don't have any further technical information - I bought 4 of them, and it seems like all 4 have the same issues. This is also not limited to HTTP interface, but also MQTT, JSON API, and just in general awful performance. A few things I've noticed:
The chip is v3 revision, from logs: The ESP has been purchased from https://www.berrybase.de/esp32-cam-development-board-inkl.-ov2640-kameramodul |
That covers basically everything I experienced yesterday afternoon/night. I tested two different manufacturers too, one had 8mb of PSRAM and one “just” 4. No difference whatsoever. Blowing (yeah, sounds weird) on the chip to cool it down did not improve packet loss or ping times (at least not within 30 seconds, I guess no temperature problems). My guess would be that it has to do with Chip rev 3, and it seems not to be able to cope with the computational demands AI on the edge has. Which is odd, shouldn't a rev3 always perform at least similar or better than other versions? As I was able to finish configuration (took hours, literally), my unit ran throughout the night, and I connected it to home assistant. Over the course of 8 hours and an update interval of 10 minutes (I doubled the default for less strain on the unit), only about 10 updates were received but also 20 warnings that the unit was not available. So, probably packet loss, again. |
|
Thank you both for the analysis you did. One thing we could try to do is to update the Expressif platform (IDF). We still are on an older one. maybe they had some silicon bugs in rev3 which require a newer IDF. 2 months ago i started to migrate for this but it was not as easy as thought, some functions are no longer available. Make sure to do a backup beforehand! |
|
Will try that out as soon as I'm back home. The second unit is not used for anything currently, therefore testing is nbd. Thanks for the hint! Maybe this doc gives some insights, too? Especially "Impact on Customer Projects". Just a long shot, though: https://www.espressif.com/sites/default/files/documentation/ESP32_ECO_V3_User_Guide__EN.pdf |
|
According to that document, not much has changed. |
I got to try it out. WebUI seems to be responding quite well for the OTA-Upload part (not sure how that is called). Uploading the remote.zip, which did not work before whatsoever! Ping times were below 10ms for some pings, never seen that before. The initial setup is way more responsive, too. NO packetloss whatsoever. Ping times go up a little, but stay around 50-250 sitting 1m from the router. When interacitng with the setup dialogue, it does load every here and there. Never had a packet drop though!
My initital feeling: updating the IDF/migrating to Platformio 6.x.x fixes this issue! |
I tried this on my rev3 and it first appeared to be performing better, but no. Just as slow as with the stable version. |
|
@penapena did you do a fresh install and completely wipe the sd card as well? |
Did erase_flash before install. Didn't format the sdcard, but deleted old files and pasted new ones. |
|
The SD-Card content should not matter for the performance. I built now a version (based on rolling) which specifically only supports rev3. I can't test it myself, so make sure you have USB access to revert if it does not work: https://github.com/jomjol/AI-on-the-edge-device/actions?query=branch%3Aset-min-version-to-rev3 As for the framework update: Since that at least for one of you might help, I want to follow this path one step deeper. I am trying to take the latest rolling and update the framework to |
|
I would like to join this conversation as I face the same issue. |
|
Later today, I will try another device and see if the update performs better on that one, too. Just to make sure that I did not by chance pick, out of the four rev 3's I own, one that is somehow functional. |
|
I completely fresh install the watermeter with https://github.com/jomjol/AI-on-the-edge-device/actions?query=branch%3Aset-min-version-to-rev3 and I can tell that it doesn't make any changes for me. |
|
I also installed the version (https://github.com/jomjol/AI-on-the-edge-device/actions/runs/4152463509) yesterday and the ping response times were dramatically reduced. My times correspond to those of @lterfloth. The image build times of the GUI have not really changed but I could at least go through completely. |
Yes, image build times remained more or less the same. Biggest difference is the package loss which was just not happening anymore (i.e. the system was stable). The package loss leads to many problems down the line. |
which pcb revision do you have? check the log for it. |
With this the rev3 works somehow if you put CPU to 240 in the config file. With 160 I couldn't access it at all. However on RSSI -70 it is very slow, but I can still access it (rev1 works great even at RSSI -80). Close to the router with around -50 it works good, but not as fast as rev1. |
|
@penapena @ all: |
Rev3. Sorry, I should have added this before. |
Pls send me a PN (if possbile here). I'm willing to do so. |
|
@caco3 is branch platformio6 already suitable for testing here at home? I mean, the device I am using right now does not work reliably anyway, therefore I would not be mad if it has issue. Just wondering whether the code needed for migration is already done. I'd be happy to test it. I don't mind a slow web UI and other usability related issues. As long as the device is able to read the values and send it to home assistant via MQTT, I'm happy :-) |
|
@lterfloth No, sorry, somehow there was a change between the |
Github does not provide such feature, but you find my contact information at https://www.ruinelli.ch/about |
Both are on internal antenna. |
|
@HolBaum5 Das ist super! Bitte teste doch auch mal den neusten Build mit 6.1.0. (siehe mein letzter Eintrag hier) .Das basiert auf rolling und das möchte ich gerne nächstens ins rolling mergen. |
|
doch, du hast Recht, sorry. |
|
Ist das nun bereits in der Testphase? Soll ich die SW auf mein Testsystem laden? |
|
Ich hab das Release auf mein zusammengepfuschtes (i.a.W. thermisch optimiertes) System gespielt. Läuft bis dato reibungslos und einwandfrei in Einklang mit Home Assistant und Datentransfer via MQTT. |
|
Es gibt kleine Kühlkörper bei amazon zu kaufen, sind für den RBP 4, aber ich verwende diese auch immer wieder für andere Basteleien, diese kommen mit wärmeleitfähigen Selbstklebeband:
|
|
@friedpa Du darfst gerne Testen und ein Feedback geben. Das hilft uns. |
|
I can also confirm that finger in left corner really helps. Could it be some invalid initialization for camera I/O ports where "finger" makes workaround? I think this anyway means that root cause is not heating problem, instead something else. I also tried to push with antistatic material this corner without help - which sounds a bit that root cause might not be bad soldering neither.
|
|
The whole camera initialization and driver are all provided by Espressif, thats not implemented by us. Because of this I don't think it is something we could catch/fix! For me it looks like a hardware issue (bad batch), @penapena also got other rev3 devices which seem to be fine |
|
Off Topic: |
at least I got reasonable value when using |
I agree. Doesn't seem to be project issue. Most probably driver (perhaps some pull-up is missing) or HW issue (perhaps wrong component etc). Where might be rev3 schema and pcb pictures ? Would like to do some oscilloscope debugging to understand is there anything we could make these problematic board working... |
I didn't find any usefull docs on it. There is a schematics floating around with version 1.6, not sure to what this translates. ALso the PCB revision does not give any information on the used ESP32S-Module on it! https://github.com/SeeedDocument/forum_doc/blob/master/reg/ESP32_CAM_V1.6.pdf |
|
Ich habe jetzt 4 Module von Amazon gekauft und getestet (Lieferant: "diymore"). Keines dieser Module funktioniert. |
I'm interested in this too as my ones ordered seem somehow faulty. Anyone has a good source to get high quality boards? |
|
Short update: The UI is now much better to be used and also the package https://github.com/jomjol/AI-on-the-edge-device/actions/runs/4726081126 seems to do a good job. One thing left: the soft reboot doesn't always picks up the latest configuration with my board. After pulling the plug and attach it again to the board the new settings are picked up correctly. Now the board runs. |
|
Would you mind posting a picture of your solution for adding a heat sink? |
Can you show that with a log file? Please do so in a separate issue. It would be rater odd and does not make sense to me! |
|
Has anybody who thinks he has thermal issues tried if running it in the fridge or freezer solvers that issue? That would be the best indicator to prove it is a thermal issue! |
|
I tried a raspberrypi-heatsink directly on top of the marked area without any changes in performance. Also the "copper-wire as a heatsink" didn't work for me. |
tried freezer and fridge without any help for me. Stays slow. |
|
Thanks! One possible scenario I see: Nevertheless I am sure it is a hardware issue. Therefore I am closing this bug. But feel free to continue sharing your experiences in here. |
caco3
changed the title
|
I bought these from Amazon.de and they are OK (also rev3). I tried 4 from a different seller without success. I don't know if I was just lucky. |
|
I have now tested the platformio6 in my testsystem, afterwards in my both production system and I have then integrated the newest rolling. I have no problems dtetected. Very smart change! Thanks! |
|
I have the same issue with 3.1 PCB. Is there a solution now you can reommend? I tried like mentioned the 6.1.0. So I did this:
Its also very slow and dont take Images for reference, so its not useable for me like the way I did. Is there any way to get this working? Otherwise, is there a store, which sells Rev. >3? Want to present this cool project as "easy to setup AI application for smart home" in a few weeks 😢 |
|
use the latest release, not the experimental builds listed here. everything is now in the release. |
The Problem
I can answer all questions with "yes". The PSRAM is ESP PSRAM64H 462021 / 1B00286, it's an ESP32CAM Module("diymore ESP32 CAM Entwicklungsplatine, WLAN/Bluetooth, ESP32 DC 5V Dual-Core-Entwicklungsplatine mit 2640 Kamera-TF-Karten-Modul").
I use a 16GB SD Card. I flash using the webinstaller & Chrome. I tried holding IO0 and not holding it while flashing. I tried two units, both have the same issues (I bought a two pack).
What happens:
Flashing works fine, but the web interface is stupidly slow. The ESP is beside my Macbook so it can't be the AP signal. I tried uploading remote.zip which most of the time does not work fails after a while. I attached a log file.
I also tried manually uploading the config to the SD card. That got me a step further, yet the webinterface gets unresponsive when creating a reference image using the pre-configured SD card.
Version
15.1.1
Logfile
Expected Behavior
No response
Screenshots
No response
Additional Context
No response
The text was updated successfully, but these errors were encountered: