Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crashes when using -D WLED_ENABLE_DMX builds with led output #4298

Open
1 task done
dosipod opened this issue Nov 21, 2024 · 20 comments
Open
1 task done

Crashes when using -D WLED_ENABLE_DMX builds with led output #4298

dosipod opened this issue Nov 21, 2024 · 20 comments
Labels
backburner low priority compared to other issues bug confirmed The bug is reproducable and confirmed

Comments

@dosipod
Copy link
Contributor

dosipod commented Nov 21, 2024

What happened?

When compiling -D WLED_ENABLE_DMX and the audio usermod is also included ( it is there by default now ) you would get the error
image

Compile is fine if the audio usermod is disabled
I have no dmx fixture to verify functionally afterword but just helping someone with the compile so not sure that is expected or not

To Reproduce Bug

Use -D WLED_ENABLE_DMX on 15.0-b7

Expected Behavior

No failure

Install Method

Self-Compiled

What version of WLED?

0.15.0-b7 (build 2410270)

Which microcontroller/board are you seeing the problem on?

ESP32

Relevant log/trace output

No response

Anything else?

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@dosipod dosipod added the bug label Nov 21, 2024
@softhack007
Copy link
Collaborator

@dosipod thanks, interesting find

This message was actually not added by me 🤔

20241121_192028

@blazoncek do you remember why you've added this error?
Does DMX output require I2S ?

@softhack007 softhack007 added the confirmed The bug is reproducable and confirmed label Nov 21, 2024
@blazoncek
Copy link
Collaborator

I no longer remember why, but it was definitely intentional.
As DMX is not a default feature I would assume that the person compiling would understand the message and remove audioreactive usremod from compilation.
This is true for some other compile settings as well.

@dosipod
Copy link
Contributor Author

dosipod commented Nov 21, 2024

netmindz mentioned that when using dmx that the unit might be not used for anything else even led output ( I see the unit crashes when i use led output ) so might be the case its dmx only unit

@netmindz
Copy link
Collaborator

netmindz commented Nov 22, 2024

DMX output is just serial output so I don't think it's a conflict but perhaps more an issue of lack of CPU time

@dosipod
Copy link
Contributor Author

dosipod commented Nov 22, 2024

I went back to v0.14.4 to compare and the audio usermod seems to be working fine with dmx out enabled
and I do not see any issue when enabling led output so must be a reason for the change on 15.0-b7

@dosipod
Copy link
Contributor Author

dosipod commented Nov 22, 2024

The behavior we see is as follows ( before we try softhack fix ) :

A) On 14.4 with dmx out enabled and AR included :
A.1) Enabling AR does not seem to cause any issue at least for audio sync .
A.2) Setup of led output also works fine with dmx out (Proxy Universe set to 0 then fps is very low)
and (Proxy Universe set to 1 then fps is normal ) ,we can change Proxy Universe back and forth without any issue

B) On 15.0-b7 with dmx out enabled and AR disabled :
If you setup led output and try to change Proxy Universe from 1 to 0 then you will see a crash ( not enabled debug yet to keep changes to minimum )

[06:49:49]Rebooting...
[06:49:49]ets Jun  8 2016 00:22:57
[06:49:49]
[06:49:49]rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[06:49:49]configsip: 0, SPIWP:0xee
[06:49:49]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[06:49:49]mode:DOUT, clock div:2
[06:49:49]load:0x3fff0030,len:1184
[06:49:49]load:0x40078000,len:12784
[06:49:49]load:0x40080400,len:3032
[06:49:49]entry 0x400805e4
[06:49:50]Ada
[06:49:50]Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[06:49:50]Core 1 register dump:
[06:49:50]PC      : 0x400813d7  PS      : 0x00060c30  A0      : 0x800e92d3  A1      : 0x3ffb1ec0  
[06:49:50]A2      : 0x3f000000  A3      : 0x00060c20  A4      : 0x4008e90f  A5      : 0x3ffb1ee0  
[06:49:50]A6      : 0x00050c23  A7      : 0x3ffb8058  A8      : 0x00000081  A9      : 0x3ffb1ea0  
[06:49:50]A10     : 0x000001fa  A11     : 0x00000000  A12     : 0x000001fa  A13     : 0x0000d100  
[06:49:50]A14     : 0x00000003  A15     : 0x00060c23  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[06:49:50]EXCVADDR: 0x000000e5  LBEG    : 0x40002390  LEND    : 0x4000239f  LCOUNT  : 0x00000000  
[06:49:50]
[06:49:50]ELF file SHA256: 0000000000000000
[06:49:50]
[06:49:50]Backtrace: 0x400813d4:0x3ffb1ec0 0x400e92d0:0x3ffb1ee0 0x40118dd8:0x3ffb1f40 0x40119302:0x3ffb1f90 0x401390fd:0x3ffb1fb0 0x4008c172:0x3ffb1fd0
[06:49:50]
[06:49:50]Rebooting...
[06:49:50]ets Jun  8 2016 00:22:57
[06:49:50]
[06:49:50]rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
[06:49:50]configsip: 0, SPIWP:0xee
[06:49:50]clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
[06:49:50]mode:DOUT, clock div:2
[06:49:50]load:0x3fff0030,len:1184
[06:49:50]load:0x40078000,len:12784
[06:49:50]load:0x40080400,len:3032
[06:49:50]entry 0x400805e4
[06:49:51]Ada
[06:49:51]Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
[06:49:51]Core 1 register dump:
[06:49:51]PC      : 0x400813d7  PS      : 0x00060c30  A0      : 0x800e92d3  A1      : 0x3ffb1ec0  
[06:49:51]A2      : 0x3f000000  A3      : 0x00060c20  A4      : 0x4008e90f  A5      : 0x3ffb1ee0  
[06:49:51]A6      : 0x00050c23  A7      : 0x3ffb8058  A8      : 0x00000081  A9      : 0x3ffb1ea0  
[06:49:51]A10     : 0x000001f8  A11     : 0x00000000  A12     : 0x000001f8  A13     : 0x0000d100  
[06:49:51]A14     : 0x00000003  A15     : 0x00060c23  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c  
[06:49:51]EXCVADDR: 0x000000e5  LBEG    : 0x40002390  LEND    : 0x4000239f  LCOUNT  : 0x00000000  
[06:49:51]
[06:49:51]ELF file SHA256: 0000000000000000
[06:49:51]
[06:49:51]Backtrace: 0x400813d4:0x3ffb1ec0 0x400e92d0:0x3ffb1ee0 0x40118dd8:0x3ffb1f40 0x40119302:0x3ffb1f90 0x401390fd:0x3ffb1fb0 0x4008c172:0x3ffb1fd0
[06:49:51]
[06:49:51]Rebooting...

@dosipod
Copy link
Contributor Author

dosipod commented Nov 22, 2024

@softhack007 I can still see the crash now even after the fix by just setting led output with 256 leds with dmx enabled build . So seems the whole dmx is busted but I am going to leave this be as I have no fixture nor a real interest in DMX but still if you need us to test the crash part then that we could do

@blazoncek
Copy link
Collaborator

@dosipod please use exception decoder when posting crash dumps. Without it they are almost meaningless.
The only usable information from this crash dump is that it is caused by invalid pointer.

@dosipod
Copy link
Contributor Author

dosipod commented Nov 22, 2024

@blazoncek When I enabled debug there was nothing useful , might be you could give that a shot

@softhack007
Copy link
Collaborator

softhack007 commented Nov 22, 2024

@dosipod could you upload a cfg.json just before you change Proxy Universe from 1 to 0 ?
I'm not very familiar with DMX out, and was not able to reproduce the crash yet.

@softhack007
Copy link
Collaborator

softhack007 commented Nov 22, 2024

Update: I've managed to bring my esp32 into a boot loop with DMX out enabled , and one output with 256 leds
I even removed audioreactive and espNow, and its still boot-looping. The crash location looks random, not sure what's going on...

---WLED 0.15.0-b7 2410270 INIT---

esp32 v3.3.6-16-gcc5440f6a2
arduino-esp32 v1.0.x

CPU:   ESP32-D0WDQ5 rev.3, 2 core(s), 240 MHz.
FLASH: 4 MB, Mode 3 (DOUT), speed 40 MHz.
heap 312560
JSON buffer allocated: 32767
PIN ALLOC: Pin 1 successfully allocated by 0x89.
PIN ALLOC: 0x00000000000002.
PIN ALLOC: Pin 2 successfully allocated by 0x8A.
PIN ALLOC: 0x00000000000006.
Registering usermods ...
heap 279724
Reading config
Reading settings from /wsec.json...
JSON buffer locked. (3)
JSON buffer released. (3)
JSON buffer locked. (1)
Reading settings from /cfg.json...
Heap before buses: 277408
Removing all.
Maximum LEDs on a bus: 256
Digital buses: 1
PIN ALLOC: Pin 16 successfully allocated by 0x82.
PIN ALLOC: 0x00000000010006.
Successfully inited strip 0 (len 256) with type 22 and pins 16,255 (itype 23). mA=55/0
LED buffer size: 2304B
Heap after buses: 272416
PIN ALLOC: Pin 0 successfully allocated by 0x85.
PIN ALLOC: 0x00000000010007.
Starting usermod config.
JSON buffer released. (1)
heap 272372
Initializing strip
Loading custom palettes
heap 272168
Usermods setup
heap 272168
Defaulf WiFi used.

Ada
initServer
heap 217080
initIR
heap 217080
Initial connect or forced reconnect.
Defaulf WiFi used.
initConnection() called.
Access point disabled (init).
Connecting to WLAN2...
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC      : 0x40081437  PS      : 0x00060c30  A0      : 0x800e8e0f  A1      : 0x3ffb1e90
A2      : 0x00000000  A3      : 0x00060c20  A4      : 0x4008e847  A5      : 0x3ffb1eb0  
A6      : 0x00050c23  A7      : 0x3ffb8058  A8      : 0x800825a2  A9      : 0x3ffb1e70
A10     : 0x0000024f  A11     : 0x00000000  A12     : 0x0000024f  A13     : 0x00001180  
A14     : 0x00000003  A15     : 0x00060c23  SAR     : 0x0000000a  EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000040  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff  

ELF file SHA256: 0000000000000000

Backtrace: 0x40081434:0x3ffb1e90 0x400e8e0c:0x3ffb1eb0 0x40119db8:0x3ffb1f10 0x4011a2e2:0x3ffb1f90 0x4013b339:0x3ffb1fb0 0x4008c2a2:0x3ffb1fd0
  #0  0x40081434:0x3ffb1e90 in Segment::progress() const at wled00/FX_fcn.cpp:1549
  #1  0x400e8e0c:0x3ffb1eb0 in Segment::handleTransition() at wled00/FX_fcn.cpp:1549
      (inlined by) WS2812FX::service() at wled00/FX_fcn.cpp:1322
  #2  0x40119db8:0x3ffb1f10 in WLED::loop() at wled00/bus_manager.h:122
  #3  0x4011a2e2:0x3ffb1f90 in loop() at wled00/wled_main.cpp:23
  #4  0x4013b339:0x3ffb1fb0 in loopTask(void*) at C:/Users/frank/.platformio/packages/framework-arduinoespressif32@src-e9b1fbd6563a55e19ddae15e1fc09589/cores/esp32/main.cpp:23
  #5  0x4008c2a2:0x3ffb1fd0 in vPortTaskWrapper at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...

@softhack007
Copy link
Collaborator

I can create a crash on a freshly wiped esp32. just by installing a build with DMX out enabled, then setting one led output (ws2812b) with 256 leds, and toggleling DMX output (Proxy Universe to 1), then rebooting.

I think we should understand what's happening before releasing 0.15.0-RC1

@dosipod
Copy link
Contributor Author

dosipod commented Nov 22, 2024

@dosipod could you upload a cfg.json just before you change Proxy Universe from 1 to 0 ? I'm not very familiar with DMX out, and was not able to reproduce the crash yet.

@softhack007 Not sure you still need that as the unit like you noticed would crash by just setting up led output if dmx out is enabled in the build ( I never changed any other setting so it is fresh serial install ) .

dmx_out_crash_fresh_wled_cfg.json

I am starting to think there could be other areas and specially usermods that might cause crashes as it seems a lot of changes are not tested as it is not possible to test on all different types of devices with all the available configuration options and usermods , that is without releasing for the masses to test so it is really hard decision to make

@softhack007
Copy link
Collaborator

softhack007 commented Nov 22, 2024

I am starting to think there could be other areas and specially usermods that might cause crashes as it seems a lot of changes are not tested as it is not possible to test on all different types of devices with all the available configuration options and usermods , that is without releasing for the masses to test so it is really hard decision to make

Yes that's a tricky question - but first of all, we are all very thankful for the time and effort that you and your guys put into testing WLED 👍

I think that mixed outputs, as well as led settings page could be put aside for a moment (until bugs are fixed).

  • You could try some of the major usermods, like four-line-display + rotary, animated staircase, multirelay, etc
  • the new brightness limiter feature could need some more testing - global limiter or per-port limiter. I remember that some users had problems with unequal brightness on panels driven by several output pins. Maybe even measure ampere and compare to what WLED says (wled ampere estimation is not exact, but shout still be similar to the real value)
  • esp-now to sync WLED instances that do/don't share the same wifi
  • wled behaviour when several wifi stations are around (it should always connect to the strongest signal)
  • regression testing (anything that used to work in 0.14)

Just a few ideas - maybe other maintainers have other areas that could use some more testing....

@dosipod
Copy link
Contributor Author

dosipod commented Nov 22, 2024

Thank you for the kind words ,unfortunately there is no more my guys so I am alone now testing , on occasions a student would try to help but that is more of an effort to get them up to speed only to lose them after word due to lack of interest .

We could talk about testing ( If that was not part of the usual practice for PRs ) but I rather not in this post as it might be more of a direction rather then case to case bases

@blazoncek
Copy link
Collaborator

@softhack007 just for clarification:

  • brightness limiter is always per port. One option just distributes current proportionally
  • ESP-NOW sync is not yet implemented (PR is in draft)
  • multiple WiFi does connect to the strongest signal according to RSSI obtained, it will not switch to another SSID unless connection is lost, though
  • of the most used usermods I never tested Staircase usermods, 4LD, rotary, temperature, multi relay, audio, pwm fan, autosave are always in my builds.
  • I don't use: DMX or any live apart from DDP from WLED to WLED

@DedeHai
Copy link
Collaborator

DedeHai commented Nov 23, 2024

looking at the register dumps: both crash dumps hint to a null-pointer access. or rather: an array or struct access that is a null pointer: EXCVADDR: 0x00000040 is the address that caused the access fault. progress() accesses the _t temporary segment transition struct without check if it is valid (which it should be if isInTransition==true ).
@softhack007 are other crashes also related to the _t struct or are they happening randomly?

@blazoncek
Copy link
Collaborator

@DedeHai @dosipod if you want, test with speed optimisations branch (IDK where that was) where progress() was moved to a variable to rule out uninitialised _t pointer.

@DedeHai
Copy link
Collaborator

DedeHai commented Nov 23, 2024

if you mean #4256 I don't think it will make much difference, it still uses the _t pointer, just not accessing it as often.

@5t0n3r1pp3r
Copy link

I am working on new dmx ways currently and have 42fps and 42 hz output on the dmx line and another version that allows up to 101 fps on wled again but we only get 22 hz max on dmx. i will be adding the code here #4239 for both versions.

this should allow dmx again on the sound reactive and also other cpu intensive usermods

@dosipod dosipod changed the title Using -D WLED_ENABLE_DMX when audio is included result in compile failure Crashes when using -D WLED_ENABLE_DMX builds with led output Nov 24, 2024
@softhack007 softhack007 added the backburner low priority compared to other issues label Dec 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backburner low priority compared to other issues bug confirmed The bug is reproducable and confirmed
Projects
None yet
Development

No branches or pull requests

6 participants