setjmp issue

[akouz]

Hardware:

Board: ESP32 Dev Module
Core Installation/update date: 11/jul/2017?
IDE name: Arduino IDE
Flash Frequency: 80Mhz
Upload Speed: 921600

Description:

ESP32 crashes when I run a cooperative operating system https://github.com/akouz/a_coos based on the standard "setjmp.h" library. I can run the sketch "a_coos.ino" on many microcontrollers, including ESP8266, but it does not work on ESP32.

As far as ESP32 Dev Board does not have built-in LED, to compile the sketch modify line 23 of "a_coos.ino" to

static int led = 5;

Debug Messages:

== COOS demo ==
.Guru Meditation Error of type LoadProhibited occurred on core 1. Exception was unhandled.
Register dump:
PC : 0x400d0ae1 PS : 0x00060930 A0 : 0x800d09d5 A1 : 0x3ffcf290
A2 : 0x800d09d5 A3 : 0x00000000 A4 : 0x80083a78 A5 : 0x3ffcf1c0
A6 : 0x00000000 A7 : 0x3ffcd380 A8 : 0x800d0bd5 A9 : 0x3ffcf2a0
A10 : 0x0000028b A11 : 0x0000028b A12 : 0x0001e2fb A13 : 0x3ffc2788
A14 : 0x3ffcf2b0 A15 : 0x3ffcf280 SAR : 0x00000017 EXCCAUSE: 0x0000001c
EXCVADDR: 0x800d0c39 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xffffffff

Backtrace: 0x400d0ae1:0x3ffcf290 0x400d09d2:0x3ffcf2c0 0x400dfe79:0x3ffcf2e0

Rebooting...
ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0010,len:4
load:0x3fff0014,len:716
load:0x40078000,len:0
load:0x40078000,len:11572
entry 0x40078a14
== COOS demo ==
.Guru Meditation Error: Core 1 panic'ed (Double exception)
Register dump:
PC : 0x40086b95 PS : 0x00050836 A0 : 0x800d0adc A1 : 0x3fffffdc
A2 : 0x4000004c A3 : 0x00000000 A4 : 0x3ffc2428 A5 : 0x400d088c
A6 : 0x3ffcf280 A7 : 0x3ffcf260 A8 : 0x3ffcf280 A9 : 0x09f53009
A10 : 0x00000077 A11 : 0x3ffc2430 A12 : 0x1009c500 A13 : 0xe52009d5
A14 : 0x00000020 A15 : 0xff000000 SAR : 0x00000017 EXCCAUSE: 0x00000002
EXCVADDR: 0x09f53008 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xffffffff

Backtrace: 0x40086b95Guru Meditation Error of type LoadStoreError occurred on core 1. Exception was unhandled.

Register dump:
PC : 0x400867d1 PS : 0x00060035 A0 : 0x80086910 A1 : 0x3ffffdfc
A2 : 0x40080b5c A3 : 0x00000000 A4 : 0x00000000 A5 : 0x3ffcf2e0
A6 : 0x00000000 A7 : 0x00000000 A8 : 0x40080b5c A9 : 0x3ffffddc
A10 : 0x00000035 A11 : 0x3ffcf2c0 A12 : 0x800dfe7c A13 : 0x3ffcf2c0
A14 : 0x000237bc A15 : 0x3ffc2788 SAR : 0x00000017 EXCCAUSE: 0x00000003
EXCVADDR: 0x40080b5c LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0x00000000

Backtrace: 0x400867d1Guru Meditation Error of type LoadStoreError occurred on core 1. Exception was unhandled.

... etc, keeps repeating from "Register dump". Sorry, Exception Decoder decodes nothing out of it.

With some extra prints in the sketch I can see the following:

1. Setup function void Coos::start(void) processed successfully, tasks invoked and return control to the scheduler. Thus, setjmp and longjmp functions work, I can store main context and then restore it.

2. Function void Coos::run(void) crashes on line 123 of "coos.cpp", eg when it tries to invoke task using stored task context:

longjmp(task_context[task_no], 1);

[akouz]

It looks like messages are not very consistent. Now I've got the following after reset:

`ets Jun 8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0010,len:4
load:0x3fff0014,len:716
load:0x40078000,len:0
load:0x40078000,len:11572
entry 0x40078a14
== COOS demo ==
.Guru Meditation Error of type LoadProhibited occurred on core 1. Exception was unhandled.
Register dump:
PC : 0x400d0ae1 PS : 0x00060930 A0 : 0x800d09d5 A1 : 0x3ffcf290
A2 : 0x800d09d5 A3 : 0x00000000 A4 : 0x80083a78 A5 : 0x3ffcf1c0
A6 : 0x00000000 A7 : 0x3ffcd380 A8 : 0x800d0bd5 A9 : 0x3ffcf2a0
A10 : 0x0000028b A11 : 0x0000028b A12 : 0x0001e2fb A13 : 0x3ffc2788
A14 : 0x3ffcf2b0 A15 : 0x3ffcf280 SAR : 0x00000017 EXCCAUSE: 0x0000001c
EXCVADDR: 0x800d0c39 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xffffffff

Backtrace: 0x400d0ae1:0x3ffcf290 0x400d09d2:0x3ffcf2c0 0x400dfe79:0x3ffcf2e0

Rebooting...
ets Jun 8 2016 00:22:57

rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0010,len:4
load:0x3fff0014,len:716
load:0x40078000,len:0
load:0x40078000,len:11572
entry 0x40078a14
== COOS demo ==
.Guru Meditation Error: Core 1 panic'ed (Double exception)
Register dump:
PC : 0x40086b95 PS : 0x00050836 A0 : 0x800d0adc A1 : 0x3fffffdc
A2 : 0x4000004c A3 : 0x00000000 A4 : 0x3ffc2428 A5 : 0x400d088c
A6 : 0x3ffcf280 A7 : 0x3ffcf260 A8 : 0x3ffcf280 A9 : 0x09f53009
A10 : 0x00000077 A11 : 0x3ffc2430 A12 : 0x1009c500 A13 : 0xe52009d5
A14 : 0x00000020 A15 : 0xff000000 SAR : 0x00000017 EXCCAUSE: 0x00000002
EXCVADDR: 0x09f53008 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xffffffff

Backtrace: 0x40086b95Guru Meditation Error of type LoadStoreError occurred on core 1. Exception was unhandled.
Register dump:
PC : 0x400867d1 PS : 0x00060035 A0 : 0x80086910 A1 : 0x3ffffdfc
A2 : 0x40080b5c A3 : 0x00000000 A4 : 0x00000000 A5 : 0x3ffcf2e0
A6 : 0x00000000 A7 : 0x00000000 A8 : 0x40080b5c A9 : 0x3ffffddc
A10 : 0x00000035 A11 : 0x3ffcf2c0 A12 : 0x800dfe7c A13 : 0x3ffcf2c0
A14 : 0x000237bc A15 : 0x3ffc2788 SAR : 0x00000017 EXCCAUSE: 0x00000003
EXCVADDR: 0x40080b5c LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0x00000000

Backtrace: 0x400867d1Guru Meditation Error of type LoadStoreError occurred on core 1. Exception was unhandled.
Register dump:
PC : 0x400867d1 PS : 0x00060033 A0 : 0x80086910 A1 : 0x3ffffc3c
A2 : 0x40080b5c A3 : 0x00000000 A4 : 0x80086c75 A5 : 0x3ffffe5c
A6 : 0x3fffff1c A7 : 0x00000017 A8 : 0x40080b5c A9 : 0x3ffffc1c
A10 : 0x00000031 A11 : 0x3ffffe3c A12 : 0x80086b76 A13 : 0x3ffffe3c
A14 : 0x3fffff1c A15 : 0x3fffffdc SAR : 0x0000001e EXCCAUSE: 0x00000003
EXCVADDR: 0x40080b5c LBEG : 0x4000c2e0 LEND : 0x4000c2f6 LCOUNT : 0x00000000 `

... etc

I've added some debug prints to the tasks and to the scheduler. After reset by pushbutton, first it crahes after exit from task:

== COOS demo ==
start: setjmp
setjmp exit, res=0
. task0 pt1, back to coos
setjmp exit, res=501
start: setjmp
setjmp exit, res=0
task1 pt1, back to coos
setjmp exit, res=101
run: setjmp
setjmp exit, res=0
run: longjmp
task1 pt2, back to coos
setjmp exit, res=651
Guru Meditation Error of type LoadProhibited occurred on core 1. Exception was unhandled.

... then it reboots and keeps crashing on longjmp:

== COOS demo ==
start: setjmp
setjmp exit, res=0
. task0 pt1, back to coos
setjmp exit, res=501
start: setjmp
setjmp exit, res=0
task1 pt1, back to coos
setjmp exit, res=101
run: setjmp
setjmp exit, res=0
run: longjmp
Guru Meditation Error: Core 1 panic'ed (Double exception)
... etc

[stickbreaker]

How is longjump handling the stack?
The esp32 uses a small (32k-64k) code cache to move code from the external FLASH memory to ICACHE. So, if the next instruction to be executed is not in ICACHE, program flow halts, a cache refill interrupt is executed.

This error is usually indicative of a bad pointer address, LoadProhibited is saying that an attempt was made to read from an unsupported memory address(usually 0). The memory map of ESP32 is NON-contiguous.

Guru Meditation Error of type LoadProhibited occurred on core 1. Exception was unhandled.

I would look for problems with the code, how it switches execution/task.

I don't know why you don't want to use the built in multi-task support?

Chuck.

[akouz]

Hi stickbreaker, thank you for your reply,

I guess the multi-task support you mentioned is specific to ESP-32, while COOS is a very generic cooperative scheduler. It supposed to run on any Arduino board supporting standard "setjmp.h" library.

If I understand correctly what you said regarding to ESP-32 cash handling and flash memory, ESP-32 implements "setjmp.h" in a non-standard way with a kind of unpredictable behaviour. I expected ESP-32 "setjmp.h" to follow description such as http://www.cplusplus.com/reference/csetjmp/setjmp/

Regards
Alex

[copercini]

Not sure if it help you somehow, but commenting all COOS_DELAY() on .ino file, it's running without crashes

[copercini]

I observed 2 issues using the bellow code (and without changing anything on coos.cpp and coos.h files):
COOS_DELAY(); doesn't delay, just switch to the next task, and when it goes to the last task (task 2 on bellow code) then it crashes, and not return to task0

#include "coos.h"

//##############################################################################
// Var
//##############################################################################

static int led = 25;

//##############################################################################
// Func
//##############################################################################

// ========================================
// Task 0 toggles led every 0.5 sec
// ========================================
void coos_task0(void)
{
  Serial.print("coos_task0");
  static uchar cnt;
  while(1)
  {
    Serial.print(".");
    digitalWrite(led, LOW); 
    delay(500);          // 500 ms
    digitalWrite(led, HIGH); 
    COOS_DELAY(50000);   //it doesn't really delay, just goes to the next task
    delay(500);          // 500 ms
    if (++cnt >= 60)          // every minute
    {
      cnt = 0;
      Serial.print(" uptime = ");
      Serial.print(coos.uptime);
      Serial.println(" sec ");
    }
  }
}
// ========================================
// Task 1 sets led OFF every 0.65 sec
// ========================================
void coos_task1(void)
{
  Serial.print("coos_task1");
  delay(100);    
  while(1)
  {
    delay(650);        // 650 ms
    Serial.print("loop1");
    digitalWrite(led, LOW); 
    COOS_DELAY(1); //switch to task 2 
  }
}
// ========================================
// Task 2 print some text
// ========================================
void coos_task2(void)
{
  Serial.print("coos_task2");
  delay(100);    
  while(1)
  {
    delay(650);        // 650 ms
    Serial.print("loop2");
    //COOS_DELAY(1); //CRASH FOREVER, CRASH FOREVER, CRASH FOREVER, CRASH FOREVER, CRASH FOREVER, CRASH FOREVER
  }
}
// ========================================
// Setup
// ========================================
void setup()
{
  Serial.begin(115200);
  Serial.println("== COOS demo ==");
  pinMode(led, OUTPUT);
  coos.register_task(coos_task0);    
  coos.register_task(coos_task1);
  coos.register_task(coos_task2);
  coos.start();                     // init registered tasks
}

// ========================================
// Main loop 
// ========================================
void loop()
{  
  coos.run();  // Coos scheduler 
}
/* EOF */

[akouz]

Hi copercini,

In your sketch the coos_task2() takes control and runs the loop forever. It defeats the whole point of the cooperative OS, where tasks should release control to the scheduler allowing other tasks to run. Control released by the COOS_DELAY() macro where setjmp/longjmp processed:

#define COOS_DELAY(__delay) if (!setjmp(coos.task_context[coos.task_no])) {longjmp(coos.main_context, __delay+1);} else

Your sketch does not crash because it executes only coos.start(), but never executes coos.run().

Currently every task successfully invoked from coos.start() and returns control back to coos.start() by executing the first COOS_DELAY() in the task body. At this point the task context is stored. The problem appears when coos.run() tries to invoke the task using the stored task context.

In my prints above it is a noticible difference between the very first run after reset and consecuitive runs after reboot.

// After reset  coos.run() can execute task1 once and then ESP-32 crashes:
// ---------- setup() -------------------
== COOS demo ==
start: setjmp              // Coos::start(), before line 96
setjmp exit, res=0             
                           // Coos::start() invokes task 0 at line 100
. task0 pt1, back to coos  // task0 releases control to Coos::start(), task0 context stored
setjmp exit, res=501       // task0 delay will be set to 500 ms
start: setjmp              // Coos::start(), before line 96
setjmp exit, res=0            
                           // Coos::start() invokes task 1 at line 100
task1 pt1, back to coos    // task1 releases control to Coos::start(), task1 context stored
setjmp exit, res=101       // task1 delay will be set to 100 ms
// ---------- loop() -------------------
run: setjmp                // 100 ms passed, Coos::run(), before line 120
setjmp exit, res=0         // Coos::run() after line 120
run: longjmp               // Coos::run() invokes task 1, before line 123
task1 pt2, back to coos    // task1 releases control to Coos::run(), task1 context stored
setjmp exit, res=651       // Coos::run() after line 120, task1 delay will be set to 650 ms
                           // ESP-32 crashed  
Guru Meditation Error of type LoadProhibited occurred on core 1. Exception was unhandled.

Thus, longjmp(coos.main_context, 651); in the macro COOS_DELAY(650) was processed, main context restored, but after that ESP-32 crashed somewhere...

// After reboot ESP-32 crashes when coos.run() tries to invoke task1 first time:
// ---------- setup() -------------------
== COOS demo ==
start: setjmp              // Coos::start()
setjmp exit, res=0
                           // Coos::start() invokes task 0
. task0 pt1, back to coos  // task0 releases control to Coos::start(), task0 context stored
setjmp exit, res=501       // task0 delay will be set to 500 ms
start: setjmp              // Coos::start()
setjmp exit, res=0            
                           // Coos::start() invokes task 1
task1 pt1, back to coos    // task1 releases control to Coos::start(), task1 context stored
setjmp exit, res=101       // task1 delay will be set to 100 ms
// ---------- loop() -------------------
run: setjmp                // 100 ms passed,  Coos::run(), before line 120
setjmp exit, res=0         // Coos::run() after line 120             
run: longjmp               //  Coos::run() invokes task 1, before line 123
                           // ESP-32 crashed
Guru Meditation Error: Core 1 panic'ed (Double exception)

[akouz]

I modified Coos::run() as follows:

void Coos::run(void)
{
  static uchar pr = 0;
  int   res;
  int tmp;
  void (*tsk)(void);
  if (task_delay[task_no] == 0)
  {
    Serial.println("run: setjmp");
    res = setjmp(main_context);           // set return point and get delay value from the task 
    Serial.print(" setjmp exit, res=");
    Serial.println(res);
    if (res == 0)                         // after setting return point
    {
      Serial.println("run: longjmp");
      pr = 1;
      longjmp(task_context[task_no], 1);  // invoke task 
    }
    else                                  // after returning from task 
    {
      Serial.println("run: set delay");
      tmp = --res;              // added for debugging
      Serial.println("run: reload");
      task_delay[task_no] = tmp;        // **it crashes here, it is line 137**
      Serial.println("run: delay set");
    }
  }
  if (pr)  Serial.println("run: next task");
  task_cnt = (task_cnt > COOS_MAX_TASKS)? COOS_MAX_TASKS : task_cnt;
  if (++task_no > task_cnt)               // next task
  {
    task_no = 0;                          // round-robin
  }
  if (pr) Serial.println("run: update time");
  update_time(); 
}

After that I occasionally get the following output:

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0010,len:4
load:0x3fff0014,len:716
load:0x40078000,len:0
load:0x40078000,len:11572
entry 0x40078a14
== COOS demo ==
.run: setjmp
 setjmp exit, res=0
run: longjmp
 setjmp exit, res=651
run: set delay
run: reload
Guru Meditation Error of type LoadProhibited occurred on core  1. Exception was unhandled.
Register dump:
PC      : 0x400d0b0f  PS      : 0x00060930  A0      : 0x800d099d  A1      : 0x3ffcf250  
A2      : 0x3ffc2794  A3      : 0x0000028a  A4      : 0x800d099d  A5      : 0x3ffcf180  
A6      : 0x00000000  A7      : 0x3ffcd340  A8      : 0x800d0b9d  A9      : 0x3ffcf230  
A10     : 0x0000000d  A11     : 0x0000000b  A12     : 0x0000000a  A13     : 0x3ffc278c  
A14     : 0x3ffcf270  A15     : 0x3ffcf240  SAR     : 0x00000017  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x800d0c01  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff  

Backtrace: 0x400d0b0f:0x3ffcf250 0x400d099a:0x3ffcf280 0x400dc82d:0x3ffcf2a0

Rebooting...

Backtrace decoding:

0x400d0b0f: Coos::run() at C:\Users\user\AppData\Local\Temp\arduino_build_289468\sketch\coos.cpp line 137
0x400d099a: loop() at E:\Proj\18_Arduino\Sketchbook\a_coos/a_coos.ino line 81
0x400dc82d: loopTask(void*) at C:\Users\user\Documents\Arduino\hardware\espressif\esp32\cores\esp32\main.cpp line 18

coos.cpp line 137 is:

task_delay[task_no] = tmp;

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This stale issue has been automatically closed. Thank you for your contributions.

[f16falcona46]

I believe such a crash is legal behavior for a C or C++ runtime, since longjmp'ing down the stack is undefined behavior.

If the function that called setjmp has exited (whether by return or by a different longjmp higher up the stack), the behavior is undefined. In other words, only long jumps up the call stack are allowed.

[akouz]

This would be an issue is someone tries to use setjmp/longjmp in functions. But for COOS it is not the case. There is a clear warning about that:

COOS_DELAY(x) must be used in the task itself; it cannot be used in functions called from task.

[f16falcona46]

The task itself is a function, too. When a task firsts calls COOS_DELAY, it longjmp's back to Coos::start. At this point the task has exited (since it has longjmp'ed up the call stack), so the jmp_buf set by the setjmp inside the task is no longer valid. That means that the longjmp that Coos::run performs to that jmp_buf is also invalid.

[akouz]

The issue will arise if stack is changed. As soon as stack is the same, setjump/longjump operate correctly.

If one registers coos_task0() as coos.register_task(coos_task0), it must call COOS_DELAY() from coos_task0() itself. If another function inside coos_task0() is used to call COOS_DELAY(), stack will be changed and system will crash. But it will operate correctly when COOS_DELAY() called from coos_task0() because stack is intact.

It is very easy to test, and it has been tested many-many times. While stack was not changed, COOS operates correctly. Must not use functions to call COOS_DELAY(), must use it within registered function itself.