[BUG] ISR dispatch and GPTIMER problematic enable (IDFGH-13527)

[filzek]

Answers checklist.

• I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
• I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
• I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

### Issue Description:
We are encountering an issue with the ESP32 GPTimer system when attempting to execute two timers in rapid succession, both being set from within an ISR. Specifically, the first GPTimer (gptimer_triacs) always executes perfectly, but the second GPTimer (gptimer_triacs_all_off) sometimes fails to execute entirely. No errors are reported during the arming or setting of the timers, but the callback for gptimer_triacs_all_off occasionally does not get dispatched.

### Code Overview:
The following GPTimers are created and configured:

// Global configuration for both timers
gptimer_config_t IRAM_ATTR timer_config_us_up = {
    .clk_src = GPTIMER_CLK_SRC_DEFAULT,
    .direction = GPTIMER_COUNT_UP,
    .resolution_hz = 1 * 1000 * 1000, // 1MHz, 1 tick = 1us
    .intr_priority = 3,
    .flags.intr_shared = false,
};

// Creating and configuring the first timer (gptimer_triacs)
gptimer_new_timer(&timer_config_us_up, &gptimer_triacs);
gptimer_event_callbacks_t callback_source_for_triacs_arm = {
    .on_alarm = execute_alarm_for_triacs_arm,
};
gptimer_register_event_callbacks(gptimer_triacs, &callback_source_for_triacs_arm, NULL);
gptimer_enable(gptimer_triacs);

// Creating and configuring the second timer (gptimer_triacs_all_off)
gptimer_new_timer(&timer_config_us_up, &gptimer_triacs_all_off);
gptimer_event_callbacks_t callback_source_for_triacs_disarm_all = {
    .on_alarm = timer_disabletriacs,
};
gptimer_register_event_callbacks(gptimer_triacs_all_off, &callback_source_for_triacs_disarm_all, NULL);
gptimer_enable(gptimer_triacs_all_off);

### Timer Arming in ISR:
Inside an ISR, both timers are re-armed sequentially:

// Re-arm and start gptimer_triacs
gptimer_stop(gptimer_triacs);
gptimer_set_raw_count(gptimer_triacs, 0);
gptimer_alarm_config_t us_triac_dispatch = {
    .alarm_count = diff,  // TRIAC timing
};
gptimer_set_alarm_action(gptimer_triacs, &us_triac_dispatch);
gptimer_start(gptimer_triacs);

// Re-arm and start gptimer_triacs_all_off
gptimer_stop(gptimer_triacs_all_off);
gptimer_alarm_config_t us_dispatch_all_triacs_off = {
    .alarm_count = diff_off,  // Delay for TRIACs off
};
esp_err_t ret = gptimer_set_raw_count(gptimer_triacs_all_off, 0);
if (ret != ESP_OK) {
    printf("Failed to set raw count for gptimer_triacs_all_off: %s\n", esp_err_to_name(ret));
}
ret = gptimer_set_alarm_action(gptimer_triacs_all_off, &us_dispatch_all_triacs_off);
if (ret != ESP_OK) {
    printf("Failed to set alarm action for gptimer_triacs_all_off: %s\n", esp_err_to_name(ret));
}
ret = gptimer_start(gptimer_triacs_all_off);
if (ret != ESP_OK) {
    printf("Failed to start gptimer_triacs_all_off: %s\n", esp_err_to_name(ret));
}

Problem Summary:

• gptimer_triacs: Executes perfectly every time.
• gptimer_triacs_all_off: Sometimes does not execute at all, meaning the callback (timer_disabletriacs) does not get called.

This issue persists even though:

• Both timers are configured similarly with appropriate priority levels (tested with priorities 1, 2, and 3).
• The timers are armed and started in sequence with no errors reported.
• The delay (diff_off) for gptimer_triacs_all_off is roughly 250μs after gptimer_triacs.

Test Conditions:

• Frequency: The system operates at 60Hz.
• Dimmable Range Example: If the dimmable range is set to 7500μs in an 8333μs 60Hz zone, with a 30% duty cycle, the timings would be:
  - gptimer_triacs: 5250μs
  - gptimer_triacs_all_off: 5500μs
  Even under these conditions, the issue occurs.

Observations:

• The problem persists regardless of whether the .intr_priority flag is set to 1, 2, or 3.
• The .flags.intr_shared option was tested both as true and false, with no difference in behavior.
• There are no visible errors during the setting or starting of the timers, and yet gptimer_triacs_all_off sometimes fails to fire.

Request for Guidance or any Changes to Try to Solve Addressed Issue

We are seeking clarification on whether there are known issues with GPTimer under the following conditions:

• Simultaneous re-arming of multiple GPTimers from within an ISR.
• Rapid succession of timers being re-armed and executed (e.g., within 250 microseconds of each other) with 2 or more GPTimers.

Is there any potential for a race condition or other underlying issue that could cause the second timer not to dispatch its callback? Any advice or suggested debugging steps to identify why this is happening would be greatly appreciated.

[filzek]

Startup as:
D (10410) gptimer: new group (0) @0x3ffcf090
D (10410) gptimer: new gptimer (0,0) at 0x3ffe3de0, resolution=1000000Hz
D (10410) gptimer: new gptimer (0,1) at 0x3ffe3ea4, resolution=1000000Hz

[Kainarx]

I think there is no problem with sharing resources between two separate timers. Can you provide a simple reproduction project? Also, which version of the idf and which chip are you using? I will try to reproduce it.

[filzek]

Hi, the problem is among the GPTIMER dispatch, it has a serious problem.

gptimer_config_t IRAM_ATTR timer_config_us_up = {
.clk_src = GPTIMER_CLK_SRC_DEFAULT,
.direction = GPTIMER_COUNT_UP,
.resolution_hz = 1 * 1000 * 1000, // 1MHz, 1 tick = 1us
.intr_priority = 3, //3 tons of error, 1 still with some errors
.flags.intr_shared = false,
};

timer_creation_result = gptimer_new_timer(&timer_config_us_up, &gptimer_timelapse_action);
gptimer_event_callbacks_t callback_source_timelapse = {
.on_alarm = time_lapse_action,
};
gptimer_register_event_callbacks(gptimer_timelapse_action, &callback_source_timelapse, NULL);
gptimer_alarm_config_t us_triac_dispatch2 = {
.reload_count = 0,
.alarm_count = 75,
.flags.auto_reload_on_alarm = true,
};
gptimer_set_alarm_action(gptimer_timelapse_action, &us_triac_dispatch2);
gptimer_enable(gptimer_timelapse_action);
gptimer_start(gptimer_timelapse_action);

int IRAM_ATTR onoff = 1;
static bool IRAM_ATTR time_lapse_action(gptimer_handle_t timer, const gptimer_alarm_event_data_t *edata, void *user_data) {
int64_t testing = esp_timer_get_time(); // Record the current time
xQueueSendFromISR(gpio_evt_queue_zc, &testing, NULL);

if (onoff){
GPIO.out_w1ts = (1 << 26); // Enable GPIO output high
}
else {
GPIO.out_w1tc = (1 << 26); // Clear GPIO output low
}
onoff = !onoff;
return true;
}

Ocisloscope DS1104z with 100MHz 1GSa/s
With this example running we can capture a lot of dispatch failure, timer trigger and actions with odd behavior with a 75us continous dispatch as in above code:

1. GPTIMER double act in dispatch, so its dispatch double at 5us even in a miss act.
   

2. Miss the dispatch twice concurrent
   

To have a minimum ammount of failures the highest level 1 is needed to have small ammount of error.

There is no free error GPTImer at all.

[florentbr]

@filzek
Your variables are tagged as IRAM_ATTR.
Is it not supposed to be DRAM_ATTR since they are data and not instruction ?

[filzek]

@filzek Your variables are tagged as IRAM_ATTR. Is it not supposed to be DRAM_ATTR since they are data and not instruction ?

@florentbr in this case makes no difference as the size part of the memory read and write is always exacly the same, it could have a problem with a uint/string larger type, but for int family there is no such as problem.

In the timer handle and others strucs the compiler moves it to the correct aligned part as well, it does it since sdk v3.

[Kainarx]

Hi @filzek
I tested it with your configuration and didn't find this problem. Did you configure the options CONFIG_GPTIMER_CTRL_FUNC_IN_IRAM=y and CONFIG_GPTIMER_ISR_IRAM_SAFE=y?
Beacues gptimer_timelapse_action is user's callback function which we call it in the gptimer_default_isr. We need to ensure both of them are in the IRAM.

[AxelLin]

@filzek Does the comment in #14418 (comment) help?

[filzek]

CONFIG_GPTIMER_ISR_IRAM_SAFE

@AxelLin, we have already configured it to use IRAM. The issue seems to be directly related to how the SDK manages processes. Shifting all applications, including WiFi and BLE, to core 0 and isolating only the TIMER and ISR on core 1 helps reduce the problem, but it still persists.

[Kainarx]

@filzek Can this commit 0f8e6f6 solve you problem ?

[rickou]

I have a similar problem, regularily the GPTimer is delayed (about 2 times per seconds but vary)
i configured the timer for 50kHz (priority 1)
Because i'm driving stepper motor using the GPTimer, i need a very stable timer..which is clearly not..
hope the callback is called directly from ISR ?

have you found a solution ?

[Kainarx]

How many microseconds of delay are there each time? And have you tried installing GPTimer on another core or putting your code to IRAM? @rickou

[rickou]

i haven't the measure exactly (need to get the oscilloscope) but i can heard the delay in the motor noise 😁
yes yesterday i tryed to move Wifi LWIP and some other tasks to Core 1
but without (or unsensitive) effect.
I also try to change the timer priority 1 or 3 without any effect.
Put all functions in IRAM

i think it is due to Wifi (STA mode) and/or UDP socket i use.

After many readings on internet, there is only 2 things more that i can/need do..
-I use the GPIO function to clear/set one GPIO (i need to change my code to directly update the registers)
-I also use math sqrtf function (but i haven't alternative choice to put in IRAM....)

I will try to make some measure for frequency and delay.

[rickou]

ok, so i just make some measurements.

i got quite large delay every 250ms

Detail:

during all the steps I also get some times longer than expected, for ex:

reminder, the timer ISR rate is 50kHz, and i use simple variable to toggle the pin

[rickou]

in addition, i replaced calls to GPIO function to direct register acces, replaced the sqrtf calls by my own implementation in IRAM
no changes observed..

but i still have some calls to xthal without IRAM_ATTR regarding saving float coprocessor registers, but i have no alternatives for theses functions:
-xthal_get_cpenable
-xthal_save_cp0
-xthal_set_cpenable
-xthal_restore_cp0

[suda-morris]

driving stepper motor using the GPTimer

Have you tried this example?

reminder, the timer ISR rate is 50kHz

So the alarm interval is around 20us, which is a high real time requirement IMO. I'm afraid you can hardly achieve it. The biggest problem is, we have many code that can disable the interrupt (enter critical section).

[rickou]

Have you tried this example?

Just trying to implement my own version.. 😜

reminder, the timer ISR rate is 50kHz

So the alarm interval is around 20us, which is a high real time requirement IMO. I'm afraid you can hardly achieve it. The biggest problem is, we have many code that can disable the interrupt (enter critical section).

yes, but ESP run quite faster... on 2 cores ! so 20us should not be a real problem.. ok for some small drifts..
critical sections should be very very limited in time so 20us should be an eternity regarding this.
and in my case, i get around 300us...

to be honest, i'm more comfortable with much smaller MCU like PIC or STM32. ok i don't use Wifi and OS.. but using interrupts should not be so different...