Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xTaskCreateWithCaps causing memory leak when deleting. (IDFGH-13294) #14222

Closed
3 tasks done
RoniRaad opened this issue Jul 20, 2024 · 18 comments
Closed
3 tasks done

xTaskCreateWithCaps causing memory leak when deleting. (IDFGH-13294) #14222

RoniRaad opened this issue Jul 20, 2024 · 18 comments
Assignees
Labels
Resolution: NA Issue resolution is unavailable Status: Done Issue is done internally Type: Bug bugs in IDF

Comments

@RoniRaad
Copy link

RoniRaad commented Jul 20, 2024

Answers checklist.

  • I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
  • I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
  • I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

5.2.2

Espressif SoC revision.

ESP-32-S3

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

PowerShell

Development Kit.

ESP-32-S3

Power Supply used.

USB

What is the expected behavior?

When calling vTaskDeleteWithCaps(NULL) from within the task pxTCB and task stack should be freed.

What is the actual behavior?

pxTCB and task stack are not freed and leaking DMA heap and caps memory.

I believe this is due to the last two free's in vTaskDeleteWithCaps being after a call to vTaskDelete which ends the current tasks execution not allowing those two free's to run.

Steps to reproduce.

  1. Log current available DMA memory.
  2. Create a task using xTaskCreateWithCaps and specify SPIRAM.
  3. call vTaskDeleteWithCaps(NULL) from within the task.
  4. compare the logged DMA memory to new memory, should be about 368 bytes missing.

Debug Logs.

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DOUT, clock div:1
load:0x3fce3820,len:0x18e8
load:0x403c9700,len:0x4
load:0x403c9704,len:0xe5c
load:0x403cc700,len:0x2f54
SHA-256 comparison failed:
Calculated: 1be28fcfc098f229adaf353cce7ce22ad284c9f453ea5f1f74e2614b21dc7667
Expected: ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
Attempting to boot anyway...
entry 0x403c993c
I (45) boot: ESP-IDF v5.2.2-dirty 2nd stage bootloader
I (45) boot: compile time Jul 20 2024 00:43:46
I (45) boot: Multicore bootloader
I (49) boot: chip revision: v0.2
I (53) qio_mode: Enabling default flash chip QIO
I (58) boot.esp32s3: Boot SPI Speed : 80MHz
I (63) boot.esp32s3: SPI Mode       : QIO
I (67) boot.esp32s3: SPI Flash Size : 8MB
I (72) boot: Enabling RNG early entropy source...
I (78) boot: Partition Table:
I (81) boot: ## Label            Usage          Type ST Offset   Length
I (88) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (96) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (103) boot:  2 factory          factory app      00 00 00010000 002ee000
I (111) boot: End of partition table
I (115) esp_image: segment 0: paddr=00010020 vaddr=3c030020 size=0d29ch ( 53916) map
I (129) esp_image: segment 1: paddr=0001d2c4 vaddr=3fc92500 size=02d54h ( 11604) load
I (134) esp_image: segment 2: paddr=00020020 vaddr=42000020 size=23f58h (147288) map
I (156) esp_image: segment 3: paddr=00043f80 vaddr=3fc95254 size=00384h (   900) load
I (156) esp_image: segment 4: paddr=0004430c vaddr=40374000 size=0e4d4h ( 58580) load
I (176) boot: Loaded app from partition at offset 0x10000
I (176) boot: Disabling RNG early entropy source...
I (187) cpu_start: Multicore app
I (187) flash HPM: with HPM-DC support
I (188) flash HPM: Enabling flash high speed mode by dummy
I (191) MSPI Timing: Flash timing tuning index: 2
I (196) octal_psram: vendor id    : 0x0d (AP)
I (201) octal_psram: dev id       : 0x02 (generation 3)
I (207) octal_psram: density      : 0x03 (64 Mbit)
I (212) octal_psram: good-die     : 0x01 (Pass)
I (218) octal_psram: Latency      : 0x01 (Fixed)
I (223) octal_psram: VCC          : 0x01 (3V)
I (228) octal_psram: SRF          : 0x01 (Fast Refresh)
I (234) octal_psram: BurstType    : 0x01 (Hybrid Wrap)
I (240) octal_psram: BurstLen     : 0x01 (32 Byte)
I (245) octal_psram: Readlatency  : 0x02 (10 cycles@Fixed)
I (251) octal_psram: DriveStrength: 0x00 (1/1)
I (264) MSPI Timing: PSRAM timing tuning index: 2
I (264) esp_psram: Found 8MB PSRAM device
I (267) esp_psram: Speed: 120MHz
I (282) mmu_psram: Instructions copied and mapped to SPIRAM
I (286) mmu_psram: Read only data copied and mapped to SPIRAM
I (582) esp_psram: SPI SRAM memory test OK
I (590) cpu_start: Pro cpu start user code
I (590) cpu_start: cpu freq: 240000000 Hz
I (591) cpu_start: Application information:
I (594) cpu_start: Project name:     discord_display_esp32_s3
I (600) cpu_start: App version:      6c10e98-dirty
I (605) cpu_start: Compile time:     Jul 20 2024 00:43:02
I (611) cpu_start: ELF file SHA256:  2ce72579b...
I (617) cpu_start: ESP-IDF:          v5.2.2-dirty
I (622) cpu_start: Min chip rev:     v0.0
I (627) cpu_start: Max chip rev:     v0.99
I (632) cpu_start: Chip rev:         v0.2
I (637) heap_init: Initializing. RAM available for dynamic allocation:
I (644) heap_init: At 3FC960F0 len 00039F10 (231 KiB): RAM
I (650) heap_init: At 3FCD4000 len 0000C000 (48 KiB): RAM
I (656) heap_init: At 3FCE4000 len 00005710 (21 KiB): RAM
I (662) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (668) heap_init: At 3FCF0000 len 00008000 (32 KiB): DRAM
I (674) heap_init: At 600FE010 len 00001FD8 (7 KiB): RTCRAM
I (845) esp_psram: Adding pool of 7936K of PSRAM memory to heap allocator
I (846) spi_flash: detected chip: gd
I (846) spi_flash: flash io: qio
I (850) sleep: Configure to isolate all GPIO pins in sleep state
I (857) sleep: Enable automatic switching of GPIO sleep configuration
I (00:00:00.274) main_task: Started on CPU0
I (00:00:00.279) main_task: Calling app_main()
I (00:00:00.283) Main: DMA Avalible: 326712

I (00:00:02.288) Main: DMA Avalible: 326344

I (00:00:04.288) Main: DMA Avalible: 325976

I (00:00:06.288) Main: DMA Avalible: 325608

I (00:00:08.288) Main: DMA Avalible: 325240

I (00:00:10.288) Main: DMA Avalible: 324872

I (00:00:12.288) Main: DMA Avalible: 324504

More Information.

code used to reproduce logs.

#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"

static const char* TAG = "Main";

extern "C" {


void app_main(void);
}

void insta_free_task()
{
    vTaskDeleteWithCaps(NULL);
}

void check_dma_task()
{
    while (1)
    {
        size_t dma_size = heap_caps_get_free_size(MALLOC_CAP_DMA);
        ESP_LOGI(TAG, "DMA Avalible: %u\n", dma_size);
        xTaskCreateWithCaps( (TaskFunction_t)insta_free_task, "insta_free_task", 2042*3, nullptr, tskIDLE_PRIORITY, nullptr, MALLOC_CAP_SPIRAM);
        vTaskDelay(pdMS_TO_TICKS(2000));
    }
}


void app_main(void) 
{
    check_dma_task();
}

here is my local fix that modifies vTaskDeleteWithCaps

    void vTaskDeleteWithCapsTask(void * pvParameters)
    {
        TaskHandle_t xTaskToDelete = (TaskHandle_t) pvParameters;
        vTaskDeleteWithCaps( xTaskToDelete );

        vTaskDelete(NULL);
    }

    void vTaskDeleteWithCaps( TaskHandle_t xTaskToDelete )
    {
        BaseType_t xResult;
        StaticTask_t * pxTaskBuffer;
        StackType_t * puxStackBuffer;
        TaskHandle_t xTakeCurrentHandle = xTaskGetCurrentTaskHandle();

        if ( xTaskToDelete == NULL || xTaskToDelete == xTakeCurrentHandle )
        {
            // Create a task to handle destroying the current task allowing for us to delete the stack and pxTCB below.
            if (xTaskCreate( (TaskFunction_t)vTaskDeleteWithCapsTask, "vTaskDeleteWithCapsTask", 2048, xTakeCurrentHandle, tskIDLE_PRIORITY, NULL) != pdFAIL)
            {
                // We delay indefinity as we have already created a new task to destroy the current task.
                vTaskDelay(portMAX_DELAY);
                return;
            }
        }

        xResult = xTaskGetStaticBuffers( xTaskToDelete, &puxStackBuffer, &pxTaskBuffer );
        configASSERT( xResult == pdTRUE );

        /* Delete the task */
        vTaskDelete( xTaskToDelete );

        /* Free task buffer */
        vPortFree( pxTaskBuffer );

        /* Free task buffer */
        heap_caps_free( puxStackBuffer );
    }
@RoniRaad RoniRaad added the Type: Bug bugs in IDF label Jul 20, 2024
@github-actions github-actions bot changed the title xTaskCreateWithCaps causing memory leak when deleting. xTaskCreateWithCaps causing memory leak when deleting. (IDFGH-13294) Jul 20, 2024
@espressif-bot espressif-bot added the Status: Opened Issue is new label Jul 20, 2024
@KonstantinKondrashov
Copy link
Collaborator

Hi @RoniRaad!
Yes, we are aware that vTaskDeleteWithCaps can not be done for the calling task. As I remember we did not come to a conclusion about the way to resolve this issue. But there is a workaround, you need to call the Delete from another task.

TaskHandle_t check_dma_task_handler;

void task1() {
    ...
    xTaskCreateWithCaps( (TaskFunction_t)insta_free_task, "insta_free_task", 2042*3, nullptr, tskIDLE_PRIORITY, check_dma_task_handler, MALLOC_CAP_SPIRAM);
}

void task2(){
    ...
    vTaskDeleteWithCaps(check_dma_task_handler);
}

@RoniRaad
Copy link
Author

RoniRaad commented Jul 21, 2024

Hi @KonstantinKondrashov,
Is it possible to mention this quirk in the documentation for now? I was going crazy looking in my code for a missing free

@espressif-bot espressif-bot added Status: Done Issue is done internally Resolution: NA Issue resolution is unavailable and removed Status: Opened Issue is new labels Jul 29, 2024
ivan-ljubicic pushed a commit to ivan-ljubicic/esp-idf that referenced this issue Jul 30, 2024
vTaskDeleteWithCaps() leaked memory when a task uses the API to delete
itself. This commit adds a fix to avoid the memory leak.

Closes espressif#14222

(cherry picked from commit c3da2ac)
ivan-ljubicic pushed a commit to ivan-ljubicic/esp-idf that referenced this issue Jul 30, 2024
vTaskDeleteWithCaps() leaked memory when a task uses the API to delete
itself. This commit adds a fix to avoid the memory leak.

Closes espressif#14222

(cherry picked from commit c3da2ac)
espressif-bot pushed a commit that referenced this issue Aug 2, 2024
vTaskDeleteWithCaps() leaked memory when a task uses the API to delete
itself. This commit adds a fix to avoid the memory leak.

Closes #14222
espressif-bot pushed a commit that referenced this issue Aug 7, 2024
vTaskDeleteWithCaps() leaked memory when a task uses the API to delete
itself. This commit adds a fix to avoid the memory leak.

Closes #14222
@ermacv
Copy link

ermacv commented Sep 5, 2024

Hi @KonstantinKondrashov ! Applied fix ruined my application logic because I sent a signal to task when it should wrap up, free resources and self delete. Is there a plan to enable tasks with caps to be self deleted? Or it is not possible?

@RoniRaad
Copy link
Author

RoniRaad commented Sep 5, 2024

@ermacv What is the current fix breaking in your application? If the task receives a signal to self delete it should create a sub task to do it allowing the resources to be freed as implemented in this fix for vTaskDeleteWithCaps(NULL)

@ermacv
Copy link

ermacv commented Sep 5, 2024

Hi @RoniRaad ! When I want to stop the task I set an event flag and wait for response. It looks like this:

if (bits & EVENT_TASK_STOP_REQUEST) {
  xEventGroupSetBits(icfg.flags_sync, EVENT_ACTION_DONE);
  vTaskDeleteWithCaps(NULL);
}

But now this assertion is triggered:

assert failed: 0x40396138
0x40396138: __assert_func at /home/ermacv/esp/esp-idf/components/newlib/assert.c:37



Backtrace: 0x40375bfe:0x3fcdb350 0x40385655:0x3fcdb380 0x4039613d:0x3fcdb3b0 0x421441bb:0x3fcdb400
0x40375bfe: panic_abort at /home/ermacv/esp/esp-idf/components/esp_system/panic.c:463
0x40385655: esp_system_abort at /home/ermacv/esp/esp-idf/components/esp_system/port/esp_system_chip.c:92
0x4039613d: __assert_func at /home/ermacv/esp/esp-idf/components/newlib/assert.c:39
0x421441bb: prvTaskDeleteWithCapsTask at /home/ermacv/esp/esp-idf/components/freertos/esp_additions/idf_additions.c:91 (discriminator 1)

And it is related to this code that was added in the recent patch:

/* The task to be deleted must not be running */
configASSERT( eRunning != eTaskGetState( xTaskToDelete ) );

@ermacv
Copy link

ermacv commented Sep 5, 2024

@ermacv What is the current fix breaking in your application? If the task receives a signal to self delete it should create a sub task to do it allowing the resources to be freed as implemented in this fix for vTaskDeleteWithCaps(NULL)

But who will delete the second task?))

@RoniRaad
Copy link
Author

RoniRaad commented Sep 5, 2024

@ermacv looks like it could be a bug in the new code, the vTaskDeleteWithCaps should suspend the task. You could try calling vTaskSuspend(NULL) right after vTaskDeleteWithCaps, but regardless I think this would be a new bug ticket

@RoniRaad
Copy link
Author

RoniRaad commented Sep 5, 2024

@ermacv What is the current fix breaking in your application? If the task receives a signal to self delete it should create a sub task to do it allowing the resources to be freed as implemented in this fix for vTaskDeleteWithCaps(NULL)

But who will delete the second task?))

Second task is created without caps allowing for it to delete itself without requiring a sub task

@sudeep-mohanty
Copy link
Collaborator

@ermacv In your use-case, are you sending an event to the task which must be deleted? I would like to understand the scenario before we can investigate this. It would be good if you could open a new ticket for us, preferably with a minimal demo which could help us reproduce the problem you are facing. Thank you and apologies for the inconvenience!

@SoucheSouche SoucheSouche reopened this Sep 5, 2024
@SoucheSouche SoucheSouche removed the Status: Done Issue is done internally label Sep 5, 2024
@espressif-bot espressif-bot added Status: Opened Issue is new Status: Done Issue is done internally Resolution: Done Issue is done internally and removed Status: Opened Issue is new labels Sep 5, 2024
@KonstantinKondrashov
Copy link
Collaborator

@ermacv
I am wondering If you do not call the xEventGroupSetBits before vTaskDeleteWithCaps the issue persists or not. Maybe the Event does not allow it to go into a suspended state. Could you try it? Thanks.

@ermacv
Copy link

ermacv commented Sep 5, 2024

Thank you all for response! I will check your comments and respond you.

Another question. Will it be ok to stop the task this way?

Task code:

if (bits & EVENT_TASK_STOP_REQUEST) {
  xEventGroupSetBits(icfg.flags_sync, EVENT_ACTION_DONE);
  // block task with suspend
  vTaskSuspend(NULL);
  // or infinite delay
  vTaskDelay(portMAX_DELAY);
}

Caller of the task stop:

xEventGroupSetBits(icfg.flags_sync, EVENT_TASK_STOP_REQUEST);
xEventGroupWaitBits(icfg.flags_sync, EVENT_ACTION_DONE, pdTRUE, pdFALSE, portMAX_DELAY);
vTaskDeleteWithCaps(task_handle);

I mean if vTaskDeleteWithCaps function expects that task should be suspended before deleting is caller code ok? Or vTaskDeleteWithCaps also calls vTaskSuspend forcibly for the requested handle?

@KonstantinKondrashov
Copy link
Collaborator

KonstantinKondrashov commented Sep 5, 2024

@ermacv
It is a preferable way to delete the task with caps from another task. Your changes will work.
In this case, the vTaskDeleteWithCaps does not create an additional task for deletion, it will delete the task even if it is running just waiting when it is stopped (this function has a condition for this case).

Could you give some info about your case:

  • is the Task code task pinned to a certain core or not?
  • what chip do you use?
  • How many CPUs are used?
    We want to reproduce it on your side and fix it.
    Thanks.

@ermacv
Copy link

ermacv commented Sep 5, 2024

@ermacv It is a preferable way to delete the task with caps from another task. Your changes will work. In this case, the vTaskDeleteWithCaps does not create an additional task for deletion, it will delete the task even if it is running just waiting when it is stopped (this function has a condition for this case).

Could you give some info about your case:

  • is the Task code task pinned to a certain core or not?
  • what chip do you use?
  • How many CPUs are used?
    We want to reproduce it on your side and fix it.
    Thanks.
  1. Tasks are not pinned to any core
  2. ESP32-S3 with 2 Mb PSRAM
  3. Both 2 cores are used

@sudeep-mohanty
Copy link
Collaborator

Hello @ermacv,
I do not see a crash if I perform an operation similar to the one you mentioned here, i.e, send to an event group and then self-deleting a task created WithCaps. Would you mind sharing a minimal reproducible code with us so that we can reproduce the problem at our side?

@ermacv
Copy link

ermacv commented Sep 9, 2024

Hi @sudeep-mohanty !
Sorry I was a bit busy and it was tricky to create a reproducible example. Please check the attachment.
I guess the main thing with my application was that the issue happened when a lot of tasks were deleted at the same time, so they definitely worked on a different cores with different priorities and they were deleted from different (main) tasks. So I guess I made something similar in example.

main_delete_in_main_task.c - Task is deleted from another task. Works fine as expected.
main_delete_in_sub_main_task.c - Task is deleted from itself. It ends up with:


assert failed: prvTaskDeleteWithCapsTask idf_additions.c:91 (eRunning != eTaskGetState( xTaskToDelete ))


Backtrace: 0x40375f0e:0x3fca3a70 0x4037d255:0x3fca3aa0 0x4038abf2:0x3fca3ad0 0x42017471:0x3fca3c30
0x40375f0e: panic_abort at /home/ermacv/esp/esp-idf/components/esp_system/panic.c:463
0x4037d255: esp_system_abort at /home/ermacv/esp/esp-idf/components/esp_system/port/esp_system_chip.c:92
0x4038abf2: __assert_func at /home/ermacv/esp/esp-idf/components/newlib/assert.c:80
0x42017471: prvTaskDeleteWithCapsTask at /home/ermacv/esp/esp-idf/components/freertos/esp_additions/idf_additions.c:91 (discriminator 1)

I also attached the sdkconfig with options that are enabled in my application.

task_delete_bug.zip

@ermacv
Copy link

ermacv commented Sep 9, 2024

Hm, I didn't notice this comment in the code:

             * 2. This approach is wasteful and can be error prone. The temporary clean up task will need
             *    system resources to get scheduled and cleanup the WithCaps task. It can be a problem if the system
             *    has several self-deleting WithCaps tasks.

I guess my issue could be related to the last sentence in this comment.

@sudeep-mohanty
Copy link
Collaborator

Thanks @ermacv for the code. I shall investigate and post my findings.

@espressif-bot espressif-bot added Status: Opened Issue is new and removed Status: Done Issue is done internally Resolution: Done Issue is done internally labels Sep 11, 2024
@sudeep-mohanty
Copy link
Collaborator

Hi @ermacv, Thank you for sharing your code. I could reproduce the issue at my end with some modifications to it. Indeed, the assert failure happens when the system is overloaded and there are multiple unpinned tasks which could get scheduled on the other core before they are deleted. The best way to avoid this issue is what you are already doing, i.e, delete such tasks from another task's context instead of deleting themselves. In case you still have a case of self-deletion, please apply this patch and re-try. Thanks.
taskdeletewithcaps.patch

@espressif-bot espressif-bot added Status: In Progress Work is in progress and removed Status: Opened Issue is new labels Sep 12, 2024
espressif-bot pushed a commit that referenced this issue Sep 12, 2024
vTaskDeleteWithCaps() leaked memory when a task uses the API to delete
itself. This commit adds a fix to avoid the memory leak.

Closes #14222
@espressif-bot espressif-bot added Status: Reviewing Issue is being reviewed and removed Status: In Progress Work is in progress labels Sep 13, 2024
@espressif-bot espressif-bot added Status: Done Issue is done internally and removed Status: Reviewing Issue is being reviewed labels Sep 20, 2024
espressif-bot pushed a commit that referenced this issue Sep 22, 2024
This commit fixes an assert failure in vTaskDeleteWithCaps() when
multiple un-pinned tasks are created with stack in the external memory
and such tasks delete themselves.

Closes #14222
espressif-bot pushed a commit that referenced this issue Oct 16, 2024
This commit fixes an assert failure in vTaskDeleteWithCaps() when
multiple un-pinned tasks are created with stack in the external memory
and such tasks delete themselves.

Closes #14222
espressif-bot pushed a commit that referenced this issue Oct 18, 2024
This commit fixes an assert failure in vTaskDeleteWithCaps() when
multiple un-pinned tasks are created with stack in the external memory
and such tasks delete themselves.

Closes #14222
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Resolution: NA Issue resolution is unavailable Status: Done Issue is done internally Type: Bug bugs in IDF
Projects
None yet
Development

No branches or pull requests

6 participants