Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions doc/develop/languages/c/common_libc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
.. _c_library_common:

Common C library code
#####################

Zephyr provides some C library functions that are designed to be used in
conjunction with multiple C libraries. These either provide functions not
available in multiple C libraries or are designed to replace functionality
in the C library with code better suited for use in the Zephyr environment

Time function
*************

This provides an implementation of the standard C function, :c:func:`time`,
relying on the Zephyr function, :c:func:`clock_gettime`. This function can
be enabled by selecting :kconfig:option:`COMMON_LIBC_TIME`.

Dynamic Memory Management
*************************

The common dynamic memory management implementation can be enabled by
selecting the :kconfig:option:`CONFIG_COMMON_LIBC_MALLOC` in the
application configuration file.

The common C library internally uses the :ref:`kernel memory heap API
<heap_v2>` to manage the memory heap used by the standard dynamic memory
management interface functions such as :c:func:`malloc` and :c:func:`free`.

The internal memory heap is normally located in the ``.bss`` section. When
userspace is enabled, however, it is placed in a dedicated memory partition
called ``z_malloc_partition``, which can be accessed from the user mode
threads. The size of the internal memory heap is specified by the
:kconfig:option:`CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE`.

The default heap size for applications using the common C library is zero
(no heap). For other C library users, if there is an MMU present, then the
default heap is 16kB. Otherwise, the heap uses all available memory.

There are also separate controls to select :c:func:`calloc`
(:kconfig:option:`COMMON_LIBC_CALLOC`) and :c:func:`reallocarray`
(:kconfig:option:`COMMON_LIBC_REALLOCARRAY`). Both of these are enabled by
default as that doesn't impact memory usage in applications not using them.

The standard dynamic memory management interface functions implemented by
the common C library are thread safe and may be simultaneously called by
multiple threads. These functions are implemented in
:file:`lib/libc/common/source/stdlib/malloc.c`.
1 change: 1 addition & 0 deletions doc/develop/languages/c/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ application.
.. toctree::
:maxdepth: 2

common_libc.rst
minimal_libc.rst
newlib.rst
picolibc.rst
Expand Down
21 changes: 3 additions & 18 deletions doc/develop/languages/c/minimal_libc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,24 +29,9 @@ service documentation.
Dynamic Memory Management
*************************

Dynamic memory management in the minimal libc can be enabled by selecting the
:kconfig:option:`CONFIG_MINIMAL_LIBC_MALLOC` in the application configuration
file.

The minimal libc internally uses the :ref:`kernel memory heap API <heap_v2>` to
manage the memory heap used by the standard dynamic memory management interface
functions such as :c:func:`malloc` and :c:func:`free`.

The internal memory heap is normally located in the ``.bss`` section. When
userspace is enabled, however, it is placed in a dedicated memory partition
called ``z_malloc_partition``, which can be accessed from the user mode
threads. The size of the internal memory heap is specified by the
:kconfig:option:`CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE`.

The standard dynamic memory management interface functions implemented by the
minimal libc are thread safe and may be simultaneously called by multiple
threads. These functions are implemented in
:file:`lib/libc/minimal/source/stdlib/malloc.c`.
The minimal libc uses the malloc api family implementation provided by the
:ref:`common C library <c_library_common>`, which itself is built upon the
:ref:`kernel memory heap API <heap_v2>`.

Error numbers
*************
Expand Down
32 changes: 24 additions & 8 deletions include/zephyr/sys/libc-hooks.h
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,31 @@ __syscall size_t zephyr_fwrite(const void *ZRESTRICT ptr, size_t size,
size_t nitems, FILE *ZRESTRICT stream);
#endif /* CONFIG_NEWLIB_LIBC */

/* Handle deprecated malloc arena size configuration values */
#ifdef CONFIG_COMMON_LIBC_MALLOC
# if CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE == 0
# undef CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE
# ifdef CONFIG_MINIMAL_LIBC
# define CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE
# else
# define CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE 0
# endif
# endif
#endif

#ifdef CONFIG_USERSPACE
#if defined(CONFIG_NEWLIB_LIBC)
#ifdef CONFIG_COMMON_LIBC_MALLOC

/* When using the common malloc implementation with CONFIG_USERSPACE, the
* heap will be in a separate partition when there's an MPU or MMU
* available.
*/
#if CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE != 0 && \
(defined(CONFIG_MPU) || defined(CONFIG_MMU))
#define Z_MALLOC_PARTITION_EXISTS 1
#endif

#elif defined(CONFIG_NEWLIB_LIBC)
/* If we are using newlib, the heap arena is in one of two areas:
* - If we have an MPU that requires power of two alignment, the heap bounds
* must be specified in Kconfig via CONFIG_NEWLIB_LIBC_ALIGNED_HEAP_SIZE.
Expand All @@ -54,13 +77,6 @@ __syscall size_t zephyr_fwrite(const void *ZRESTRICT ptr, size_t size,
#define Z_MALLOC_PARTITION_EXISTS 1
extern struct k_mem_partition z_malloc_partition;
#endif
#elif defined(CONFIG_MINIMAL_LIBC)
#if (CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE > 0)
/* Minimal libc by default has no malloc arena, its size must be set in
* Kconfig via CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE
*/
#define Z_MALLOC_PARTITION_EXISTS 1
#endif /* CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE > 0 */

#elif defined(CONFIG_PICOLIBC)
/*
Expand Down
2 changes: 2 additions & 0 deletions lib/libc/common/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# SPDX-License-Identifier: Apache-2.0

zephyr_library()
zephyr_library_property(ALLOW_EMPTY TRUE)
zephyr_library_sources_ifdef(CONFIG_COMMON_LIBC_TIME source/time/time.c)
zephyr_library_sources_ifdef(CONFIG_COMMON_LIBC_MALLOC source/stdlib/malloc.c)
32 changes: 32 additions & 0 deletions lib/libc/common/Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,35 @@ config COMMON_LIBC_TIME
bool
help
common implementation of time().

config COMMON_LIBC_MALLOC
bool "Common C library malloc implementation"
help
Common implementation of malloc family that uses the kernel heap
API.

config COMMON_LIBC_MALLOC_ARENA_SIZE
int "Size of the common C library malloc arena"
depends on COMMON_LIBC_MALLOC
default 0
help
Indicate the size in bytes of the memory arena used for
common C library malloc() implementation.

config COMMON_LIBC_CALLOC
bool "Common C library calloc"
depends on COMMON_LIBC_MALLOC
default n if MINIMAL_LIBC && !MINIMAL_LIBC_CALLOC
default y
help
Enable the common C library trivial implementation of calloc,
which forwards to malloc and memset.

config COMMON_LIBC_REALLOCARRAY
bool "Common C library reallocarray"
depends on COMMON_LIBC_MALLOC
default n if MINIMAL_LIBC && !MINIMAL_LIBC_REALLOCARRAY
default y
help
Enable the common C library trivial implementation of
reallocarray, which forwards to realloc.
Original file line number Diff line number Diff line change
Expand Up @@ -13,23 +13,24 @@
#include <zephyr/app_memory/app_memdomain.h>
#include <zephyr/sys/mutex.h>
#include <zephyr/sys/sys_heap.h>
#include <zephyr/sys/libc-hooks.h>
#include <zephyr/types.h>

#define LOG_LEVEL CONFIG_KERNEL_LOG_LEVEL
#include <zephyr/logging/log.h>
LOG_MODULE_DECLARE(os, CONFIG_KERNEL_LOG_LEVEL);

#ifdef CONFIG_MINIMAL_LIBC_MALLOC
#ifdef CONFIG_COMMON_LIBC_MALLOC

#if (CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE > 0)
#if (CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE > 0)
#ifdef CONFIG_USERSPACE
K_APPMEM_PARTITION_DEFINE(z_malloc_partition);
#define POOL_SECTION K_APP_DMEM_SECTION(z_malloc_partition)
#else
#define POOL_SECTION .bss
#endif /* CONFIG_USERSPACE */

#define HEAP_BYTES CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE
#define HEAP_BYTES CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE

Z_GENERIC_SECTION(POOL_SECTION) static struct sys_heap z_malloc_heap;
Z_GENERIC_SECTION(POOL_SECTION) struct sys_mutex z_malloc_heap_mutex;
Expand Down Expand Up @@ -121,7 +122,7 @@ void *malloc(size_t size)
{
ARG_UNUSED(size);

LOG_ERR("CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE is 0");
LOG_ERR("CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE is 0");
errno = ENOMEM;

return NULL;
Expand All @@ -139,9 +140,9 @@ void *realloc(void *ptr, size_t size)
}
#endif

#endif /* CONFIG_MINIMAL_LIBC_MALLOC */
#endif /* CONFIG_COMMON_LIBC_MALLOC */

#ifdef CONFIG_MINIMAL_LIBC_CALLOC
#ifdef CONFIG_COMMON_LIBC_CALLOC
void *calloc(size_t nmemb, size_t size)
{
void *ret;
Expand All @@ -159,19 +160,15 @@ void *calloc(size_t nmemb, size_t size)

return ret;
}
#endif /* CONFIG_MINIMAL_LIBC_CALLOC */
#endif /* CONFIG_COMMON_LIBC_CALLOC */

#ifdef CONFIG_MINIMAL_LIBC_REALLOCARRAY
#ifdef CONFIG_COMMON_LIBC_REALLOCARRAY
void *reallocarray(void *ptr, size_t nmemb, size_t size)
{
#if (CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE > 0)
if (size_mul_overflow(nmemb, size, &size)) {
errno = ENOMEM;
return NULL;
}
return realloc(ptr, size);
#else
return NULL;
#endif
}
#endif /* CONFIG_MINIMAL_LIBC_REALLOCARRAY */
#endif /* CONFIG_COMMON_LIBC_REALLOCARRAY */
1 change: 0 additions & 1 deletion lib/libc/minimal/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@ zephyr_library_sources(
source/stdlib/strtoul.c
source/stdlib/strtoll.c
source/stdlib/strtoull.c
source/stdlib/malloc.c
source/stdlib/bsearch.c
source/stdlib/exit.c
source/stdlib/qsort.c
Expand Down
22 changes: 17 additions & 5 deletions lib/libc/minimal/Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -14,31 +14,43 @@ config MINIMAL_LIBC_NON_REENTRANT_FUNCTIONS
option may require an additional memory protection region.

config MINIMAL_LIBC_MALLOC
bool "Minimal libc malloc implementation"
bool "[DEPRECATED] Minimal libc malloc implementation"
default y
imply COMMON_LIBC_MALLOC
help
[DEPRECATED] Use COMMON_LIBC_MALLOC

Enable the minimal libc's implementation of malloc, free, and realloc.
Disable if you wish to provide your own implementations of these functions.

config MINIMAL_LIBC_MALLOC_ARENA_SIZE
int "Size of the minimal libc malloc arena"
int "[DEPRECATED] Size of the minimal libc malloc arena"
default 0
depends on MINIMAL_LIBC_MALLOC
depends on COMMON_LIBC_MALLOC
help
[DEPRECATED] Use COMMON_LIBC_MALLOC_ARENA_SIZE

If set to zero, then the value of COMMON_LIBC_MALLOC_ARENA_SIZE
will be used.

Indicate the size in bytes of the memory arena used for
minimal libc's malloc() implementation.

config MINIMAL_LIBC_CALLOC
bool "Minimal libc trivial calloc implementation"
bool "[DEPRECATED] Minimal libc trivial calloc implementation"
default y
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To deprecate these, they must not be selected by default, and they must select DEPRECATED

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how to make that compatible with existing uses where this item is enabled by default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't COMMON_LIBC_CALLOC a direct replacement for this, i.e. users would notice no difference by switching? If so, these can be unselected by default and the new ones selected.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that doesn't seem aligned with the Zephyr deprecation guidelines which ask that existing uses remain supported for two releases -- existing configurations selecting CONFIG_MINIMAL_LIBC_CALLOC=n should have calloc disabled, even if they don't include CONFIG_COMMON_LIBC_CALLOC=n, right?

Configurations without any setting for CONFIG_MINIMAL_LIBC_CALLOC or CONFIG_COMMON_LIBC_CALLOC should have calloc enabled. Suggestions on how to achieve that while MINIMAL_LIBC_CALLOC doesn't use default y would be most appreciated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People can still use the existing method, they will just get a deprecation warning when they configure the build, the default configuration does not enable any deprecated options, it would move to the new options. If the existing way is default y for the old Kconfig, then use the same for the new or default y if !MINIMAL_LIBC_MALLOC, and do not default y for the old options

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These minimal libc options are a bit unusual -- they would only be specifically configured as 'n', never 'y'. If the default is set to 'n', how do we detect an explicit setting of 'n' so that the existing configuration would continue to work?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new system should be a drop in replacement for the old, if people want to use the old system, it still needs to be possible, but I would say allowing them to de-select the new options and select the old options is a good enough compromise, that will give them the deprecated option notice, and then after 2 releases the old options can be removed entirely. The new system can be selected by default (I thought the idea was to switch to picolibc as the default anyway?)

Copy link
Contributor Author

@keith-packard keith-packard Apr 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how to meet your goals and still keep existing configurations working. In particular, how would an existing configuration which selects CONFIG_MINIMAL_LIBC_CALLOC=n but does not include CONFIG_COMMON_LIBC_CALLOC=n work?

We must be able to distinguish configurations which set CONFIG_MINIMAL_LIBC_CALLOC=n from configurations which have no value at all, and I don't know how to do that without using a default value of y.

When I found these two configuration options, I was a bit mystified -- I can't see how they are useful in practice. An application not using calloc or reallocarray would benefit only in a tiny reduction in compilation speed. Applications using these functions would need to implement their own versions, and as they must be interoperable with the provided malloc and free implementations, they can't really do anything other than what the provided versions do.

So, maybe we should take this opportunity to simply remove this functionality? How can we deprecate a boolean setting for which the only useful configuration option is to set it to the value 'n'?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for moving to picolibc by default, this is part of that effort -- we want to switch picolibc configurations to use the system allocator instead of the picolibc malloc implementation. Having two separate allocators in the system isn't a good use of resources.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tejlmand for comment/review

help
[DEPRECATED] Use COMMON_LIBC_CALLOC

Enable the minimal libc's trivial implementation of calloc, which
forwards to malloc and memset.

config MINIMAL_LIBC_REALLOCARRAY
bool "Minimal libc trivial reallocarray implementation"
bool "[DEPRECATED] Minimal libc trivial reallocarray implementation"
default y
help
[DEPRECATED] Use COMMON_LIBC_REALLOCARRAY

Enable the minimal libc's trivial implementation of reallocarray, which
forwards to realloc.

Expand Down
2 changes: 1 addition & 1 deletion samples/basic/hash_map/Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ config TEST_LIB_HASH_MAP_MAX_ENTRIES
heap memory. For test scenarios using the Minimal C library, the heap
size is controlled via

CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE

For native_posix_64, the number of entries can be configured
independently of the arena size since the native libc is used.
Expand Down
2 changes: 1 addition & 1 deletion samples/basic/hash_map/prj.conf
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ CONFIG_BOOT_BANNER=n
CONFIG_LOG=y

CONFIG_TEST_RANDOM_GENERATOR=y
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_NEWLIB_LIBC_MIN_REQUIRED_HEAP_SIZE=8192
CONFIG_PICOLIBC_HEAP_SIZE=8192

Expand Down
2 changes: 1 addition & 1 deletion samples/bluetooth/peripheral_hr/prj_minimal.conf
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ CONFIG_ISR_STACK_SIZE=1024

# Disable features not needed
CONFIG_TIMESLICING=n
CONFIG_MINIMAL_LIBC_MALLOC=n
CONFIG_COMMON_LIBC_MALLOC=n
CONFIG_LOG=n
CONFIG_ASSERT=n

Expand Down
2 changes: 1 addition & 1 deletion samples/cpp/cpp_synchronization/prj.conf
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
CONFIG_CPP=y
CONFIG_CPP_MAIN=y
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=128
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=128
4 changes: 2 additions & 2 deletions samples/net/cloud/tagoio_http_post/prj.conf
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
CONFIG_MAIN_STACK_SIZE=4096

# C Library
# Default use minimal libc and with MINIMAL_LIBC_MALLOC_ARENA_SIZE defining
# Default use minimal libc and with COMMON_LIBC_MALLOC_ARENA_SIZE defining
# HEAP size (512 bytes) are enough to run DNS.
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=4096
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=4096

# Networking config
CONFIG_NETWORKING=y
Expand Down
2 changes: 1 addition & 1 deletion samples/subsys/fs/littlefs/boards/nucleo_h743zi.conf
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ CONFIG_FLASH_MAP=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FLASH_STM32_QSPI=y

CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=8192

# Littlefs configuration to utilize QSPI operation
CONFIG_FS_LITTLEFS_CACHE_SIZE=256
Expand Down
2 changes: 1 addition & 1 deletion samples/subsys/fs/littlefs/boards/nucleo_h743zi_blk.conf
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@
# SPDX-License-Identifier: Apache-2.0
#
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=8192

CONFIG_FS_LITTLEFS_FC_HEAP_SIZE=8192
2 changes: 1 addition & 1 deletion samples/subsys/usb/mass/boards/stm32l562e_dk_ns.conf
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# SPDX-License-Identifier: Apache-2.0
#
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=8192

CONFIG_DISK_ACCESS=y
CONFIG_DISK_DRIVERS=y
Expand Down
2 changes: 1 addition & 1 deletion tests/lib/cpp/cxx/prj.conf
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ CONFIG_CPP=y
CONFIG_NET_BUF=y
CONFIG_ZTEST=y
CONFIG_ZTEST_STACK_SIZE=2048
CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=128
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=128
CONFIG_ZTEST_NEW_API=y
CONFIG_CRC=y

Expand Down
2 changes: 1 addition & 1 deletion tests/lib/hash_map/Kconfig
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ config TEST_LIB_HASH_MAP_MAX_ENTRIES
heap memory. For test scenarios using the Minimal C library, the heap
size is controlled via

CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE

For native_posix_64, the number of entries can be configured
independently of the arena size since the native libc is used.
Expand Down
2 changes: 1 addition & 1 deletion tests/lib/hash_map/prj.conf
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
CONFIG_ZTEST=y
CONFIG_ZTEST_NEW_API=y

CONFIG_MINIMAL_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_COMMON_LIBC_MALLOC_ARENA_SIZE=8192
CONFIG_NEWLIB_LIBC_MIN_REQUIRED_HEAP_SIZE=8192
CONFIG_PICOLIBC_HEAP_SIZE=8192

Expand Down
Loading