Sleep rework, RTOS API for bare metal, wait deprecations #10104
Conversation
ciarmcom
requested review from
bridadan,
bulislaw,
c1728p9,
geky,
sg-,
theotherjimmy and
a team
|
@kjbracey-arm Love the concept. I'm thinking that adding |
|
@kjbracey-arm see that |
Doesn't work in general, unfortunately. The ARM architecture doesn't let you just run arbitrary code between LDREX and STREX - any intervening memory writes (such as storing stuff to the stack) could make the STREX fail, potentially causing a deadlock. Thus the C code as it currently stands is already technically dodgy, as we can't be sure the compiler won't be putting any stores in, and a compiler warning about the dodginess is being suppressed with a pragma. LDREX/STREX should only really be used in assembler - the C intrinsics are deprecated. Putting in a callback would greatly increase the chance of failure at low optimisation levels as it would be inclined to push a return address to the stack. So, if I were to do anything, I would be taking the LDREX/STREX intrinsics out, converting either to inline assembler or complete op compiler intrinsics. Compilers do these days have intrinsics for these ops, but afaict, they insist on them being SMP-compatible, thus adding DMB instructions we don't need. Certainly it would make sense to reduce boilerplate by using macros to generate the code chunks. The C/C++ general pattern for arbitrary ops, which doesn't suffer from the LDREX/STREX problem is: There the LDREX/STREX pair is contained in the exchange, and you can have your code as complex as you like in the loop. I guess that construct could be wrapped into a "callback" pattern, but I don't fancy inventing a new style different from the standard. |
Should do. The only static things are the "thread flags" for the main thread, and the SysTimer, and they should be stripped if no-one uses them. There is a code cost of actually doing the sleep - when you change blinky from
In a bootloader type build or secure library, you'd presumably never call sleep or anything time related, so it should all stay out. |
kjbracey
force-pushed
the
sleep_api
branch
3 times, most recently
from
7301f41 to
5a58612
Compare
|
Junked separate "rtos-impl" directory - no significant file restructuring in this version. Originally I thought I was unable to nest "rtos-impl" inside "rtos", but it was just operator error. |
👍 Once we get concept agreed, split would help to few PRs (fixes can go separately - first 4 commits are good fixes on their own).
+1 for I am interested in learning about RTOS API for baremetal builds - what subset we want to provide? What is 3b7ddc5ceab29ffc96006c3cc58361f7be313f7f achieving ? RTOS API for baremetal - two worlds unite? It's large change, commit msg might help to understand the goal. Do we have design document for this RTOS API? (would do some reading before I dig into implementation details) |
|
No design doc as yet, this is kind of the proposal. I've filled in the message for the RTOS API commit, which should help. The wait/sleep clarification is associated with https://jira.arm.com/browse/ISGDEVPREQ-1083 |
|
@kjbracey-arm In the future be aware that nested libraries are designed to function independently WRT "requires". If you notice a corner case where this does not work, let me know. |
|
Started splitting a bunch of the minor fixes into separate PRs. |
That explains it. |
Provide partial RTOS API for bare metal builds - things that can be done in a single threaded environment. Allows more code to work in both RTOS and bare metal builds without change, and in particular gives easy access to the ability to efficiently wait for something occurring in interrupt. Available in bare-metal: * ThisThread * osThreadFlagsSet to set flags on main thread (can be set from IRQ) * EventFlags (can be set from IRQ) * Semaphores (can be released from IRQ) * Mutex (dummy implementation) Not useful: * ConditionVariable (could only be signalled from 2nd thread) * RtosTimer (calls in a second thread context) * Thread Unimplemented: * Mail, Queue, MemoryPool Possible future work: * ConditionVariableCS to act as IRQ signalled ConditionVariable
Switch from CMSIS-RTOS to mbed C++ API, which are available in bare metal build. Other minor tidies, like removing unnecessary volatile.
Revert back to older behaviour where we hold deep sleep lock only while timing a sleep. Previous version was a speed optimisation, but broke some tests.
Kernel::get_ms_count is documented as not working from IRQ. In RTOS builds it can return misleading answers - see ARM-software/CMSIS_5#625 In non-RTOS builds, it can trigger an assert, as it upsets the sleep logic. Modified code is still not ideal - could be improved further if there was a fast path for "post now" that didn't bother looking at timers (both at post time and dispatch time).
Some build errors seen with NULL not being defined. Rather than add cstddef includes, switch to C++11 nullptr.
|
CI restarted |
Test run: FAILEDSummary: 2 of 11 test jobs failed Failed test jobs:
|
Previous fix assumed OsTimer is in use - it is for tickless RTOS builds and non-RTOS builds, but non-tickless RTOS builds avoid it altogether and use the SysTick peripheral. Restore previous behaviour for those builds, and just always read the tick count - there is no downside to this when ticking. While in the code, make equeue_tick initialisation explicit. This was problem if the OsTimer is not yet initialised when th done while debugging - turns out not to be part of the fix, but it speeds up the runtime code.
|
CI restarted - last fix broke non-tickless builds like the K66F in CI. |
Test run: SUCCESSSummary: 11 of 11 test jobs passed |
|
Lingering "change requested" was from @bulislaw, think it's resolved. Could you approve? |
|
Hi @kjbracey-arm , This PR is ancient so I'm not sure what I'm expecting you to do or say but any thoughts would be greatly appreciated. We've been using a version of mbed-os circa Jan 19. We recently decided to do a bit of work on the project and thought it might be a good idea to get mbed-os up to date. I'm beginning to think this was a bad idea because it is proving very hard work. Naturally just trying to get the latest version causes a storm of errors so we've been trying to go forward a few PRs at a time. We've got stuck on this PR because it increases the code size and causes the application to behave erratically. The target is an STM32L4 with If I apply this PR the .text section increases by 768 bytes which makes me think functionality is changing which doesn't seem to be the aim of the PR. I'm wondering if I need to add something to the config? Here's some of the memory map which show the .text changes. | Module | .text | .data | .bss | |-------------------------------------------------------------------------------------------------------------|--------------|----------|-----------| ... | [fill] | 264(-22) | 10(+0) | 85(-7) | ... | mbed-os\events\EventQueue.o | 100(+0) | 0(+0) | 0(+0) | | mbed-os\events\equeue\equeue.o | 1326(+4) | 0(+0) | 0(+0) | | mbed-os\events\equeue\equeue_mbed.o | 110(+16) | 0(+0) | 0(+0) | ... | mbed-os\hal\mbed_ticker_api.o | 1150(+0) | 0(+0) | 0(+0) | | mbed-os\hal\mbed_us_ticker_api.o | 30(+0) | 4(+0) | 64(+0) | | mbed-os\platform\SysTimer.o | 818(+818) | 0(+0) | 0(+0) | | mbed-os\platform\TARGET_CORTEX_M\TOOLCHAIN_GCC\except.o | 204(+0) | 0(+0) | 0(+0) | | mbed-os\platform\TARGET_CORTEX_M\mbed_fault_handler.o | 380(+0) | 0(+0) | 84(+0) | | mbed-os\platform\internal\mbed_atomic_impl.o | 78(+22) | 0(+0) | 0(+0) | | mbed-os\platform\mbed_alloc_wrappers.o | 412(+0) | 0(+0) | 68(+0) | | mbed-os\platform\mbed_assert.o | 20(+0) | 0(+0) | 0(+0) | | mbed-os\platform\mbed_board.o | 124(+0) | 0(+0) | 0(+0) | | mbed-os\platform\mbed_critical.o | 92(+0) | 0(+0) | 4(+0) | | mbed-os\platform\mbed_error.o | 652(+0) | 0(+0) | 114(+0) | | mbed-os\platform\mbed_mktime.o | 386(+0) | 0(+0) | 0(+0) | | mbed-os\platform\mbed_os_timer.o | 268(+268) | 0(+0) | 60(+60) | | mbed-os\platform\mbed_retarget.o | 1248(+0) | 260(+0) | 16(+0) | | mbed-os\platform\mbed_rtc_time.o | 198(+0) | 16(+0) | 40(+0) | | mbed-os\platform\mbed_sleep_manager.o | 152(+20) | 0(+0) | 2(+0) | | mbed-os\platform\mbed_stats.o | 136(+0) | 0(+0) | 0(+0) | | mbed-os\platform\mbed_wait_api_no_rtos.o | 16(+0) | 0(+0) | 0(+0) | | mbed-os\platform\mbed_wait_api_rtos.o | 190(+0) | 0(+0) | 0(+0) | | mbed-os\rtos\EventFlags.o | 150(+6) | 0(+0) | 0(+0) | | mbed-os\rtos\Kernel.o | 56(+56) | 0(+0) | 8(+8) | | mbed-os\rtos\Mutex.o | 216(+2) | 0(+0) | 0(+0) | | mbed-os\rtos\Semaphore.o | 132(+0) | 0(+0) | 0(+0) | | mbed-os\rtos\TARGET_CORTEX\SysTimer.o | 0(-408) | 0(+0) | 0(+0) | | mbed-os\rtos\TARGET_CORTEX\TOOLCHAIN_GCC_ARM\mbed_boot_gcc_arm.o | 196(+0) | 0(+0) | 96(+0) | | mbed-os\rtos\TARGET_CORTEX\mbed_boot.o | 84(+0) | 0(+0) | 8(+0) | | mbed-os\rtos\TARGET_CORTEX\mbed_rtos_rtx.o | 148(+0) | 0(+0) | 4248(+0) | | mbed-os\rtos\TARGET_CORTEX\mbed_rtx_handlers.o | 190(+0) | 0(+0) | 4(+0) | | mbed-os\rtos\TARGET_CORTEX\mbed_rtx_idle.o | 238(+54) | 4(+0) | 0(-60) | | mbed-os\rtos\TARGET_CORTEX\rtx5\RTX\Source\TOOLCHAIN_GCC\TARGET_RTOS_M4_M7\irq_cm4f.o | 196(+0) | 0(+0) | 0(+0) | ... | mbed-os\rtos\TARGET_CORTEX\rtx5\RTX\Source\rtx_timer.o | 200(+0) | 0(+0) | 0(+0) | | mbed-os\rtos\TARGET_CORTEX\rtx5\Source\os_systick.o | 0(-64) | 0(+0) | 0(-1) | | mbed-os\rtos\ThisThread.o | 150(+0) | 0(+0) | 0(+0) | | mbed-os\rtos\Thread.o | 656(-4) | 0(+0) | 0(+0) | | mbed-os\targets\TARGET_STM\TARGET_STM32L4\TARGET_STM32L432xC\TARGET_NUCLEO_L432KC\system_clock.o | 260(+0) | 0(+0) | 0(+0) | | mbed-os\targets\TARGET_STM\TARGET_STM32L4\TARGET_STM32L432xC\device\TOOLCHAIN_GCC_ARM\startup_stm32l432xx.o | 454(+0) | 0(+0) | 0(+0) | ... | Subtotals | 184204(+768) | 3208(+0) | 28976(+0) | Total Static RAM memory (data + bss): 32184(+0) bytes Total Flash memory (text + data): 187412(+768) bytes I wondered if /*
* mbed SDK
* Copyright (c) 2017 ARM Limited
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// Automatically generated configuration file.
// DO NOT EDIT, content will be overwritten.
#ifndef __MBED_CONFIG_DATA__
#define __MBED_CONFIG_DATA__
// Configuration parameters
#define CLOCK_SOURCE USE_PLL_MSI // set by target:NUCLEO_L432KC
#define LPTICKER_DELAY_TICKS 0 // set by application[*]
#define MBED_CONF_APP_FLASH_CONFIG_END 0x08040000 // set by application
#define MBED_CONF_APP_FLASH_CONFIG_START 0x0803E000 // set by application
#define MBED_CONF_APP_FLASH_LOG_END 0x0803E000 // set by application
#define MBED_CONF_APP_FLASH_LOG_START 0x08035800 // set by application
#define MBED_CONF_DRIVERS_UART_SERIAL_RXBUF_SIZE 256 // set by library:drivers
#define MBED_CONF_DRIVERS_UART_SERIAL_TXBUF_SIZE 256 // set by library:drivers
#define MBED_CONF_EVENTS_PRESENT 1 // set by library:events
#define MBED_CONF_EVENTS_SHARED_DISPATCH_FROM_APPLICATION 0 // set by library:events
#define MBED_CONF_EVENTS_SHARED_EVENTSIZE 768 // set by library:events
#define MBED_CONF_EVENTS_SHARED_HIGHPRIO_EVENTSIZE 256 // set by library:events
#define MBED_CONF_EVENTS_SHARED_HIGHPRIO_STACKSIZE 1024 // set by library:events
#define MBED_CONF_EVENTS_SHARED_STACKSIZE 2048 // set by library:events
#define MBED_CONF_EVENTS_USE_LOWPOWER_TIMER_TICKER 0 // set by library:events
#define MBED_CONF_LORA_ADR_ON 1 // set by library:lora
#define MBED_CONF_LORA_APPLICATION_EUI {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} // set by library:lora
#define MBED_CONF_LORA_APPLICATION_KEY {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} // set by library:lora
#define MBED_CONF_LORA_APPSKEY {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} // set by library:lora
#define MBED_CONF_LORA_APP_PORT 15 // set by library:lora
#define MBED_CONF_LORA_AUTOMATIC_UPLINK_MESSAGE 1 // set by library:lora
#define MBED_CONF_LORA_DEVICE_ADDRESS 0x00000000 // set by library:lora
#define MBED_CONF_LORA_DEVICE_EUI {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} // set by library:lora
#define MBED_CONF_LORA_DOWNLINK_PREAMBLE_LENGTH 5 // set by library:lora
#define MBED_CONF_LORA_DUTY_CYCLE_ON 1 // set by library:lora
#define MBED_CONF_LORA_DUTY_CYCLE_ON_JOIN 1 // set by library:lora
#define MBED_CONF_LORA_FSB_MASK {0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0x00FF} // set by library:lora
#define MBED_CONF_LORA_FSB_MASK_CHINA {0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF, 0xFFFF} // set by library:lora
#define MBED_CONF_LORA_LBT_ON 0 // set by library:lora
#define MBED_CONF_LORA_MAX_SYS_RX_ERROR 5 // set by library:lora
#define MBED_CONF_LORA_NB_TRIALS 6 // set by application[*]
#define MBED_CONF_LORA_NWKSKEY {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00} // set by library:lora
#define MBED_CONF_LORA_OVER_THE_AIR_ACTIVATION 1 // set by library:lora
#define MBED_CONF_LORA_PHY EU868 // set by library:lora
#define MBED_CONF_LORA_PUBLIC_NETWORK 1 // set by library:lora
#define MBED_CONF_LORA_TX_MAX_SIZE 242 // set by application[*]
#define MBED_CONF_LORA_UPLINK_PREAMBLE_LENGTH 8 // set by library:lora
#define MBED_CONF_LORA_WAKEUP_TIME 5 // set by library:lora
#define MBED_CONF_MBED_TRACE_ENABLE 1 // set by application[*]
#define MBED_CONF_PLATFORM_CRASH_CAPTURE_ENABLED 0 // set by library:platform
#define MBED_CONF_PLATFORM_CTHUNK_COUNT_MAX 8 // set by library:platform
#define MBED_CONF_PLATFORM_DEFAULT_SERIAL_BAUD_RATE 9600 // set by library:platform
#define MBED_CONF_PLATFORM_ERROR_ALL_THREADS_INFO 0 // set by library:platform
#define MBED_CONF_PLATFORM_ERROR_FILENAME_CAPTURE_ENABLED 0 // set by library:platform
#define MBED_CONF_PLATFORM_ERROR_HIST_ENABLED 0 // set by library:platform
#define MBED_CONF_PLATFORM_ERROR_HIST_SIZE 4 // set by library:platform
#define MBED_CONF_PLATFORM_ERROR_REBOOT_MAX 1 // set by library:platform
#define MBED_CONF_PLATFORM_FATAL_ERROR_AUTO_REBOOT_ENABLED 0 // set by library:platform
#define MBED_CONF_PLATFORM_FORCE_NON_COPYABLE_ERROR 0 // set by library:platform
#define MBED_CONF_PLATFORM_MAX_ERROR_FILENAME_LEN 16 // set by library:platform
#define MBED_CONF_PLATFORM_POLL_USE_LOWPOWER_TIMER 0 // set by library:platform
#define MBED_CONF_PLATFORM_STDIO_BAUD_RATE 460800 // set by application[*]
#define MBED_CONF_PLATFORM_STDIO_BUFFERED_SERIAL 0 // set by library:platform
#define MBED_CONF_PLATFORM_STDIO_CONVERT_NEWLINES 0 // set by library:platform
#define MBED_CONF_PLATFORM_STDIO_CONVERT_TTY_NEWLINES 0 // set by library:platform
#define MBED_CONF_PLATFORM_STDIO_FLUSH_AT_EXIT 1 // set by library:platform
#define MBED_CONF_PLATFORM_USE_MPU 0 // set by application[*]
#define MBED_CONF_RTOS_API_PRESENT 1 // set by library:rtos-api
#define MBED_CONF_RTOS_IDLE_THREAD_STACK_SIZE 512 // set by library:rtos
#define MBED_CONF_RTOS_IDLE_THREAD_STACK_SIZE_DEBUG_EXTRA 128 // set by library:rtos[STM]
#define MBED_CONF_RTOS_IDLE_THREAD_STACK_SIZE_TICKLESS_EXTRA 256 // set by library:rtos
#define MBED_CONF_RTOS_MAIN_THREAD_STACK_SIZE 4096 // set by library:rtos
#define MBED_CONF_RTOS_PRESENT 1 // set by library:rtos
#define MBED_CONF_RTOS_THREAD_STACK_SIZE 4096 // set by library:rtos
#define MBED_CONF_RTOS_TIMER_THREAD_STACK_SIZE 768 // set by library:rtos
#define MBED_CONF_SX1276_LORA_DRIVER_BUFFER_SIZE 255 // set by library:sx1276-lora-driver
#define MBED_CONF_SX1276_LORA_DRIVER_RADIO_VARIANT SX1276MB1MAS // set by application[PENGUIN]
#define MBED_CONF_SX1276_LORA_DRIVER_SPI_FREQUENCY 8000000 // set by library:sx1276-lora-driver
#define MBED_CONF_TARGET_BOOT_STACK_SIZE 0x400 // set by library:rtos[*]
#define MBED_CONF_TARGET_CONSOLE_UART 1 // set by target:Target
#define MBED_CONF_TARGET_DEEP_SLEEP_LATENCY 3 // set by target:FAMILY_STM32
#define MBED_CONF_TARGET_INIT_US_TICKER_AT_BOOT 1 // set by target:FAMILY_STM32
#define MBED_CONF_TARGET_LPTICKER_LPTIM 0 // set by application[*]
#define MBED_CONF_TARGET_LPTICKER_LPTIM_CLOCK 1 // set by target:FAMILY_STM32
#define MBED_CONF_TARGET_LPUART_CLOCK_SOURCE USE_LPUART_CLK_PCLK1 // set by application[*]
#define MBED_CONF_TARGET_LSE_AVAILABLE 1 // set by target:FAMILY_STM32
#define MBED_CONF_TARGET_MPU_ROM_END 0x0fffffff // set by target:Target
#define MBED_CONF_TARGET_TICKLESS_FROM_US_TICKER 0 // set by application[*]
#define MBED_HEAP_STATS_ENABLED 1 // set by application[*]
#define MBED_STACK_STATS_ENABLED 1 // set by application[*]
#define MBED_TRACE_MAX_LEVEL TRACE_LEVEL_DEBUG // set by application[*]
#define MEM_ALLOC malloc // set by library:mbed-trace
#define MEM_FREE free // set by library:mbed-trace
// Macros
#define MBEDTLS_USER_CONFIG_FILE "mbedtls_lora_config.h" // defined by application
#define MBED_TRACE_CONFIG TRACE_ACTIVE_LEVEL_ALL // defined by application
#define _RTE_ // defined by library:rtos
#endif
|
|
As I recall there were some STM fixes following this PR, particularly around tickless, so if you're seeing stability problems "at this version", I'd be inclined to try something later. Note that this was merged quite early in the 5.14 release cycle, so you're looking at something quite some way from 5.14 release, let alone the patch version. Going incrementally, I would tend to go release by release, unless in an emergency (in which case maybe just git bisect between releases?) For an RTOS build, rather than a non-RTOS build, there shouldn't be any significant functional change - I can't immediately explain why there's that much code size change. I can imagine it being a bit bigger though - it is working a bit harder around various edge cases in an attempt to tighten up behaviour, and the core code is more generic to permit non-RTOS sleeping. |
|
@mattbrown015 thanks for the feedback, would like to know more about going through the update (errors there, what should be improved to make this transition smoother), please let us know on the forum (if you can mention me there to get visibility, would be 👍 ). |
Description
A chunk of work bringing together connected themes.
waitAPIs that actually sleep - only remaining ones spin and are safe from weird contexts.Details:
rtos/mbed_lib.jsonmoved down tortos/TARGET_CORTEX, still withname: "rtos"so it's still checked byMBED_CONF_RTOS_PRESENT.rtos/mbed_lib.jsonhasname: "rtos-api", so checked byMBED_CONF_RTOS_API_PRESENT, and accessible asrtos/Semaphore.hstill.mbed_rtos_types.hinstead ofcmsis_os2.h(cmsis_os2.h is not there in bare-metal, just the C++ RTOS APIs).Points for discussion:
waitAPIs - deprecate all and split intosleepanddelay?TODO: Make event loop use the RTOS API for its sleepdoneTODO: Recheck exact changes made to sleep code, for discussion with Russ (GitHub diff isn't useful)
With this PR, you can take blinky, add
"requires" : [ "bare-metal" ]to its mbed_app.json change itswait_mstoThisThread::sleep_for, and you've now got a bare metal blinky that properly deep sleeps, with all the latency compensation stuff you get in the RTOS. (For backwards compatibility,wait_msremains non-sleeping in non-RTOS builds).Pull request type
Reviewers
@c1728p9, @sg-, @geky, @theotherjimmy, @bridadan, @bulislaw
Release Notes
Migration notes
waitandwait_msshould be reviewed to check whether sleeping is appropriate or not in the context the call occurs - if so, change toThisThread::sleep_for/thread_sleep_for, else change towait_us.