RFC: New Testing Approach for Power Management Subsystem.

[gbarkadiusz]

This proposal suggests a new method for testing the Power Management (PM) subsystem. It expands the testing environment by adding an extra power monitor to confirm that the system reaches the correct power states. This improvement aims to make power management tests more accurate and reliable.

Problem:

Verifying the system's ability to achieve all available power states. Zephyr’s Power Management subsystem provides multiple sleep modes. For instance, the stm32l562e_dk board in Zephyr supports four sleep states: k_cpu_idle, stop_0, stop_1, and stop_2. The main challenge is reliably confirming that each specific power state is reached as expected.

Solution:

I propose using an external Power Monitor to track the current consumption of the target device. For this process, I will use the stm32l562e_dk board, which has an onboard Power Monitor. While the application runs, the power monitor measures the current consumption of the target. At the end of the test, this approach utilizes pytest to calculate the RMS (Root Mean Square) value of each detected power state. This method simplifies the verification of whether each power state has been achieved.

Setup

Documentation:
STM32L562E-DK Evaluation Board: STMicroelectronics STM32L562E-DK
STM32L562E-DK User Manual: User Manual DM0063555
Power Monitor Firmware Documentation: Getting Started with PowerShield Firmware
Configuration: Based on the documentation provided, the stm32l562e_dk board should be configured as follows:

Implementation:

As an example of usage, the onboard power monitor is utilized. A script named pwsh_stm32l562.py has been created to facilitate communication with the board. All necessary information regarding the communication protocol can be found in the Power Monitor Firmware documentation.

If you wish to use a different power monitor, you will need to develop your own pwsh_stm32l562.py script to communicate with your selected power monitoring device.

Power Monitor Abstract Class
The Power_Monitor class serves as a base class for implementing power monitoring functionality. It defines the essential interface for any power monitoring device, ensuring that the following methods are implemented:

Methods:
init_power_monitor(device_id: str) -> bool:
Abstract method for initializing the power monitor with a specified device ID. It returns a boolean indicating whether the initialization was successful.

measure_current(duration: int):
Abstract method for measuring the current over a specified duration. This method does not return a value, as its primary purpose is to initiate the measurement process.

get_data(duration: int) -> List[float]:
Abstract method for retrieving an array of current measurements taken over the specified duration, returning the values in amperes.

Demonstration:

To showcase my approach, I have prepared three tests along with a pwsh_stm32l562.py script that implements the PowerMonitor class based on the abstract class described earlier.

Test: pm.states
This test is designed to verify all available power states on the stm32l562e_dk board. The results from the measure_current function can be found in the measurementData.csv file.

The chart above illustrates the measured current consumption during the execution of the pm_states test. Key elements of the chart are highlighted as follows:

Startup: The current peak observed at the beginning of the test.
Boot Delay: The delay associated with the additional configuration set to CONFIG_BOOT_DELAY=500.
k_cpu_idle: The low power state achieved using k_msleep(Tms), where Tms is shorter than the minimum residency time for the stop_0 power state defined in the device tree source (.dts) file.
Wake Up and Print Log: The target board wakes up between each power state and logs the corresponding messages.
stop_0, stop_1, stop_2, active_state: The measured current consumption for each respective power state.
During the test, pytest detects each step and calculates the RMS (Root Mean Square) current for every detected state. The test then compares the calculated RMS values against the expected values, which have been determined manually through experimental observation

To Reproduce:

1. Configure the stm32l562e_dk board according to the instructions provided above.
2. Connect both USB ports of the board to your host system. The board will appear as two separate targets:
   - Power Monitor: e.g., /dev/ttyACM0 or /dev/serial/by-id/usb-STMicroelectronics_PowerShield__Virtual_ComPort_in_FS_Mode__FFFFFFFEFFFF-if00
   - Target: e.g., /dev/ttyACM1 or /dev/serial/by-id/usb-STMicroelectronics_STLINK-V3_004##############39-if02
3. Copy the path of the Power Monitor and update the <path_to> in the tests/subsys/pm/power_states/testcase.yaml file under the pytest_args section as follows:

pytest_args: [--powershield=<path_to>/PowerMonitor]

4. Run the following command to execute the tests:

twister --device-testing --device-serial <path_to>/Target -p stm32l562e_dk -T tests/subsys/pm/power_states/ -x=CONFIG_BOOT_DELAY=500 -vv

Pytest log:

DEBUG   - PYTEST: k_cpu_idle:
DEBUG   - PYTEST: Expected RMS: 57.00 mA
DEBUG   - PYTEST: Tolerance: 5.70 mA
DEBUG   - PYTEST: Current RMS: 56.44 mA
DEBUG   - PYTEST: state 0:
DEBUG   - PYTEST: Expected RMS: 4.00 mA
DEBUG   - PYTEST: Tolerance: 0.40 mA
DEBUG   - PYTEST: Current RMS: 3.99 mA
DEBUG   - PYTEST: state 1:
DEBUG   - PYTEST: Expected RMS: 1.30 mA
DEBUG   - PYTEST: Tolerance: 0.13 mA
DEBUG   - PYTEST: Current RMS: 1.29 mA
DEBUG   - PYTEST: state 2:
DEBUG   - PYTEST: Expected RMS: 0.27 mA
DEBUG   - PYTEST: Tolerance: 0.03 mA
DEBUG   - PYTEST: Current RMS: 0.27 mA
DEBUG   - PYTEST: PASSED

Test: pm.residency_time
This test evaluates the residency time of different power states on the stm32l562e_dk board. The sequence of states is as follows:

The chart above illustrates the measured current consumption during the execution of the pm_residency_time test. Key elements of the chart are highlighted as follows:
Startup: The current peak observed at the beginning of the test.
Boot Delay: The delay associated with the additional configuration set to CONFIG_BOOT_DELAY=500.
k_cpu_idle: This represents the kernel idle state, where the sleep time is shorter than the residency time required for the stop_0 state.
stop_0: In this state, there is sufficient sleep time for the system to enter stop_0.
A_stop_1: Here, the sleep time is again shorter than the residency time required to transition to the stop_1 state.
B_stop_1: This state allows for enough sleep time to transition into stop_1.
A_stop_2: In this iteration, the sleep time is shorter than the residency time required to transition to the stop_2 state.
B_stop_2: The system has enough sleep time to enter the stop_2 state.
active_state: This state indicates that the application is running in an infinite loop, representing the active state of the system.
Wake Up and Print Log: The target board wakes up between each power state and logs the corresponding messages.

To Reproduce:
(1-3) As previously.
4. Run the following command to execute the tests:

twister --device-testing --device-serial <path_to>/Target -p stm32l562e_dk -T tests/subsys/pm/power_residency_time/ -x=CONFIG_BOOT_DELAY=500 -vv

Pytest Log:

DEBUG   - PYTEST: k_cpu_idle:
DEBUG   - PYTEST: Expected RMS: 57.00 mA
DEBUG   - PYTEST: Tolerance: 5.70 mA
DEBUG   - PYTEST: Current RMS: 56.37 mA
DEBUG   - PYTEST: state 0:
DEBUG   - PYTEST: Expected RMS: 4.00 mA
DEBUG   - PYTEST: Tolerance: 0.40 mA
DEBUG   - PYTEST: Current RMS: 3.78 mA
DEBUG   - PYTEST: state 0:
DEBUG   - PYTEST: Expected RMS: 4.00 mA
DEBUG   - PYTEST: Tolerance: 0.40 mA
DEBUG   - PYTEST: Current RMS: 3.92 mA
DEBUG   - PYTEST: state 1:
DEBUG   - PYTEST: Expected RMS: 1.20 mA
DEBUG   - PYTEST: Tolerance: 0.12 mA
DEBUG   - PYTEST: Current RMS: 1.19 mA
DEBUG   - PYTEST: state 1:
DEBUG   - PYTEST: Expected RMS: 1.20 mA
DEBUG   - PYTEST: Tolerance: 0.12 mA
DEBUG   - PYTEST: Current RMS: 1.18 mA
DEBUG   - PYTEST: state 2:
DEBUG   - PYTEST: Expected RMS: 0.27 mA
DEBUG   - PYTEST: Tolerance: 0.03 mA
DEBUG   - PYTEST: Current RMS: 0.27 mA
DEBUG   - PYTEST: active:
DEBUG   - PYTEST: Expected RMS: 92.70 mA
DEBUG   - PYTEST: Tolerance: 9.27 mA
DEBUG   - PYTEST: Current RMS: 94.33 mA
DEBUG   - PYTEST: PASSED

Test: pm.wakeup_timer
In this test, the stm32l562e_dk target board is configured to set the RTC Alarm for 1 second before entering the lowest power state by calling the k_msleep() function. The sequence of operations is as follows:
-The RTC Alarm is set for 1 second.
-The board enters a low-power state, effectively minimizing power consumption.
-After the alarm triggers, the RTC Alarm callback wakes up the CPU.
-The CPU then enters an anti-sleeping loop to prevent returning to sleep mode.
-Throughout this process, the power shield continuously measures the power consumption of the board.
This test helps verify the functionality of the wake-up timer and assesses the power consumption during low-power operation.

The chart above illustrates the measured current consumption during the execution of the pm_wakeup_timer test. The following key elements are highlighted in the chart:
Startup: The current peak observed at the beginning of the test.
Boot Delay: The delay associated with the additional configuration set to CONFIG_BOOT_DELAY=500.
Go Sleep: This phase represents when the alarm is set, and the board enters a low-power state by calling the k_msleep() function.
Stop 2: The measured current consumption during the stop_2 power state.
Alarm Interrupt: This indicates the moment the RTC alarm interrupt wakes the CPU, transitioning into an infinite loop.
Active State: This state signifies that the application is running in an infinite loop, representing the active operational state of the system.

To Reproduce:
(1-3) As previously.
4. Run the following command to execute the tests:

twister --device-testing --device-serial <path_to>/Target -p stm32l562e_dk -T tests/subsys/pm/power_wakeup_timer/ -x=CONFIG_BOOT_DELAY=500 -vv

Pytest_log:

DEBUG   - PYTEST: Expected RMS: 0.26 mA
DEBUG   - PYTEST: Tolerance: 0.03 mA
DEBUG   - PYTEST: Current RMS: 0.27 mA
DEBUG   - PYTEST: active:
DEBUG   - PYTEST: Expected RMS: 95.00 mA
DEBUG   - PYTEST: Tolerance: 9.50 mA
DEBUG   - PYTEST: Current RMS: 91.90 mA
DEBUG   - PYTEST: PASSED

All the tests mentioned above are available in the following pull request: zephyrproject-rtos/zephyr#80692. This pull request is currently in draft status and will remain so until the RFC is reviewed and applied.

[hakehuang]

measure with on-board adc is an options, and I from NXP would like to propose to use our mcu-link-pro as a standalone measure device to do this. see https://www.nxp.com/docs/en/user-manual/UM11673.pdf

[gbarkadiusz]

Transferred to #80989. Please move your comment there.