tests/drivers/counter/counter_basic_api failure on nRF51-DK

[mbolivar-nordic]

Describe the bug

While getting baseline test results for nRF counter APIs in order to test changes related to the new devicetree.h, I noticed counter_basic_api is failing for nRF51-DK.

To Reproduce

I'm using sanitycheck's device testing feature to test nRF51-DK, nRF52840-DK, nRF52-DK like so:

./scripts/sanitycheck --device-testing \
		      --hardware-map ~/zp/hw-map.yaml \
		      -T tests/drivers/counter \
		      --ninja

My hw-map.yaml file associates devices with serial ports and nrfjprog IDs.

Test results (with #24350 applied to satisfy the platform whitelist for nRF52840-DK) show the above mentioned failure. It is repeatable. Other tests always seem to pass.

Expected behavior

All tests should pass.

Impact

Unclear. Do we need this test to pass on this board?

Screenshots or console output

$ ./scripts/sanitycheck --device-testing  --hardware-map ~/zp/hw-map.yaml  -T tests/drivers/counter  --ninja
Renaming output directory to /home/mbolivar/zp/zephyr/sanity-out.4
INFO    - JOBS: 8
INFO    - Building initial testcase list...
INFO    - 4 test configurations selected, 5 configurations discarded due to filters.

Device testing on:

| Platform            |           ID | Serial device                                         |
|---------------------|--------------|-------------------------------------------------------|
| nrf51dk_nrf51422    | 000681986260 | /dev/serial/by-id/usb-SEGGER_J-Link_000681986260-if00 |
| nrf52840dk_nrf52840 | 000683441799 | /dev/serial/by-id/usb-SEGGER_J-Link_000683441799-if00 |
| nrf52dk_nrf52832    | 000682296740 | /dev/serial/by-id/usb-SEGGER_J-Link_000682296740-if00 |

INFO    - Adding tasks to the queue...
INFO    - Total complete:    3/   4  75%  skipped:    0, failed:    0
ERROR   - nrf51dk_nrf51422          tests/drivers/counter/counter_basic_api/drivers.counter FAILED: Failed
ERROR   - see: /home/mbolivar/zp/zephyr/sanity-out/nrf51dk_nrf51422/tests/drivers/counter/counter_basic_api/drivers.counter/handler.log
INFO    - Total complete:    4/   4  100%  skipped:    0, failed:    1
INFO    - 3 of 4 tests passed (75.00%), 1 failed, 0 skipped with 0 warnings in 30.41 seconds
INFO    - In total 3 test cases were executed on 3 out of total 241 platforms (1.24%)

Hardware distribution summary:

| Board               |           ID |   Counter |
|---------------------|--------------|-----------|
| nrf52dk_nrf52832    | 000682296740 |         1 |
| nrf51dk_nrf51422    | 000681986260 |         1 |
| nrf52840dk_nrf52840 | 000683441799 |         2 |

Failing log:

$ cat /home/mbolivar/zp/zephyr/sanity-out/nrf51dk_nrf51422/tests/drivers/counter/counter_basic_api/drivers.counter/handler.log
*** Booting Zephyr OS build zephyr-v2.2.0-1473-gd87c4556d878  ***
Running test suite test_counter
===================================================================
starting test - test_set_top_value_with_alarm
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_set_top_value_with_alarm
===================================================================
starting test - test_single_shot_alarm_notop
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_single_shot_alarm_notop
===================================================================
starting test - test_single_shot_alarm_top
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/counter/counter_basic_api/src/test_counter.c:265: alarm_handler: (now >= counter is false)
RTC_0: Alarm (655) too early now: 0 (counting up).
FAIL - test_single_shot_alarm_top
===================================================================
starting test - test_multiple_alarms
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_multiple_alarms
===================================================================
starting test - test_all_channels
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_all_channels
===================================================================
starting test - test_late_alarm
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_late_alarm
===================================================================
starting test - test_late_alarm_error
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_late_alarm_error
===================================================================
starting test - test_short_relative_alarm
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_short_relative_alarm
===================================================================
starting test - test_cancelled_alarm_does_not_expire
Testing TIMER_0
Testing TIMER_1
Assertion failed at WEST_TOPDIR/zephyr/tests/drivers/counter/counter_basic_api/src/test_counter.c:802: test_cancelled_alarm_does_not_expire_instance: (0 not equal to err)
TIMER_1: Failed to set an alarm (err: -22)
FAIL - test_cancelled_alarm_does_not_expire
===================================================================
starting test - test_set_top_value_without_alarm
Testing TIMER_0
Testing TIMER_1
Testing TIMER_2
Testing RTC_0
Testing RTC_0
PASS - test_set_top_value_without_alarm
===================================================================
Test suite test_counter failed.
===================================================================
PROJECT EXECUTION FAILED

Environment (please complete the following information):

• OS: Linux
• Toolchain: Zephyr SDK
• Commit SHA: 5a149df plus tests: counter_nrf_rtc: fix platform whitelist #24350 applied to satisfy the platform whitelist for nRF52840-DK

[mbolivar-nordic]

cc @carlescufi @anangl @ioannisg