Mbed OS 5.10 OOB - IP connectivity test cases resulted as inconclusive

[atomichan]

Description

Mbed OS 5.10 OOB - IP connectivity test resulted as inconclusive due to cmd timeout
(Target: UBLOX_EVK_ODIN_W2, Host: Mac).

Expected result

All TEST_APPS should pass.

Actual result

1 testcase skipped, 4 testcases inconclusive.
See below for more details.

Versions

$ mbed ls
mbed-os (#3fb5781af180, tag: mbed-os-5.10.0-rc1)

python 2.7.15
mbed-cli 1.8.0
icetea 1.0.1

Test result

$ mbed test -m UBLOX_EVK_ODIN_W2 -t GCC_ARM --icetea --test-config tools/test_configs/HeapBlockDeviceAndEthernetInterface.json 
[mbed] Auto-installing missing Python modules...
Building library mbed-build (UBLOX_EVK_ODIN_W2, GCC_ARM)
Scan: mbed-os
Copy: sn_config.h
Copy: lorawan_types.h

12:24:34.394 | TC     MainThread: Test 'TCPSOCKET_ECHOTEST_BURST_SHORT' FAIL, reason: /dev/tty.usbmodem14533 CMD timeout: set --retcode true
12:24:34.617 Test case TCPSOCKET_ECHOTEST_BURST_SHORT failed, No retries left.

+--------------------------------+--------------+--------------------------------------------------------+---------------------------------------------+-----------+------------+
| Testcase                       |   Verdict    |                      Fail Reason                       |                 Skip Reason                 | platforms |  duration  |
+--------------------------------+--------------+--------------------------------------------------------+---------------------------------------------+-----------+------------+
| test_cmdline                   |     skip     |                                                        | Test requiring application binary not build |           |     0      |
| UDPSOCKET_BIND_PORT            | inconclusive | /dev/tty.usbmodem14533 CMD timeout: set --retcode true |                                             |           | 165.100255 |
| TCPSOCKET_BIND_PORT            | inconclusive | /dev/tty.usbmodem14533 CMD timeout: set --retcode true |                                             |           | 153.314324 |
| TCPSERVER_ACCEPT               | inconclusive |           No suitable local device available           |                                             |           |   0.394    |
| TCPSOCKET_ECHOTEST_BURST_SHORT | inconclusive | /dev/tty.usbmodem14533 CMD timeout: set --retcode true |                                             |           | 153.835599 |
+--------------------------------+--------------+--------------------------------------------------------+---------------------------------------------+-----------+------------+
+---------------+----------------+
|    Summary    |                |
+---------------+----------------+
| Final Verdict |  INCONCLUSIVE  |
|     count     |       5        |
|    passrate   |     0.00 %     |
|      skip     |       1        |
|  inconclusive |       4        |
|    Duration   | 0:07:52.644178 |
+---------------+----------------+
12:24:34.640 Cleanup done.

Issue request type

[] Question
[ ] Enhancement
[X] Bug

[ashok-rao]

@SeppoTakalo .. can you please have a look? Thanks.

[SeppoTakalo]

@jarlamsa @OPpuolitaival Please check

[jarlamsa]

Seems that sending first command over serial timeouts @jonikula have you seen that before?

[SeppoTakalo]

Please check internal Jira: IOTSYSTOOL-675

[jupe]

Icetea has already option to wait shell before sending commands using cli_ready_trigger -option, see: https://github.com/ARMmbed/icetea/blob/d6c31645edceafd868e2680282a7cf4d9b01805e/examples/testcase_example_usage/sample_cli_init.py#L58 . Could it be used here? We are also bringing another approach to verify that device are ready for receiving commands, probably included in next minor release. I think both of approach should fix this original issue.

[jupe]

I've played a bit with UBLOX_EVK_ODIN_W2 with mac and looks that I can get it stuck relative easily so that only way to recover board is re-plugging USB cable. Same thing does not seem to happen with windows neither linux. Simplest way to reproduce this is just running test_cmdline -test in mac. Even screen connection to serial port get stuck (screen /dev/tty.usbmodem144432 115200) - so assuming whole interface is stuck somehow (or mac serial port). Pressing reset seems to reset board in some level and print shell prompt but application in UBLOX does not receive commands through serial port. I haven't debug this more deeply but maybe somebody could try also if behaviour are the same..

[atomichan]

@jupe @SeppoTakalo Thank you for the comment.
I changed the environment to windows (on a vmware virtual machine) and retried the test.
It seems the result using GCC_ARM is the same as reported on issue #8028, which I think was a expected result.