tests-ci : net: socket: reuseaddr_reuseport.userspace test Failed

[hakehuang]

Describe the bug
reuseaddr_reuseport.userspace test is Failed on v3.5.0-rc3-5-g90b9809c6f55 on mimxrt1160_evk_cm7
testcase path is tests/net/socket/reuseaddr_reuseport

see logs for details

To Reproduce
1.

scripts/twister --device-testing --device-serial /dev/ttyACM0 -p mimxrt1160_evk_cm7  --sub-test net.socket

or

# cd tests/net/socket/reuseaddr_reuseport/userspace/ipv4_tcp_identical_clients
# west build -b mimxrt1160_evk_cm7
# west flash

2. See error

Expected behavior
test pass

Impact

Logs and console output

START - test_ipv4_tcp_identical_clients

Assertion failed at WEST_TOPDIR/zephyr/tests/net/socket/reuseaddr_reuseport/src/../../socket_helpers.h:98: prepare_sock_tcp_v4: (*sock >= 0 is false)
socket open failed
FAIL - test_ipv4_tcp_identical_clients in 0.016 seconds

    

Environment (please complete the following information):

• OS: (e.g. Linux )
• Toolchain (e.g Zephyr SDK)
• Commit SHA or Version used: v3.5.0-rc3-5-g90b9809c6f55

[hakehuang]

Also fails on mimxrt1160_evk_cm7 for v3.5.0-rc3-5-g90b9809c6f55

[hakehuang]

Also fails on mimxrt1170_evk_cm7 for v3.5.0-rc3-5-g90b9809c6f55

[hakehuang]

@dleach02 this case is in failure status since it is created in #1de4e73e6d62bd90d2533f6333483f7643982119

[dleach02]

@Frauschi the bisect identified your PR #61885. Do you have any thoughts?

[Frauschi]

This seems to be a problem in the generel test setup of the failing tests, not the explicit SO_REUSEADDR implementation (the bind() calls are not the failing ones as the logs indicate). I will look at it tomorrow.

[Frauschi]

With the current main and #64180 merged, I cannot reproduce the issues anymore (I tested with a nucleo_h743zi and a stm32h573i_dk only). I'm pretty sure the issues have been related to #64180 as the logs of the failing tests indicate incorrect behavior of socket operations unrelated to SO_REUSEADDR/SO_REUSEPORT specifically.

[hakehuang]

@Frauschi this fix works, verified at zephyr-v3.5.0-468-g6dc8b55bbf on

*** Booting Zephyr OS build zephyr-v3.5.0-468-g6dc8b55bbf03 ***
Running TESTSUITE socket_reuseaddr_test_suite
===================================================================
START - test_enable_disable
 PASS - test_enable_disable in 0.002 seconds
===================================================================
START - test_ipv4_both_unspecified
 PASS - test_ipv4_both_unspecified in 0.002 seconds
===================================================================
START - test_ipv4_first_unspecified
 PASS - test_ipv4_first_unspecified in 0.002 seconds
===================================================================
START - test_ipv4_second_unspecified
 PASS - test_ipv4_second_unspecified in 0.002 seconds
===================================================================
START - test_ipv4_tcp_specified_listening
 PASS - test_ipv4_tcp_specified_listening in 0.002 seconds
===================================================================
START - test_ipv4_tcp_time_wait_specified
 PASS - test_ipv4_tcp_time_wait_specified in 0.258 seconds
===================================================================
START - test_ipv4_tcp_time_wait_unspecified
 PASS - test_ipv4_tcp_time_wait_unspecified in 0.258 seconds
===================================================================
START - test_ipv4_tcp_unspecified_listening
 PASS - test_ipv4_tcp_unspecified_listening in 0.002 seconds
===================================================================
START - test_ipv6_both_unspecified
 PASS - test_ipv6_both_unspecified in 0.002 seconds
===================================================================
START - test_ipv6_first_unspecified
 PASS - test_ipv6_first_unspecified in 0.002 seconds
===================================================================
START - test_ipv6_second_unspecified
 PASS - test_ipv6_second_unspecified in 0.002 seconds
===================================================================
START - test_ipv6_tcp_specified_listening
 PASS - test_ipv6_tcp_specified_listening in 0.002 seconds
===================================================================
START - test_ipv6_tcp_time_wait_specified
 PASS - test_ipv6_tcp_time_wait_specified in 0.258 seconds
===================================================================
START - test_ipv6_tcp_time_wait_unspecified
I: ETH_0 enabled 100M full-duplex mode.
 PASS - test_ipv6_tcp_time_wait_unspecified in 0.258 seconds
===================================================================
START - test_ipv6_tcp_unspecified_listening
 PASS - test_ipv6_tcp_unspecified_listening in 0.003 seconds
===================================================================
TESTSUITE socket_reuseaddr_test_suite succeeded
Running TESTSUITE socket_reuseport_test_suite
===================================================================
START - test_enable_disable
 PASS - test_enable_disable in 0.002 seconds
===================================================================
START - test_ipv4_both_specified_bad
 PASS - test_ipv4_both_specified_bad in 0.002 seconds
===================================================================
START - test_ipv4_both_specified_good
 PASS - test_ipv4_both_specified_good in 0.002 seconds
===================================================================
START - test_ipv4_both_unspecified_bad
 PASS - test_ipv4_both_unspecified_bad in 0.002 seconds
===================================================================
START - test_ipv4_both_unspecified_good
 PASS - test_ipv4_both_unspecified_good in 0.002 seconds
===================================================================
START - test_ipv4_first_unspecified_bad
 PASS - test_ipv4_first_unspecified_bad in 0.002 seconds
===================================================================
START - test_ipv4_first_unspecified_good
 PASS - test_ipv4_first_unspecified_good in 0.002 seconds
===================================================================
START - test_ipv4_second_unspecified_bad
 PASS - test_ipv4_second_unspecified_bad in 0.002 seconds
===================================================================
START - test_ipv4_second_unspecified_good
 PASS - test_ipv4_second_unspecified_good in 0.002 seconds
===================================================================
START - test_ipv4_tcp_identical_clients
 PASS - test_ipv4_tcp_identical_clients in 0.207 seconds
===================================================================
START - test_ipv4_udp_bad_both_not_set
 PASS - test_ipv4_udp_bad_both_not_set in 0.104 seconds
===================================================================
START - test_ipv4_udp_bad_first_not_set
 PASS - test_ipv4_udp_bad_first_not_set in 0.104 seconds
===================================================================
START - test_ipv4_udp_bad_second_not_set
 PASS - test_ipv4_udp_bad_second_not_set in 0.104 seconds
===================================================================
START - test_ipv4_udp_good
 PASS - test_ipv4_udp_good in 0.104 seconds
===================================================================
START - test_ipv6_both_specified_bad
 PASS - test_ipv6_both_specified_bad in 0.003 seconds
===================================================================
START - test_ipv6_both_specified_good
 PASS - test_ipv6_both_specified_good in 0.002 seconds
===================================================================
START - test_ipv6_both_unspecified_bad
 PASS - test_ipv6_both_unspecified_bad in 0.002 seconds
===================================================================
START - test_ipv6_both_unspecified_good
 PASS - test_ipv6_both_unspecified_good in 0.002 seconds
===================================================================
START - test_ipv6_first_unspecified_bad
 PASS - test_ipv6_first_unspecified_bad in 0.002 seconds
===================================================================
START - test_ipv6_first_unspecified_good
 PASS - test_ipv6_first_unspecified_good in 0.002 seconds
===================================================================
START - test_ipv6_second_unspecified_bad
 PASS - test_ipv6_second_unspecified_bad in 0.002 seconds
===================================================================
START - test_ipv6_second_unspecified_good
 PASS - test_ipv6_second_unspecified_good in 0.002 seconds
===================================================================
START - test_ipv6_tcp_identical_clients
 PASS - test_ipv6_tcp_identical_clients in 0.207 seconds
===================================================================
START - test_ipv6_udp_bad_both_not_set
 PASS - test_ipv6_udp_bad_both_not_set in 0.105 seconds
===================================================================
START - test_ipv6_udp_bad_first_not_set
 PASS - test_ipv6_udp_bad_first_not_set in 0.104 seconds
===================================================================
START - test_ipv6_udp_bad_second_not_set
 PASS - test_ipv6_udp_bad_second_not_set in 0.104 seconds
===================================================================
START - test_ipv6_udp_good
 PASS - test_ipv6_udp_good in 0.104 seconds
===================================================================
TESTSUITE socket_reuseport_test_suite succeeded

------ TESTSUITE SUMMARY START ------

SUITE PASS - 100.00% [socket_reuseaddr_test_suite]: pass = 15, fail = 0, skip = 0, total = 15 duration = 1.055 seconds
 - PASS - [socket_reuseaddr_test_suite.test_enable_disable] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv4_both_unspecified] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv4_first_unspecified] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv4_second_unspecified] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv4_tcp_specified_listening] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv4_tcp_time_wait_specified] duration = 0.258 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv4_tcp_time_wait_unspecified] duration = 0.258 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv4_tcp_unspecified_listening] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv6_both_unspecified] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv6_first_unspecified] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv6_second_unspecified] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv6_tcp_specified_listening] duration = 0.002 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv6_tcp_time_wait_specified] duration = 0.258 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv6_tcp_time_wait_unspecified] duration = 0.258 seconds
 - PASS - [socket_reuseaddr_test_suite.test_ipv6_tcp_unspecified_listening] duration = 0.003 seconds

SUITE PASS - 100.00% [socket_reuseport_test_suite]: pass = 27, fail = 0, skip = 0, total = 27 duration = 1.282 seconds
 - PASS - [socket_reuseport_test_suite.test_enable_disable] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_both_specified_bad] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_both_specified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_both_unspecified_bad] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_both_unspecified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_first_unspecified_bad] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_first_unspecified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_second_unspecified_bad] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_second_unspecified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_tcp_identical_clients] duration = 0.207 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_udp_bad_both_not_set] duration = 0.104 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_udp_bad_first_not_set] duration = 0.104 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_udp_bad_second_not_set] duration = 0.104 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv4_udp_good] duration = 0.104 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_both_specified_bad] duration = 0.003 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_both_specified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_both_unspecified_bad] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_both_unspecified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_first_unspecified_bad] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_first_unspecified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_second_unspecified_bad] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_second_unspecified_good] duration = 0.002 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_tcp_identical_clients] duration = 0.207 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_udp_bad_both_not_set] duration = 0.105 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_udp_bad_first_not_set] duration = 0.104 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_udp_bad_second_not_set] duration = 0.104 seconds
 - PASS - [socket_reuseport_test_suite.test_ipv6_udp_good] duration = 0.104 seconds

------ TESTSUITE SUMMARY END ------

===================================================================
PROJECT EXECUTION SUCCESSFUL