Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vstest] Only collect stdout of orchagent_restart_check in vstest #2578

Merged
merged 1 commit into from
Dec 19, 2022

Conversation

bingwang-ms
Copy link
Contributor

What I did
This PR is to fix the flaky test cases in vstest.
We frequently saw test failures in PR testing like below

test_PortAutoNegWarm failed (1 runs remaining out of 2).
	<class 'AssertionError'>
	assert 'RESTARTCHECK...unction 267\n' == 'RESTARTCHECK succeeded\n'
    RESTARTCHECK succeeded
  - profiling:/__w/1/s/common/.libs/libswsscommon_la-consumerstatetable.gcda:Merge mismatch for function 267
	[<TracebackEntry /agent/_work/1/s/tests/test_port_an.py:262>]

The output is produced by gcov. However, it's collected into the output of orchagent_restart_check and leads to test failure because the output is not as expected.

- profiling:/__w/1/s/common/.libs/libswsscommon_la-consumerstatetable.gcda:Merge mismatch for function 267
	[<TracebackEntry /agent/_work/1/s/tests/test_port_an.py:262>]

This PR fixes the issue by adding an argument include_stderr to runcmd. When include_stderr is set to False, only stdout is collected.

Why I did it
This PR is to fix the flaky test cases in vstest.

How I verified it
Verified by running vstest locally.

Details if related

@bingwang-ms bingwang-ms marked this pull request as ready for review December 19, 2022 11:55
@bingwang-ms bingwang-ms requested a review from prsunny as a code owner December 19, 2022 11:55
@bingwang-ms bingwang-ms changed the title Only collect stdout of orchagent_restart_check in vstest [vstest] Only collect stdout of orchagent_restart_check in vstest Dec 19, 2022
@prsunny prsunny merged commit c00455a into sonic-net:master Dec 19, 2022
yxieca pushed a commit that referenced this pull request Jan 4, 2023
*[vstest] Only collect stdout of orchagent_restart_check in vstest
StormLiangMS pushed a commit that referenced this pull request Feb 10, 2023
*[vstest] Only collect stdout of orchagent_restart_check in vstest
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants