Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add retry_call to test_platform_info mock check #3684

Conversation

slutati1536
Copy link
Contributor

Description of PR

the following cases:

  • test_show_platform_fanstatus_mocked
  • test_show_platform_temperature_mocked

are executing a command once and comparing the output to the expected mock data,
sometimes differences between the mock and the actual are causing the tests to fail. retry will make these tests more robust since those failures are very hard to reproduce and also they rarely happen.

Summary:
add retries to test in order to make them more stable

Type of change

  • Test case improvement

Back port request

  • 201911

Approach

What is the motivation for this PR?

those cases sometimes fail because of mock data mismatch, for example:
actual: {

"temperature": "34.0", 

"crit low th": "N/A", 

"high th": "70.0", 

"crit high th": "N/A", 

"warning": "False", 

"timestamp": "20210616 20:32:06", 

"low th": "N/A", 

"sensor": "PSU-1 Temp" 

},
expected:
"PSU-1 Temp": {

"temperature": "0.028", 

"crit low th": "N/A", 

"high th": "0.033", 

"crit high th": "N/A", 

"warning": "False", 

"low th": "N/A", 

"sensor": "PSU-1 Temp" 

},
retry will compare the actual info several times before failing, making the tests more stable.

How did you do it?

add my changes and tested them locally to check nothing was broken

How did you verify/test it?

run the cases locally

Any platform specific information?

no

@wangxin
Copy link
Collaborator

wangxin commented Jun 26, 2021

@slutati1536 Can you address the PR testing issue:

==================================== ERRORS ====================================
____________ ERROR collecting platform_tests/test_platform_info.py _____________
ImportError while importing test module '/var/src/s/tests/platform_tests/test_platform_info.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
platform_tests/test_platform_info.py:10: in <module>
    from retry.api import retry_call
E   ImportError: No module named retry.api

@liat-grozovik
Copy link
Collaborator

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Collaborator

@wangxin wangxin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the test failure caused by this change. Thanks!

@slutati1536 slutati1536 reopened this Aug 11, 2021
@slutati1536 slutati1536 reopened this Aug 12, 2021
@slutati1536
Copy link
Contributor Author

@ jleveque could you please review so this PR can be merged?

@liat-grozovik
Copy link
Collaborator

@keboliu , @Junchao-Mellanox and @wangxin kindly reminder please review and check if your comments were all addressed

@slutati1536 slutati1536 reopened this Aug 17, 2021
@liat-grozovik
Copy link
Collaborator

@wangxin could you please help to review

@slutati1536 slutati1536 requested review from wangxin and removed request for jleveque August 18, 2021 06:53
@@ -7,6 +7,7 @@
import json
import logging
import time
from retry.api import retry_call
Copy link
Collaborator

@wangxin wangxin Aug 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like a new dependency is introduced here: https://pypi.org/project/retry/
Is there a way to avoid this dependency?
If not, can you update the sonic-mgmt Dockerfile in the sonic-buildimage repo to include installation of this retry package?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I didn't notice that it has been added to the sonic-mgmt image in PR sonic-net/sonic-buildimage#7997. LGTM then.

@wangxin wangxin merged commit 734fda3 into sonic-net:master Sep 1, 2021
Blueve pushed a commit to Blueve/sonic-mgmt that referenced this pull request Sep 9, 2021
Blueve pushed a commit to Blueve/sonic-mgmt that referenced this pull request Sep 9, 2021
sonic-net#3684)"

Revert "add retry_call to test_platform_info mock check (sonic-net#3684)"

Reverted commit `734fda3d`.
wangxin pushed a commit that referenced this pull request Sep 11, 2021
What is the motivation for this PR?
The test case "test_thermal_control_fan_status" will failed after the PR #3684 is merged

How did you do it?
Change the return value of the check_result in AbnormalFanMocker, before the #3684 is merged, it dose not care about the return value of the check_result in AbnormalFanMocker, after the PR is merged, it will check the return value of the check_result

How did you verify/test it?
Run test cases in test_platform_info.py and all the test cases pass
vmittal-msft pushed a commit to vmittal-msft/sonic-mgmt that referenced this pull request Sep 28, 2021
vmittal-msft pushed a commit to vmittal-msft/sonic-mgmt that referenced this pull request Sep 28, 2021
What is the motivation for this PR?
The test case "test_thermal_control_fan_status" will failed after the PR sonic-net#3684 is merged

How did you do it?
Change the return value of the check_result in AbnormalFanMocker, before the sonic-net#3684 is merged, it dose not care about the return value of the check_result in AbnormalFanMocker, after the PR is merged, it will check the return value of the check_result

How did you verify/test it?
Run test cases in test_platform_info.py and all the test cases pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants