Add tests for new APIs for chassis and convert existing API tests to run on chassis #2985
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rawal01
changed the title
rawal01
changed the title
This comment has been minimized.
This comment has been minimized.
jleveque
suggested changes
Feb 12, 2021
rawal01
force-pushed
the
pmon-api-tests
branch
2 times, most recently
from
d3d5f94
to
2b294c5
Compare
|
This pull request introduces 1 alert when merging 2b294c53fe20fb78c88400e44221f1431bdbbc30 into 936bcac - view on LGTM.com new alerts:
|
jleveque
changed the title
jleveque
suggested changes
Mar 16, 2021
jleveque
approved these changes
Mar 16, 2021
|
@yxieca @vdahiya12 @sujinmkang Please review this PR and merge |
…n Chassis:
tests/common/helpers/platform_api/chassis.py
add new api calls: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis
tests/common/helpers/platform_api/fan_drawer.py
add new api call: get_maximum_consumed_power
tests/common/helpers/platform_api/module.py
add new api calls: get_description, get_slot, get_type, get_oper_status, get_midplane_ip,
is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state
tests/common/helpers/platform_api/psu.py
add new api calls: get_maximum_supplied_power, set_status_master_led, get_status_master_led
tests/common/helpers/platform_api/thermal.py
add new api calls: get_minimum_recorded, get_maximum_recorded
tests/platform_tests/api/conftest.py:
changes to run tests on Sonic chassis
tests/platform_tests/api/test_chassis.py:
changes to run tests on Sonic chassis
add tests for new apis for chassis: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis
tests/platform_tests/api/test_chassis_fans.py:
changes changes to run tests on Sonic chassis
tests/platform_tests/api/test_component.py:
changes changes to run tests on Sonic chassis
tests/platform_tests/api/test_fan_drawer.py:
changes changes to run tests on Sonic chassis,
add tests for new apis for fan_drawer: get_maximum_consumed_power
tests/platform_tests/api/test_fan_drawer_fans.py
changes changes to run tests on Sonic chassis
tests/platform_tests/api/test_module.py
changes changes to run tests on Sonic chassis,
add tests for new apis for module_base: get_description, get_slot, get_type, get_oper_status, get_midplane_ip,
is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state
tests/platform_tests/api/test_psu.py
changes changes to run tests on Sonic chassis,
add tests for new apis: get_maximum_supplied_power, set_status_master_led, get_status_master_led
tests/platform_tests/api/test_psu_fans.py
changes changes to run tests on Sonic chassis
tests/platform_tests/api/test_sfp.py
changes changes to run tests on Sonic chassis
tests/platform_tests/api/test_thermal.py
changes changes to run tests on Sonic chassis,
add tests for new apis for thermal: get_minimum_recorded, get_maximum_recorded
tests/platform_tests/api/test_watchdog.py
changes changes to run tests on Sonic chassis
…ns indent in test_module
|
@jleveque, could you help add the second reviewer, whoever you think best knows this area? We rebased and pushed the changes again. please help move this forward. thanks |
|
@yxieca: Can you please review this PR for sonic-mgmt best practices when you have a moment? |
wangxin
reviewed
Mar 31, 2021
wangxin
approved these changes
Apr 1, 2021
nirmalya-keysight
pushed a commit
to nirmalya-keysight/sonic-mgmt
that referenced
this pull request
Apr 5, 2021
…run on chassis (sonic-net#2985) What is the motivation for this PR? Add new test cases for api for test plan introduced by PR sonic-net#2695 sonic-net#2695 (refer to sections 2 to 6) Need to run existing api tests against a Sonice chassis but all of the tests were using duthost and hence will always run one of duthost of chassis i.e supervisor or line card. Converted these tests to use the enum_* fixture per hwsku for dut selection. Instead of running gather facts as fixture get facts within compare function for selected duthost. How did you do it? Following changes made to api helper files in folder sonic-mgmt/tests/common/helpers/platform_api/ chassis.py: add new api calls: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis fan_drawer.py: add new api call: get_maximum_consumed_power module.py: add new api calls: get_description, get_slot, get_type, get_oper_status, get_midplane_ip, is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state psu.py: add new api calls: get_maximum_supplied_power, set_status_master_led, get_status_master_led thermal.py: add new api calls: get_minimum_recorded, get_maximum_recorded changes maded to api tests to support Sonic Chassis and add tests for new apis: conftest.py: changes to run tests on Sonic chassis change plaftorm_api_conn to support multidut by using duthosts and enum_rand_one_per_hwsku_hostname to select dut at function level, change from getting ip from eth0 to duthost.mgp_ip to extract ip address for DUT change start_platform_service to support multidut by using duthosts and enum_rand_one_per_hwsku_hostname to select dut at function level, change from getting ip from eth0 to duthost.mgp_ip to extract ip address for DUT change in stop_platform_api_service: to support on all duts where the service was started for all test modules under api these are common changes : Remove gather_facts fixture and get facts within compare_value_with_platform_facts per duthost from test changes changes to run tests on Sonic chassis, Replace duthost with duthosts and enum_rand_one_per_hwsku_hostname for multidut environment for all tests other changes per module: test_chassis.py: add tests for new apis for chassis: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis test_component.py: remove range since image_list is list for image in range(image_list) test_fan_drawer.py: add test for new apis for fan_drawer: get_maximum_consumed_power test_module.py add tests for new apis for module_base: get_description, get_slot, get_type, get_oper_status, get_midplane_ip, is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state test_psu.py changes changes to run tests on Sonic chassis, add tests for new apis: get_maximum_supplied_power, set_status_master_led, get_status_master_led test_thermal.py changes changes to run tests on Sonic chassis, add tests for new apis for thermal: get_minimum_recorded, get_maximum_recorded How did you verify/test it? Validated the modified tests against chassis.
saravanansv
pushed a commit
to saravanansv/sonic-mgmt
that referenced
this pull request
May 6, 2021
…run on chassis (sonic-net#2985) What is the motivation for this PR? Add new test cases for api for test plan introduced by PR sonic-net#2695 sonic-net#2695 (refer to sections 2 to 6) Need to run existing api tests against a Sonice chassis but all of the tests were using duthost and hence will always run one of duthost of chassis i.e supervisor or line card. Converted these tests to use the enum_* fixture per hwsku for dut selection. Instead of running gather facts as fixture get facts within compare function for selected duthost. How did you do it? Following changes made to api helper files in folder sonic-mgmt/tests/common/helpers/platform_api/ chassis.py: add new api calls: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis fan_drawer.py: add new api call: get_maximum_consumed_power module.py: add new api calls: get_description, get_slot, get_type, get_oper_status, get_midplane_ip, is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state psu.py: add new api calls: get_maximum_supplied_power, set_status_master_led, get_status_master_led thermal.py: add new api calls: get_minimum_recorded, get_maximum_recorded changes maded to api tests to support Sonic Chassis and add tests for new apis: conftest.py: changes to run tests on Sonic chassis change plaftorm_api_conn to support multidut by using duthosts and enum_rand_one_per_hwsku_hostname to select dut at function level, change from getting ip from eth0 to duthost.mgp_ip to extract ip address for DUT change start_platform_service to support multidut by using duthosts and enum_rand_one_per_hwsku_hostname to select dut at function level, change from getting ip from eth0 to duthost.mgp_ip to extract ip address for DUT change in stop_platform_api_service: to support on all duts where the service was started for all test modules under api these are common changes : Remove gather_facts fixture and get facts within compare_value_with_platform_facts per duthost from test changes changes to run tests on Sonic chassis, Replace duthost with duthosts and enum_rand_one_per_hwsku_hostname for multidut environment for all tests other changes per module: test_chassis.py: add tests for new apis for chassis: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis test_component.py: remove range since image_list is list for image in range(image_list) test_fan_drawer.py: add test for new apis for fan_drawer: get_maximum_consumed_power test_module.py add tests for new apis for module_base: get_description, get_slot, get_type, get_oper_status, get_midplane_ip, is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state test_psu.py changes changes to run tests on Sonic chassis, add tests for new apis: get_maximum_supplied_power, set_status_master_led, get_status_master_led test_thermal.py changes changes to run tests on Sonic chassis, add tests for new apis for thermal: get_minimum_recorded, get_maximum_recorded How did you verify/test it? Validated the modified tests against chassis.
vmittal-msft
pushed a commit
to vmittal-msft/sonic-mgmt
that referenced
this pull request
Sep 28, 2021
…run on chassis (sonic-net#2985) What is the motivation for this PR? Add new test cases for api for test plan introduced by PR sonic-net#2695 sonic-net#2695 (refer to sections 2 to 6) Need to run existing api tests against a Sonice chassis but all of the tests were using duthost and hence will always run one of duthost of chassis i.e supervisor or line card. Converted these tests to use the enum_* fixture per hwsku for dut selection. Instead of running gather facts as fixture get facts within compare function for selected duthost. How did you do it? Following changes made to api helper files in folder sonic-mgmt/tests/common/helpers/platform_api/ chassis.py: add new api calls: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis fan_drawer.py: add new api call: get_maximum_consumed_power module.py: add new api calls: get_description, get_slot, get_type, get_oper_status, get_midplane_ip, is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state psu.py: add new api calls: get_maximum_supplied_power, set_status_master_led, get_status_master_led thermal.py: add new api calls: get_minimum_recorded, get_maximum_recorded changes maded to api tests to support Sonic Chassis and add tests for new apis: conftest.py: changes to run tests on Sonic chassis change plaftorm_api_conn to support multidut by using duthosts and enum_rand_one_per_hwsku_hostname to select dut at function level, change from getting ip from eth0 to duthost.mgp_ip to extract ip address for DUT change start_platform_service to support multidut by using duthosts and enum_rand_one_per_hwsku_hostname to select dut at function level, change from getting ip from eth0 to duthost.mgp_ip to extract ip address for DUT change in stop_platform_api_service: to support on all duts where the service was started for all test modules under api these are common changes : Remove gather_facts fixture and get facts within compare_value_with_platform_facts per duthost from test changes changes to run tests on Sonic chassis, Replace duthost with duthosts and enum_rand_one_per_hwsku_hostname for multidut environment for all tests other changes per module: test_chassis.py: add tests for new apis for chassis: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis test_component.py: remove range since image_list is list for image in range(image_list) test_fan_drawer.py: add test for new apis for fan_drawer: get_maximum_consumed_power test_module.py add tests for new apis for module_base: get_description, get_slot, get_type, get_oper_status, get_midplane_ip, is_midplane_reachable, get_maximum_consumed_power, reboot, set_admin_state test_psu.py changes changes to run tests on Sonic chassis, add tests for new apis: get_maximum_supplied_power, set_status_master_led, get_status_master_led test_thermal.py changes changes to run tests on Sonic chassis, add tests for new apis for thermal: get_minimum_recorded, get_maximum_recorded How did you verify/test it? Validated the modified tests against chassis.
gechiang
added a commit
that referenced
this pull request
Aug 17, 2022
in test_get_presence() there is a new check added by PR (#2985): name = module.get_name(platform_api_conn, i) if name in self.skip_mod_list: self.expect(presence is False, "Module {} is not present".format(i)) else: self.expect(presence is True, "Module {} is not present".format(i)) I am seeing some issue with the above logic. In our testbed where we share a physical chassis that uses the same Supervisor card but group different LCs for different "logical chassis" so for some "logical chassis" some LCs are added to "skip_mod_list" as they are not meant to be tested for that logical chassis. Our expectation is that if it is marked as skipped, it is not meant for testing and no state of those skipped modules should be used for any validation logic. We should always trust what the inventory marked as skipped and not try to look at whatever state the "skipped module" should be.
allen-xf
pushed a commit
to allen-xf/sonic-mgmt
that referenced
this pull request
Oct 28, 2022
in test_get_presence() there is a new check added by PR (sonic-net#2985): name = module.get_name(platform_api_conn, i) if name in self.skip_mod_list: self.expect(presence is False, "Module {} is not present".format(i)) else: self.expect(presence is True, "Module {} is not present".format(i)) I am seeing some issue with the above logic. In our testbed where we share a physical chassis that uses the same Supervisor card but group different LCs for different "logical chassis" so for some "logical chassis" some LCs are added to "skip_mod_list" as they are not meant to be tested for that logical chassis. Our expectation is that if it is marked as skipped, it is not meant for testing and no state of those skipped modules should be used for any validation logic. We should always trust what the inventory marked as skipped and not try to look at whatever state the "skipped module" should be.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please make sure you've read and understood our contributing guidelines;
https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md
Please provide following information to help code review process a bit easier:
-->
Description of PR
Summary:
Fixes # (issue)
Type of change
Approach
What is the motivation for this PR?
Add new test cases for api for test plan introduced by PR #2695 #2695 (refer to sections 2 to 6)
Need to run existing api tests against a Sonice chassis but all of the tests were using duthost and hence will always run one of duthost of chassis i.e supervisor or line card. Converted these tests to use the enum_* fixture per hwsku for dut selection. Instead of running gather facts as fixture get facts within compare function for selected duthost.
How did you do it?
Following changes made to api helper files in folder sonic-mgmt/tests/common/helpers/platform_api/
chassis.py:
add new api calls: get_module_index, get_supervisor_slot, get_my_slot, is_modular_chassis
changes maded to api tests to support Sonic Chassis and add tests for new apis:
for all test modules under api these are common changes :
Remove gather_facts fixture and get facts within compare_value_with_platform_facts per duthost from test
changes changes to run tests on Sonic chassis, Replace duthost with duthosts and enum_rand_one_per_hwsku_hostname for multidut environment for all tests
other changes per module:
How did you verify/test it?
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation