Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make CRM counters available for route_perf test #2477

Merged
merged 4 commits into from
Nov 6, 2020

Conversation

vaibhavhd
Copy link
Contributor

@vaibhavhd vaibhavhd commented Nov 6, 2020

Description of PR

Summary: Add a fixture to make CRM counters available for test_route_perf
Fixes # (issue)

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • Test case(new/improvement)

Approach

What is the motivation for this PR?

A regression has been noticed in test_route_perf after #2461 - in some platforms when the CRM counters are not ready, the test fails with:

>       used_routes_count = duthost.get_crm_resources().get("main_resources").get(route_tag).get("used")
E       AttributeError: 'NoneType' object has no attribute 'get'

NUM_NEIGHS = 8

How did you do it?

Reduce the CRM polling limit at the start of the test from 300s to 1s. Reset this limit at the end of the test.

How did you verify/test it?

Tested a newly warm-rebooted device (no CRM counters), the CRM counters get set due to reduced polling interval (1s) and the test passes:

------------------------------------------------------------------------ live log call -------------------------------------------------------------------------
07:21:19 INFO test_route_perf.py:test_perf_add_remove_routes:203: IP route utilization before test start: Used: 6475, Available: 67253, Test count: 10000
07:22:18 INFO test_route_perf.py:test_perf_add_remove_routes:218: Time to set 10000 ipv4 routes is 26.57 seconds.
07:23:06 INFO test_route_perf.py:test_perf_add_remove_routes:222: Time to del 10000 ipv4 routes is 27.34 seconds.
PASSED                                                                                                                                                   [ 50%]
route/test_route_perf.py::test_perf_add_remove_routes[6]
------------------------------------------------------------------------ live log setup ------------------------------------------------------------------------
07:23:28 INFO __init__.py:set_default:49: Completeness level not set during test execution. Setting to default level: CompletenessLevel.basic
07:23:28 INFO __init__.py:check_test_completeness:128: Test has no defined levels. Continue without test completeness checks
07:23:30 INFO __init__.py:loganalyzer:32: Add start marker into DUT syslog
07:23:31 INFO __init__.py:loganalyzer:34: Load config and analyze log
------------------------------------------------------------------------ live log call -------------------------------------------------------------------------
07:23:33 INFO test_route_perf.py:test_perf_add_remove_routes:203: IP route utilization before test start: Used: 6475, Available: 1717, Test count: 1717
07:23:57 INFO test_route_perf.py:test_perf_add_remove_routes:218: Time to set 1717 ipv6 routes is 5.15 seconds.
07:24:09 INFO test_route_perf.py:test_perf_add_remove_routes:222: Time to del 1717 ipv6 routes is 4.90 seconds.
PASSED                                                                                                                                                   [100%]    

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

@vaibhavhd vaibhavhd requested a review from a team November 6, 2020 07:31
@lgtm-com
Copy link

lgtm-com bot commented Nov 6, 2020

This pull request introduces 1 alert when merging c56651f into 51fb3bb - view on LGTM.com

new alerts:

  • 1 for Unused import

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants