Examine the tx counters in WRRtest instead of counting what arrives #15353
Conversation
…t PTF This test causes a burst of traffic to be transmitted by the DUT, but the intervening network between the DUT and PTF may not be capable of handling the entire burst of packets at full linerate. Instead inspect the packet transmit counters on the appropriate interface to ensure that at least the correct number of packets were transmitted, and ensure that at least 1 of the packets are observed at PTF.
byu343
force-pushed
the
master-dwrr-test
branch
from
d40f908 to
faa0f69
Compare
|
hi @XuChen-MSFT @kevinskwang could you help to take a look? |
| # At least the correct number of packets must have been transmitted | ||
| assert (q_cnt_sum <= txPacketCount) | ||
| # And at least one of the valid packets has arrived at PTF | ||
| assert (total_pkts > 1) |
There was a problem hiding this comment.
If q_cnt_sum > total_pkts is caused by PTF performance issues, especially when no any queue has fully received packets, it is impossible to perform the following check:
assert diff < limit, "Difference for %d is %d which exceeds limit %d" % (dscp, diff, limit)
In this case, we cannot determine whether the WRR test has passed. However, the following assertion allows the test to pass:
assert (total_pkts > 1)
Should we modify the test case to still fail, so we know we haven't got a test result? and then we can decide rerun or xfail it.
Regarding the PTF performance limit, can we determine the answer by comparing counters per queue (like: show que cou) instead of per port,
or implement similar test using Snappi?
There was a problem hiding this comment.
For that assert diff < limit check to test the round robin, it seems to me that it requires to know in which order packets are delivered. There is a large gap now between the number of packets delivered and total sent for the performance issue. so I agree that we could not really verify it.
As for your suggestion of comparing counters per queue, can you elaborate what and how to verify using it?
Description of PR
Summary:
test/test_qos_sai.py testQosSaiDwrr causes a burst of traffic to be transmitted by the DUT, but the intervening network between the DUT and PTF may not be capable of handling the entire burst of packets at full linerate. Instead inspect the packet transmit counters on the appropriate interface to ensure that at least the correct number of packets were transmitted, and ensure that at least 1 of the packets are observed at PTF.
Fixes # (issue)
Type of change
Back port request
Approach
What is the motivation for this PR?
test/test_qos_sai.py testQosSaiDwrr causes a burst of traffic to be transmitted by the DUT, but the intervening network between the DUT and PTF may not be capable of handling the entire burst of packets at full linerate.
How did you do it?
Instead inspect the packet transmit counters on the appropriate interface to ensure that at least the correct number of packets were transmitted, and ensure that at least 1 of the packets are observed at PTF.
How did you verify/test it?
The test can pass after the change.
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation