Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Occasional failure with test_route_flap #16438

Open
StormLiangMS opened this issue Sep 4, 2023 · 1 comment
Open

Occasional failure with test_route_flap #16438

StormLiangMS opened this issue Sep 4, 2023 · 1 comment
Assignees
Labels
NVIDIA Triaged this issue has been triaged

Comments

@StormLiangMS
Copy link
Contributor

Description

There are teardown errors when run test_route_flap, this issue should be a false alarm. Since the route which is not offloaded could be a transition state when enabled fib suppress?

In below PR, it start to check rt_frr_miss, but it may not an issue, since it is possible status during offloading.
Need to change the error print of monit, at least exclude rt_frr_miss

if results:
    print_message(syslog.LOG_WARNING, "Failure results: {",  json.dumps(results, indent=4), "}")
    print_message(syslog.LOG_WARNING, "Failed. Look at reported mismatches above")
    print_message(syslog.LOG_WARNING, "add: ", json.dumps(adds, indent=4))
    print_message(syslog.LOG_WARNING, "del: ", json.dumps(deletes, indent=4))

https://github.com/sonic-net/sonic-utilities/pull/2762/files#diff-f439f7943825c8e2474af7986da58a2b53fbbbb23cde03cef170b760a6a3f2df

Match Messages:
E Aug 24 19:08:08.485455 str2-7050cx3-acs-02 ERR monit[608]: 'routeCheck' status failed (255) -- Failure results: {{#12 "missed_FRR_routes": [#12 {#12 "destSelected": true,#12 "distance": 20,#12 "failed": true,#12 "installedNexthopGroupId": 128,#12 "internalFlags": 8,#12 "internalNextHopActiveNum": 4,#12 "internalNextHopNum": 4,#12 "internalStatus": 168,#12 "metric": 0,#12 "nexthopGroupId": 128,#12 "nexthops": [#12 {#12 "active": true,#12 "afi": "ipv4",#12 "fib": true,#12 "flags": 3,#12 "interfaceIndex": 5,#12 "interfaceName": "PortChannel101",#12 "ip": "10.0.0.57",#12 "weight": 1#012 },
E
E Aug 24 19:14:08.863385 str2-7050cx3-acs-02 ERR monit[608]: 'routeCheck' status failed (255) -- Failure results: {{#12 "missed_FRR_routes": [#12 {#12 "destSelected": true,#12 "distance": 20,#12 "failed": true,#12 "installedNexthopGroupId": 128,#12 "internalFlags": 8,#12 "internalNextHopActiveNum": 4,#12 "internalNextHopNum": 4,#12 "internalStatus": 168,#12 "metric": 0,#12 "nexthopGroupId": 128,#12 "nexthops": [#12 {#12 "active": true,#12 "afi": "ipv4",#12 "fib": true,#12 "flags": 3,#12 "interfaceIndex": 5,#12 "interfaceName": "PortChannel101",#12 "ip": "10.0.0.57",#12 "weight": 1#012 },
E
E Aug 24 19:20:09.361216 str2-7050cx3-acs-02 ERR monit[608]: 'routeCheck' status failed (255) -- Failure results: {{#12 "missed_FRR_routes": [#12 {#12 "destSelected": true,#12 "distance": 20,#12 "failed": true,#12 "installedNexthopGroupId": 128,#12 "internalFlags": 8,#12 "internalNextHopActiveNum": 4,#12 "internalNextHopNum": 4,#12 "internalStatus": 168,#12 "metric": 0,#12 "nexthopGroupId": 128,#12 "nexthops": [#12 {#12 "active": true,#12 "afi": "ipv4",#12 "fib": true,#12 "flags": 3,#12 "interfaceIndex": 5,#12 "interfaceName": "PortChannel101",#12 "ip": "10.0.0.57",#12 "weight": 1#012 },
E
E Aug 24 19:26:09.800484 str2-7050cx3-acs-02 ERR monit[608]: 'routeCheck' status failed (255) -- Failure results: {{#12 "missed_FRR_routes": [#12 {#12 "destSelected": true,#12 "distance": 20,#12 "failed": true,#12 "installedNexthopGroupId": 128,#12 "internalFlags": 8,#12 "internalNextHopActiveNum": 4,#12 "internalNextHopNum": 4,#12 "internalStatus": 168,#12 "metric": 0,#12 "nexthopGroupId": 128,#12 "nexthops": [#12 {#12 "active": true,#12 "afi": "ipv4",#12 "fib": true,#12 "flags": 3,#12 "interfaceIndex": 5,#12 "interfaceName": "PortChannel101",#12 "ip": "10.0.0.57",#12 "weight": 1#012 },
E
E Aug 24 19:32:10.217737 str2-7050cx3-acs-02 ERR monit[608]: 'routeCheck' status failed (255) -- Failure results: {{#12 "missed_FRR_routes": [#12 {#12 "destSelected": true,#12 "distance": 20,#12 "failed": true,#12 "installedNexthopGroupId": 128,#12 "internalFlags": 8,#12 "internalNextHopActiveNum": 4,#12 "internalNextHopNum": 4,#12 "internalStatus": 168,#12 "metric": 0,#12 "nexthopGroupId": 128,#12 "nexthops": [#12 {#12 "active": true,#12 "afi": "ipv4",#12 "fib": true,#12 "flags": 3,#12 "interfaceIndex": 5,#12 "interfaceName": "PortChannel101",#12 "ip": "10.0.0.57",#12 "weight": 1#012 },
E

Steps to reproduce the issue:

Describe the results you received:

Describe the results you expected:

Output of show version:

(paste your output here)

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

@prgeor
Copy link
Contributor

prgeor commented Sep 13, 2023

@stepanblyschak please help to see if introudced by your pr changes

@prgeor prgeor added the Triaged this issue has been triaged label Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NVIDIA Triaged this issue has been triaged
Projects
None yet
Development

No branches or pull requests

3 participants