Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[202012] Backport SAI failure handling to 202012 branch #1880

Merged
merged 6 commits into from
Aug 18, 2021

Conversation

shi-su
Copy link
Contributor

@shi-su shi-su commented Aug 18, 2021

What I did
Backport SAI failure handling related commits into the 202012 branch. The following is a list of backported commits:

941875a Deactivate mirror session only when session status is true in updateLagMember (#1666)
be12482 Ignore ALREADY_EXIST error in FDB creation (#1815)
c9c1aa2 Add failure handling for SAI get operations (#1768)
47b4276 [BufferOrch] Don't call SAI API for BUFFER_POOL/PROFILE handling in case the op is DEL and the SAI OID is NULL (#1786)
db9238f Add failure notification for orchagent (#1665)
fc8e43f [synchronous mode] Add failure notification for SAI failures in synchronous mode (#1596)

Why I did it
202012 image needs to include failure handling mechanism for enough notification in the presence of SAI failures.

How I verified it

Details if related

shi-su and others added 6 commits August 17, 2021 17:26
…ronous mode (sonic-net#1596)

Add function to notify users in the presence of SAI failures by throwing exceptions and trigger swss restart in synchronous mode. And include this function in routeorch and neighorch.

This is a part of the first step in the SAI failure handling in orchagent with synchronous mode:
    1. Failure notification mechanism to ensure enough notifications in the presence of SAI failures and avoid running switches with unhandled failures.
    2. Add general failure handling logic by status.
    3. Develop fine-grain failure handling mechanism for each orch to properly handle different SAI failures.

This function aims to ensure enough notifications in the presence of SAI failures and avoid running switches with unhandled failures (on-par with asynchronous mode).
What I did
Add SAI failure handling functions in aclorch, bufferorch, copporch, dtelorch, fdborch, fgnhgorch, intfsorch, mirrororch, natorch, policerorch, macsecorch, portsorch, qosorch, sfloworch, switchorch, tunneldecaporch, vrforch.

Why I did it
Failure notification mechanism to ensure enough notifications in the presence of SAI failures and avoid running switches with unhandled failures.
…ase the op is DEL and the SAI OID is NULL (sonic-net#1786)

- What I did
Don't call SAI API for BUFFER_POOL/PROFILE handling in case the op is DEL and the SAI OID is NULL in order to avoid orchagent from exiting.
We need it only in 202106 or above. In 202012 the orchagent won't exit in such case.

- Why I did it
Handle rare cases which cause SAI error eventually makes orchagent to exit.

- How I verified it
Manually test.

Signed-off-by: Stephen Sun <stephens@nvidia.com>
What I did
Add failure handling for SAI get operations. The function allows handling failures in SAI get operations according to the orch type, SAI type, SAI status.

Why I did it
Enable custom failure handling for SAI get operations.
What I did
Ignore ALREADY_EXIST error in FDB creation.
Fix: sonic-net/sonic-buildimage#7798

Why I did it
In FDB creation, there are scenarios where the hardware learns an FDB entry before orchagent. In such cases, the FDB SAI creation would report the status of SAI_STATUS_ITEM_ALREADY_EXISTS, and orchagent should ignore the error and treat it as entry was explicitly created.
…agMember (sonic-net#1666)

Deactivate mirror session only when the status is true in updateLagMember
@shi-su shi-su requested a review from qiluo-msft August 18, 2021 05:05
@shi-su shi-su marked this pull request as ready for review August 18, 2021 05:05
@shi-su
Copy link
Contributor Author

shi-su commented Aug 18, 2021

The LGTM failure does not seem related to this PR, and it fails even for the build without the changes.

@qiluo-msft
Copy link
Contributor

Could you listed the commits (master branch) you have backported in the PR description?

@lgtm-com
Copy link

lgtm-com bot commented Aug 18, 2021

LGTM pull request analysis was skipped for 941875a by shi-su. Analysis of future commits will happen as normal.

@shi-su shi-su merged commit c1cb2ca into sonic-net:202012 Aug 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants