Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Checkpoint and Rollback for Multi ASIC. #3299

Merged
merged 48 commits into from
Jun 14, 2024
Merged

Add Checkpoint and Rollback for Multi ASIC. #3299

merged 48 commits into from
Jun 14, 2024

Conversation

xincunli-sonic
Copy link
Contributor

@xincunli-sonic xincunli-sonic commented Apr 30, 2024 •
edited
Loading

What I did

Add config checkpoint, rollback, replace, list-checkpoints, delete-checkpoint support of Multi ASIC

How I did it

Add namespace for each of operation to support Multi ASIC.

How to verify it

  1. Single ASIC
Config Rollbacker: Config checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Getting current config db.
Config Rollbacker: Getting checkpoint full-path.
Config Rollbacker: Ensuring checkpoint directory exist.
Config Rollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
Config Rollbacker: Config checkpoint completed.
Checkpoint created successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

admin@str2-msn2700-spy-1:~/gcu$ sudo config rollback 20240522-xincun
Config Rollbacker: Config rollbacking starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Verifying '20240522-xincun' exists.
Config Rollbacker: Loading checkpoint into memory.
Config Rollbacker: Replacing config using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 71214.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Rollbacker: Config rollbacking completed.
Config rolled back successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config delete-checkpoint 20240522-xincun
Config Rollbacker: Deleting checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Checking checkpoint exists.
Config Rollbacker: Deleting checkpoint.
Config Rollbacker: Deleting checkpoint completed.
Checkpoint deleted successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[]
  1. Multi ASIC
stli@str2-7250-2-lc01:~/gcu$ sudo config checkpoint 20240522-xincun
MultiASICConfigRollbacker: Config checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Getting current  config db.
MultiASICConfigRollbacker: Getting current asic0 config db.
MultiASICConfigRollbacker: Getting current asic1 config db.
MultiASICConfigRollbacker: Getting checkpoint full-path.
MultiASICConfigRollbacker: Ensuring checkpoint directory exist.
MultiASICConfigRollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
MultiASICConfigRollbacker: Config checkpoint completed.
Checkpoint created successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

stli@str2-7250-2-lc01:~/gcu$ sudo config rollback 20240522-xincun
MultiASICConfigRollbacker: Config rollbacking starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Verifying '20240522-xincun' exists.
MultiASICConfigRollbacker: Loading checkpoint '20240522-xincun' into memory.
MultiASICConfigRollbacker: Replacing config '20240522-xincun' using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
MultiASICConfigRollbacker: Config rollbacking completed.
Config rolled back successfully.

stli@str2-7250-2-lc01:~/gcu$ sudo config delete-checkpoint 20240522-xincun
MultiASICConfigRollbacker: Deleting checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Checking checkpoint: 20240522-xincun exists.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun completed.
Checkpoint deleted successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[]

stli@str2-7250-2-lc01:~/gcu$ sudo config replace 20240522-xincun.cp.json 
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config replaced successfully.

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

qiluo-msft
qiluo-msft reviewed May 1, 2024
View reviewed changes
config/main.py Outdated Show resolved Hide resolved
qiluo-msft
qiluo-msft reviewed May 1, 2024
View reviewed changes
config/main.py Outdated Show resolved Hide resolved
qiluo-msft
qiluo-msft reviewed May 1, 2024
View reviewed changes
config/main.py Outdated Show resolved Hide resolved
@qiluo-msft
Copy link
Contributor

qiluo-msft commented May 1, 2024 •
edited
Loading 

             namespace=multi_asic.DEFAULT_NAMESPACE):

suggest naming: namespace -> scope #Closed

Refers to: generic_config_updater/generic_updater.py:61 in b296200. [](commit_id = b296200, deletion_comment = False)

qiluo-msft
qiluo-msft reviewed May 1, 2024
View reviewed changes
config/main.py Outdated Show resolved Hide resolved
xincunli-sonic and others added 17 commits May 17, 2024 09:37
wen587
wen587 previously approved these changes Jun 7, 2024
View reviewed changes
Copy link
Contributor

@wen587 wen587 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@xincunli-sonic
Copy link
Contributor Author

/azpw run

@mssonicbld
Copy link
Collaborator

/AzurePipelines run

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@qiluo-msft qiluo-msft merged commit d0856af into sonic-net:master Jun 14, 2024
7 checks passed
@xincunli-sonic xincunli-sonic mentioned this pull request Jun 14, 2024
Merged
arfeigin pushed a commit to arfeigin/sonic-utilities that referenced this pull request Jun 16, 2024
@xincunli-sonic @arfeigin
Add Checkpoint and Rollback for Multi ASIC. (sonic-net#3299)
cdced93 
#### What I did

Add `config` `checkpoint`, `rollback`, `replace`, `list-checkpoints`, `delete-checkpoint` support of Multi ASIC

#### How I did it
Add namespace for each of operation to support Multi ASIC.

#### How to verify it
1. Single ASIC
```admin@str2-msn2700-spy-1:~/gcu$ sudo config checkpoint 20240522-xincun
Config Rollbacker: Config checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Getting current config db.
Config Rollbacker: Getting checkpoint full-path.
Config Rollbacker: Ensuring checkpoint directory exist.
Config Rollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
Config Rollbacker: Config checkpoint completed.
Checkpoint created successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

admin@str2-msn2700-spy-1:~/gcu$ sudo config rollback 20240522-xincun
Config Rollbacker: Config rollbacking starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Verifying '20240522-xincun' exists.
Config Rollbacker: Loading checkpoint into memory.
Config Rollbacker: Replacing config using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 71214.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Rollbacker: Config rollbacking completed.
Config rolled back successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config delete-checkpoint 20240522-xincun
Config Rollbacker: Deleting checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Checking checkpoint exists.
Config Rollbacker: Deleting checkpoint.
Config Rollbacker: Deleting checkpoint completed.
Checkpoint deleted successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[]
```

2. Multi ASIC
```
stli@str2-7250-2-lc01:~/gcu$ sudo config checkpoint 20240522-xincun
MultiASICConfigRollbacker: Config checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Getting current  config db.
MultiASICConfigRollbacker: Getting current asic0 config db.
MultiASICConfigRollbacker: Getting current asic1 config db.
MultiASICConfigRollbacker: Getting checkpoint full-path.
MultiASICConfigRollbacker: Ensuring checkpoint directory exist.
MultiASICConfigRollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
MultiASICConfigRollbacker: Config checkpoint completed.
Checkpoint created successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

stli@str2-7250-2-lc01:~/gcu$ sudo config rollback 20240522-xincun
MultiASICConfigRollbacker: Config rollbacking starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Verifying '20240522-xincun' exists.
MultiASICConfigRollbacker: Loading checkpoint '20240522-xincun' into memory.
MultiASICConfigRollbacker: Replacing config '20240522-xincun' using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
MultiASICConfigRollbacker: Config rollbacking completed.
Config rolled back successfully.

stli@str2-7250-2-lc01:~/gcu$ sudo config delete-checkpoint 20240522-xincun
MultiASICConfigRollbacker: Deleting checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Checking checkpoint: 20240522-xincun exists.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun completed.
Checkpoint deleted successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[]

stli@str2-7250-2-lc01:~/gcu$ sudo config replace 20240522-xincun.cp.json 
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config replaced successfully.
```
@gechiang
Copy link
Contributor

@xincunli-sonic , please supply the MSFT ADO for this PR

@xincunli-sonic
Copy link
Contributor Author

msft ADO: 28463590

mssonicbld pushed a commit to mssonicbld/sonic-utilities that referenced this pull request Aug 2, 2024
@xincunli-sonic @mssonicbld
Add Checkpoint and Rollback for Multi ASIC. (sonic-net#3299)
0095835 
#### What I did

Add `config` `checkpoint`, `rollback`, `replace`, `list-checkpoints`, `delete-checkpoint` support of Multi ASIC

#### How I did it
Add namespace for each of operation to support Multi ASIC.

#### How to verify it
1. Single ASIC
```admin@str2-msn2700-spy-1:~/gcu$ sudo config checkpoint 20240522-xincun
Config Rollbacker: Config checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Getting current config db.
Config Rollbacker: Getting checkpoint full-path.
Config Rollbacker: Ensuring checkpoint directory exist.
Config Rollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
Config Rollbacker: Config checkpoint completed.
Checkpoint created successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

admin@str2-msn2700-spy-1:~/gcu$ sudo config rollback 20240522-xincun
Config Rollbacker: Config rollbacking starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Verifying '20240522-xincun' exists.
Config Rollbacker: Loading checkpoint into memory.
Config Rollbacker: Replacing config using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 71214.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Rollbacker: Config rollbacking completed.
Config rolled back successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config delete-checkpoint 20240522-xincun
Config Rollbacker: Deleting checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Checking checkpoint exists.
Config Rollbacker: Deleting checkpoint.
Config Rollbacker: Deleting checkpoint completed.
Checkpoint deleted successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[]
```

2. Multi ASIC
```
stli@str2-7250-2-lc01:~/gcu$ sudo config checkpoint 20240522-xincun
MultiASICConfigRollbacker: Config checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Getting current  config db.
MultiASICConfigRollbacker: Getting current asic0 config db.
MultiASICConfigRollbacker: Getting current asic1 config db.
MultiASICConfigRollbacker: Getting checkpoint full-path.
MultiASICConfigRollbacker: Ensuring checkpoint directory exist.
MultiASICConfigRollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
MultiASICConfigRollbacker: Config checkpoint completed.
Checkpoint created successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

stli@str2-7250-2-lc01:~/gcu$ sudo config rollback 20240522-xincun
MultiASICConfigRollbacker: Config rollbacking starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Verifying '20240522-xincun' exists.
MultiASICConfigRollbacker: Loading checkpoint '20240522-xincun' into memory.
MultiASICConfigRollbacker: Replacing config '20240522-xincun' using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
MultiASICConfigRollbacker: Config rollbacking completed.
Config rolled back successfully.

stli@str2-7250-2-lc01:~/gcu$ sudo config delete-checkpoint 20240522-xincun
MultiASICConfigRollbacker: Deleting checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Checking checkpoint: 20240522-xincun exists.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun completed.
Checkpoint deleted successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[]

stli@str2-7250-2-lc01:~/gcu$ sudo config replace 20240522-xincun.cp.json 
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config replaced successfully.
```
@mssonicbld mssonicbld mentioned this pull request Aug 2, 2024
Merged
@mssonicbld
Copy link
Collaborator

Cherry-pick PR to 202405: #3468

mssonicbld pushed a commit that referenced this pull request Aug 2, 2024
@xincunli-sonic @mssonicbld
Add Checkpoint and Rollback for Multi ASIC. (#3299)
a9876ee 
#### What I did

Add `config` `checkpoint`, `rollback`, `replace`, `list-checkpoints`, `delete-checkpoint` support of Multi ASIC

#### How I did it
Add namespace for each of operation to support Multi ASIC.

#### How to verify it
1. Single ASIC
```admin@str2-msn2700-spy-1:~/gcu$ sudo config checkpoint 20240522-xincun
Config Rollbacker: Config checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Getting current config db.
Config Rollbacker: Getting checkpoint full-path.
Config Rollbacker: Ensuring checkpoint directory exist.
Config Rollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
Config Rollbacker: Config checkpoint completed.
Checkpoint created successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

admin@str2-msn2700-spy-1:~/gcu$ sudo config rollback 20240522-xincun
Config Rollbacker: Config rollbacking starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Verifying '20240522-xincun' exists.
Config Rollbacker: Loading checkpoint into memory.
Config Rollbacker: Replacing config using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 71214.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Rollbacker: Config rollbacking completed.
Config rolled back successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config delete-checkpoint 20240522-xincun
Config Rollbacker: Deleting checkpoint starting.
Config Rollbacker: Checkpoint name: 20240522-xincun.
Config Rollbacker: Checking checkpoint exists.
Config Rollbacker: Deleting checkpoint.
Config Rollbacker: Deleting checkpoint completed.
Checkpoint deleted successfully.

admin@str2-msn2700-spy-1:~/gcu$ sudo config list-checkpoints 
[]
```

2. Multi ASIC
```
stli@str2-7250-2-lc01:~/gcu$ sudo config checkpoint 20240522-xincun
MultiASICConfigRollbacker: Config checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Getting current  config db.
MultiASICConfigRollbacker: Getting current asic0 config db.
MultiASICConfigRollbacker: Getting current asic1 config db.
MultiASICConfigRollbacker: Getting checkpoint full-path.
MultiASICConfigRollbacker: Ensuring checkpoint directory exist.
MultiASICConfigRollbacker: Saving config db content to /etc/sonic/checkpoints/20240522-xincun.cp.json.
MultiASICConfigRollbacker: Config checkpoint completed.
Checkpoint created successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[
    "20240522-xincun"
]

stli@str2-7250-2-lc01:~/gcu$ sudo config rollback 20240522-xincun
MultiASICConfigRollbacker: Config rollbacking starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Verifying '20240522-xincun' exists.
MultiASICConfigRollbacker: Loading checkpoint '20240522-xincun' into memory.
MultiASICConfigRollbacker: Replacing config '20240522-xincun' using 'Config Replacer'.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
MultiASICConfigRollbacker: Config rollbacking completed.
Config rolled back successfully.

stli@str2-7250-2-lc01:~/gcu$ sudo config delete-checkpoint 20240522-xincun
MultiASICConfigRollbacker: Deleting checkpoint starting.
MultiASICConfigRollbacker: Checkpoint name: 20240522-xincun.
MultiASICConfigRollbacker: Checking checkpoint: 20240522-xincun exists.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun.
MultiASICConfigRollbacker: Deleting checkpoint: 20240522-xincun completed.
Checkpoint deleted successfully.
stli@str2-7250-2-lc01:~/gcu$ sudo config list-checkpoints 
[]

stli@str2-7250-2-lc01:~/gcu$ sudo config replace 20240522-xincun.cp.json 
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 38147.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: localhost: Patch application starting.
Patch Applier: localhost: Patch: []
Patch Applier: localhost getting current config db.
Patch Applier: localhost: simulating the target full config after applying the patch.
Patch Applier: localhost: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: localhost: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: localhost: sorting patch updates.
Patch Applier: The localhost patch was converted into 0 changes.
Patch Applier: localhost: applying 0 changes in order.
Patch Applier: localhost: verifying patch updates are reflected on ConfigDB.
Patch Applier: localhost patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97546.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic0: Patch application starting.
Patch Applier: asic0: Patch: []
Patch Applier: asic0 getting current config db.
Patch Applier: asic0: simulating the target full config after applying the patch.
Patch Applier: asic0: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic0: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic0: sorting patch updates.
Patch Applier: The asic0 patch was converted into 0 changes.
Patch Applier: asic0: applying 0 changes in order.
Patch Applier: asic0: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic0 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config Replacer: Config replacement starting.
Config Replacer: Target config length: 97713.
Config Replacer: Getting current config db.
Config Replacer: Generating patch between target config and current config db.
Config Replacer: Applying patch using 'Patch Applier'.
Patch Applier: asic1: Patch application starting.
Patch Applier: asic1: Patch: []
Patch Applier: asic1 getting current config db.
Patch Applier: asic1: simulating the target full config after applying the patch.
Patch Applier: asic1: validating all JsonPatch operations are permitted on the specified fields
Patch Applier: asic1: validating target config does not have empty tables,
                               since they do not show up in ConfigDb.
Patch Applier: asic1: sorting patch updates.
Patch Applier: The asic1 patch was converted into 0 changes.
Patch Applier: asic1: applying 0 changes in order.
Patch Applier: asic1: verifying patch updates are reflected on ConfigDB.
Patch Applier: asic1 patch application completed.
Config Replacer: Verifying config replacement is reflected on ConfigDB.
Config Replacer: Config replacement completed.
Config replaced successfully.
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qiluo-msft qiluo-msft qiluo-msft approved these changes

@wen587 wen587 wen587 left review comments

Assignees
No one assigned
Labels
Approved for 202405 Branch Chassis for 202205 Branch Included in 202405 Branch Request for 202405 Branch
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@xincunli-sonic @qiluo-msft @mssonicbld @gechiang @wen587 @bingwang-ms