Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[202012] Fix CONFIG_DB_INITIALIZED flag check logic and set/reset flag for warmboot #16225

Merged
merged 2 commits into from
Aug 24, 2023

Conversation

vaibhavhd
Copy link
Contributor

Cherry pick of #15685

MSFT ADO: 24274591

Why I did it

Two changes:

1 Fix a day1 issue, where check to wait until CONFIG_DB_INITIALIZED is incorrect.

There are multiple places where same incorrect logic is used.

Current logic (until [[ $($SONIC_DB_CLI CONFIG_DB GET "CONFIG_DB_INITIALIZED") ]];) will always result in pass, irrespective of the result of GET operation.

root@str2-7060cx-32s-29:~# sonic-db-cli CONFIG_DB GET "CONFIG_DB_INITIALIZED"
1
root@str2-7060cx-32s-29:~# until [[ $(sonic-db-cli CONFIG_DB GET "CONFIG_DB_INITIALIZED") ]]; do echo "entered here"; done
root@str2-7060cx-32s-29:~# 

root@str2-7060cx-32s-29:~# 
root@str2-7060cx-32s-29:~# sonic-db-cli CONFIG_DB GET "CONFIG_DB_INITIALIZED"                                             
0
root@str2-7060cx-32s-29:~# until [[ $(sonic-db-cli CONFIG_DB GET "CONFIG_DB_INITIALIZED") ]]; do echo "entered here"; done
root@str2-7060cx-32s-29:~# 

Fix this logic by checking for value of flag to be "1".

root@str2-7060cx-32s-29:~# until [[ $(sonic-db-cli CONFIG_DB GET "CONFIG_DB_INITIALIZED") -eq 1 ]]; do echo "entered here"; done
entered here
entered here
entered here

This gap in logic was highlighted when another fix was merged: #14933
The issue being fixed here caused warmboot-finalizer to not wait until config-db is initialized.

2 Set and unset CONFIG_DB_INITIALIZED for warm-reboot case

Currently, during warm shutdown CONFIG_DB_INITIALIZED's value is stored in redis db backup. This is restored back when the dump is loaded during warm-recovery.
So the value of CONFIG_DB_INITIALIZED does not depend on config db's state, however it remain what it was before reboot.

Fix this by setting CONFIG_DB_INITIALIZED to 0 as when the DB is loaded, and set it to 1 after db_migrator is done.

Work item tracking
  • Microsoft ADO (number only):

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006
  • 202012
  • 202106
  • 202111
  • 202205
  • 202211
  • 202305

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

…mboot (sonic-net#15685)

* Fix CONFIG_DB_INITIALIZED flag check logic and set/reset flag for warm-reboot
* Fix db-cli usage
* Handle same image warm-reboot and generalize handling of INIT flag
* Cover boot from ONIE case: set config init flag when minigraph, config_db are missing
* Handle case: first boot of SONiC
* Check for config init flag
* Simplify logic, and do not call db_migrator for same image reboot
@vaibhavhd vaibhavhd changed the title Fix CONFIG_DB_INITIALIZED flag check logic and set/reset flag for warmboot [202012] Fix CONFIG_DB_INITIALIZED flag check logic and set/reset flag for warmboot Aug 21, 2023
@vaibhavhd vaibhavhd requested a review from qiluo-msft August 21, 2023 22:08
@qiluo-msft qiluo-msft requested a review from yxieca August 21, 2023 22:42
@vaibhavhd
Copy link
Contributor Author

@qiluo-msft please take this change to 202012. Thanks

@qiluo-msft qiluo-msft merged commit 908933b into sonic-net:202012 Aug 24, 2023
@vaibhavhd vaibhavhd deleted the 202012 branch August 24, 2023 21:24
qiluo-msft pushed a commit that referenced this pull request Sep 25, 2023
… missing FAST_REBOOT system flag (#16669)

### Why I did it

Fast reboot is failing on 202012 after PR #15685 was cherrypicked to 202012 as part of #16225

The master branch change is good, but the cherry pick to 202012 is bad.
Change was needed on master as the code added here was not effective (as it was unreachable) and not required (as fast-reboot on master uses warm-reboot infra of db dump and reconc).

However, this code was still being used in 202012, and should not have been removed. The DB flag needs to be set to allow services do fast recovery. In the latest 202012 images, fast reboot fails as syncd does cold restart:

Good case on 202012 (before PR 16225)
```
Sep 14 13:25:55.435266 str3-s6100-acs-6 NOTICE syncd#syncd: :- Syncd: command line:  
	EnableDiagShell=YES EnableTempView=YES DisableExitSleep=NO EnableUnittests=NO EnableConsistencyCheck=NO 
	EnableSyncMode=YES RedisCommunicationMode=redis_async EnableSaiBulkSuport=NO 
	StartType=fast               <----------------------
	ProfileMapFile=/etc/sai.d/sai.profile GlobalContext=0 ContextConfig= BreakConfig=/tmp/break_before_make_objects
```

Bad case on 202012 (after PR 16225)
```
Sep 22 22:00:19.619381 str-s6100-acs-2 NOTICE syncd#syncd: :- Syncd: command line:  
	EnableDiagShell=YES EnableTempView=YES DisableExitSleep=NO EnableUnittests=NO EnableConsistencyCheck=NO 
	EnableSyncMode=YES RedisCommunicationMode=redis_async EnableSaiBulkSuport=NO 
	StartType=cold               <----------------------
	ProfileMapFile=/etc/sai.d/sai.profile GlobalContext=0 ContextConfig= BreakConfig=/tmp/break_before_make_objects
```
##### Work item tracking
- Microsoft ADO **(number only)**: 25227065

#### How I did it

Set system flag for fast reboot during boot up path

#### How to verify it

Change restores the state as it was before PR 16225, and fast-reboot worked before 16225

Tested locally w/ the change by replacing database.sh on the device.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants