SONiC to SONiC update

SONiC-to-SONiC update design

The SONiC-to-SONiC update feature enables software update of the running switch without having to reboot it and interrupt the traffic flow.

Overview

SONiC-to-SONiC update in general includes three steps:

1. Download of the new software
2. Execution of the update (while the switch continues to run the old software and process traffic)
3. Reboot to activate the new software

All the switch software is contained in one file called ONE Image. Download could be done in any available way (wget, scp, etc).
Execute the downloaded file to run the update. For example:

sh sonic-image.bin  

The system will not reboot automatically after the update. User can manually reboot it after the update or schedule it for any time later. After the reboot the new software will load by default, while the previous software version will still remain on the switch as a fail over backup. Note: there should be a writable file system available with sufficient free space to keep the new SONiC image and some spare space to unpack the image [TODO: Calculate and specify rough free space expectations].

General approach

• Update is initiated by running SONiC ONE Image on the switch
• Update can be initiated in either switch running SONiC or in the ONIE Rescue console
• Switch runs normally during and after the update. New image will be loaded only after reboot
• Reboot will not be invoked by the update scripts
• Number of recent installations that are kept on the device: 2
• Each installation reside on /dev/sda3 in the separate directory. Directory name is “image-xxx” (where xxx is the build hash or SONiC version+hash – TBD)
• The most recent installation will be marked in GRUB to load by default. The oldest or the not running one (if the total count is 2) will be removed from GRUB and from the disk.

Technical details

Directories and files

SONiC version specific files and directories

The list of directories and files to be placed under the installation specific directory “image-xxx”

• [boot] – kernel, initrd, etc
• [docker] – docker filesystem
• [rw] – r/w part of the root file system
• [platform]
• fs.squashfs – r/o part of the root file system
• onie-support.tar.bz2

Common directories and files

• [disk-img] – directory holding file var-log.ext4 – a loop device for logs
• [grub] – SONiC GRUB submenu
• machine.conf

Mounts

• /image-xxx/fs.squashfs mounted to / (RO)
• /dev/sda3 to /host
• /image-xxx/rw mounted to / (RW)
• /image-xxx/docker to /var/lib/docker
• /image-xxx/boot to /boot
• /disk-img/val-log.ext4 to /var/log

Implementation

Installation process of the ONE Image needs to be changed in order to implement the concept of the SONiC-to-SONiC update.
Most of the installation processes are implemented in the install.sh script. This script is a part of the ONE Image package.

Changes need to be done in the installation script

Installation script is located at sonic-buildimage/installer/x86_64/install.sh
Currently installation always re-formats the /dev/sda3 partition before unpacking its components. SONiC-to-SONiC update design assumes keeping two the most recent SONiC software installations as a backup. To implement this the process of installation should include two modes:

1. Initial installation which can go as it is now (format partition, etc) but SONiC components are unpacked and copied according to the new concept.
2. Incremental installation which does not format the partition, but removes the oldest SONiC version instead and unpacks itself next to the existing and currently running version.

Both of these installation modes will consist of two parts: preparation of the file system and the actual copy, unpacking and enlisting into the GRUB.

Initial installation. Part 1

Actions to be taken:

• format /dev/sda3
• create a basic version of the grub configuration
• create file and initialize the log file system

Incremental installation. Part 1

Actions to be taken:

• determine, whether there is a SONiC installation to be deleted. If yes, delete it. This assumes detecting how many sonic installations there are on the device and which one should be deleted. If there are two installations, determine which one is not running now and delete it both from the disk and from the GRUB menu. Deleting from the disk is simple removing the installation-specific folder called "image-xxx". Deleting from the GRUB assumes finding and deleting the menu item called SONiC-xxx from the /grub/grub.cfg.

Both installations. Part 2

In case of initial installation the "root" will be actually the root folder of the /dev/sda3, while in case of incremental installation the "root" is mounted to /host. In order to have a common code for both types of the installation, this difference should be handled in the install.sh script.
Actions to be taken:

• create a directory image-xxx in the root, where xxx is the hash representing software revision (it was used previously in the GRUB entry).
• copy or unpack components listed in Chapter "Directories and files" into the image directory
• update grub menu located in {root}/grub/grub.cfg. Create a new menu entry pointing to the new SONiC installation and update the default item to point to the new installation.

Changes need to be done to update mounts

Script located at sonic-buildimage/files/initramfs-tools/union-mount.j2 mounts aufs, docker and log file systems on the post boot stage of SONiC load. It needs to be updated to use current installation components which will now be located inside the version-specific directory. The full list of mounts is in the chapter [link| Mounts].
Maybe slave.mk will have to be changed to allow the j2 template processor access the installation revision hash.