[Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923

xumia · 2022-05-25T14:24:09Z

Why I did it

Support to use symbol links in platform folder to reduce the image size.
The current solution is to copy each lazy installation targets (xxx.deb files) to each of the folders in the platform folder. The size will keep growing when more and more packages added in the platform folder. For cisco-8000 as an example, the size will be up to 2G, while most of them are duplicate packages in the platform folder.

How I did it

Create a new folder in platform/common, all the deb packages are copied to the folder, any other folders where use the packages are the symbol links to the common folder.

Why platform.tar?
We have implemented a patch for it, see #10775, but the problem is the the onie use really old unzip version, cannot support the symbol links.
The current solution is similar to the PR 10775, but make the platform folder into a tar package, which can be supported by onie. During the installation, the package.tar will be extracted to the original folder and removed.

How to verify it

Verified Arista installation
Installed by sonic-installer.

+ unzip -oq /tmp/tmp.0QZcH4zcM2/sonic_image.swi -x boot0 dockerfs.tar.gz platform.tar.gz -d /host/image-master-10923.107767-68d3228d0
+ info Extracting platform.tar.gz
+ cut -f1 -d  /proc/uptime
+ printf %04.2f: Extracting platform.tar.gz\n 28172.20
+ mkdir -p /host/image-master-10923.107767-68d3228d0/platform
+ tar xzf - -C /host/image-master-10923.107767-68d3228d0/platform
+ unzip -oqp /tmp/tmp.0QZcH4zcM2/sonic_image.swi platform.tar.gz
+ grep  /host  /proc/mounts

root@str-a7060cx-acs-1:/host/image-master-10923.107767-68d3228d0# ls
allowlist_paths.conf  boot  docker  fs.squashfs  kernel-cmdline  platform  sonic-config  swi-signature  version
root@str-a7060cx-acs-1:/host/image-master-10923.107767-68d3228d0# find platform/
platform/
platform/x86_64-grub
platform/x86_64-grub/grub-pc-bin_2.04-20_amd64.deb
platform/firsttime

Boot from Aboot.

Aboot# boot flash:sonic.swi
202.64: Cleaning flash content /mnt/flash
211.48: Generating boot-config, machine.conf and cmdline
211.59: Installing image under /mnt/flash/image-master-10923.107767-68d3228d0
211.60: Moving swi to a tmpfs
213.50: Extracting swi content
215.34: Extracting platform.tar.gz
215.38: Extracting dockerfs.tar.gz from swi

root@sonic:/host# ls
acl.json     image-master-10923.107767-68d3228d0  machine.conf  warmboot
boot-config  kernel-params-base                   reboot-cause
root@sonic:/host# ls image-master-10923.107767-68d3228d0/
allowlist_paths.conf  docker       kernel-cmdline  rw             version
boot                  fs.squashfs  platform        swi-signature  work
root@sonic:/host# ls image-master-10923.107767-68d3228d0/platform/
x86_64-grub
root@sonic:/host#

Verified Mellanox, installed by onie

Installing SONiC to /tmp/tmp.dm5r6G/image-master-10923.107767-68d3228d0
Archive:  fs.zip
   creating: boot/
  inflating: boot/config-5.10.0-12-2-amd64
  inflating: boot/initrd.img-5.10.0-12-2-amd64
  inflating: boot/vmlinuz-5.10.0-12-2-amd64
  inflating: boot/System.map-5.10.0-12-2-amd64
  inflating: fs.squashfs
Success: Support tarball created: /tmp/onie-support-mlnx_msn2700.tar.bz2
Installing for i386-pc platform.
Installation finished. No error reported.
Switch CPU vendor is: GenuineIntel
Switch CPU cstates are: disabled
EXTRA_CMDLINE_LINUX=sonic_fips=1
Installed SONiC base image SONiC-OS successfully


root@sonic:/host/image-master-10923.107767-68d3228d0# ls platform/
x86_64-grub

Verified on cisco-8000, installed by onie, it worked fine.

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

lguohan · 2022-05-27T08:18:22Z

please add description, why you are doing this?

xumia · 2022-05-27T12:53:04Z

please add description, why you are doing this?

@lguohan , updated the description.

lguohan · 2022-05-30T16:54:36Z

files/build_templates/sonic_debian_extension.j2

-sudo cp {{ deb }} $FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/
+sudo mkdir -p $FILESYSTEM_ROOT/$PLATFORM_DIR/common
+sudo cp {{ deb }} $FILESYSTEM_ROOT/$PLATFORM_DIR/common/
+sudo ln -sf "../common/{{ debfilename }}" "$FILESYSTEM_ROOT/$PLATFORM_DIR/{{dev}}/{{ debfilename }}"


why don't we do the same thing (symbol link approach for deb in /var/cache/apt/archives?

It is to copy the packages to installer folder. The packages will be installed by the script (install.sh) by each of the boot loader. In the platform folder, it contains some sub folders, such as x86_64-8101_32fh_o-r0, x86_64-grub, the symbol links are created in the sub folders.
@lguohan , do you mean to copy the packages into /var/cache/apt/archives? The platform folder is created before the filesystem ready in boot loader (Aboot/Uboot/Grub), maybe the folder can not be used.

lguohan · 2022-05-30T16:55:03Z

i am wondering what is the saving for sonic-broadcom.bin?

xumia · 2022-05-30T23:49:53Z

i am wondering what is the saving for sonic-broadcom.bin?

There is a zip file "fs.zip" in installer folder in the .bin file. In the fs.zip, it contains boot/, platform/, docker.tar.gz, and fs.squashfs.
There are some of the .deb files in platform folders, we change to use symbol links, because of multiple duplicate files, to reduce the image size.
We change the platform/ to platform.tar.gz, since the unzip program in ONIE does not support to extract the symbol links in .zip file (fs.zip). During installing, the platform.tar.gz is changed back to platform/.

xumia · 2022-06-06T01:12:00Z

@lguohan , could you please help review it again?

xumia · 2022-06-06T01:16:28Z

@Staphylo, could you please help review the PR? Thanks.

abdosi · 2022-06-07T01:39:08Z

cc @anamehra

Staphylo · 2022-06-08T09:08:27Z

files/Aboot/boot0.j2

@@ -350,6 +350,12 @@ extract_image() {
    ## Unzip the image except boot0 and dockerfs archive
    unzip -oq "$swipath" -x boot0 "$dockerfs" -d "$image_path"

+    ## Extrat the platform.tar.gz


nit: Extract

Fixed, thanks.

Staphylo · 2022-06-08T09:09:07Z

files/Aboot/boot0.j2

@@ -350,6 +350,12 @@ extract_image() {
    ## Unzip the image except boot0 and dockerfs archive
    unzip -oq "$swipath" -x boot0 "$dockerfs" -d "$image_path"

+    ## Extrat the platform.tar.gz
+    info "Extracting platform.tar.gz"
+    mkdir -p $image_path/platform


add quotes: "$image_path/platform"
same for the lines below

Staphylo · 2022-06-08T09:09:17Z

files/Aboot/boot0.j2

+    ## Extrat the platform.tar.gz
+    info "Extracting platform.tar.gz"
+    mkdir -p $image_path/platform
+    tar xzf $image_path/platform.tar.gz -C $image_path/platform


Even though this works perfectly there is an increase in spacial complexity due to this change.
After this command there is both the tarball and the extracted content on the flash.
It's mostly fine since platform.tar.gz is small but if it's meant to grow bigger I would suggest another approach.

The best option is to add platform.tar.gz to the exclusion list at line 351 and then extract the tarball by streaming it out of the zip file.
We've already done such a thing for dockerfs.

Thanks for your comment, changed to unzip to pipe, and extract in memory.

LGTM, thanks for addressing this

…ic-net#10775)" (sonic-net#10916)" This reverts commit f0dfd39.

xumia · 2022-06-14T06:02:02Z

@lguohan , @Staphylo , do you have more comments? If not, could you please help approve it? Thanks.

Staphylo

LGTM on the platform independent code and Aboot boot0

… reduce the image size (#10923) Why I did it Support to use symbol links in platform folder to reduce the image size. The current solution is to copy each lazy installation targets (xxx.deb files) to each of the folders in the platform folder. The size will keep growing when more and more packages added in the platform folder. For cisco-8000 as an example, the size will be up to 2G, while most of them are duplicate packages in the platform folder. How I did it Create a new folder in platform/common, all the deb packages are copied to the folder, any other folders where use the packages are the symbol links to the common folder. Why platform.tar? We have implemented a patch for it, see #10775, but the problem is the the onie use really old unzip version, cannot support the symbol links. The current solution is similar to the PR 10775, but make the platform folder into a tar package, which can be supported by onie. During the installation, the package.tar will be extracted to the original folder and removed.

… reduce the image size (sonic-net#10923) Why I did it Support to use symbol links in platform folder to reduce the image size. The current solution is to copy each lazy installation targets (xxx.deb files) to each of the folders in the platform folder. The size will keep growing when more and more packages added in the platform folder. For cisco-8000 as an example, the size will be up to 2G, while most of them are duplicate packages in the platform folder. How I did it Create a new folder in platform/common, all the deb packages are copied to the folder, any other folders where use the packages are the symbol links to the common folder. Why platform.tar? We have implemented a patch for it, see sonic-net#10775, but the problem is the the onie use really old unzip version, cannot support the symbol links. The current solution is similar to the PR 10775, but make the platform folder into a tar package, which can be supported by onie. During the installation, the package.tar will be extracted to the original folder and removed.

…ration of lazy packages (#12164) Why I did it The current lazy installer relies on a filename sort for both unpack and configuration steps. When systemd services are configured [started] by multiple packages the order is by filename not by the declared package dependencies. This can cause the start order of services to differ between first-boot and subsequent boots. Declared systemd service dependencies further exacerbate the issue (e.g. blocking the first-boot script). The current installer leaves packages un-configured if the package dependency order does not match the filename order. This also fixes a trivial bug in [Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923 where externally downloaded dependencies are duplicated across lazy package device directories. How I did it Changed the staging and first-boot scripts to use apt-get: dpkg -i /host/image-$SONIC_VERSION/platform/$platform/*.deb becomes apt-get -y install /host/image-$SONIC_VERSION/platform/$platform/*.deb when dependencies are detected during image staging. How to verify it Apt-get critical rules Add a Depends= to the control information of a package. Grep the syslog for rc.local between images and observe the configuration order of packages change.

…202012 Due to confliction in files/image_config/platform/rc.local Related work items: sonic-net#9248, sonic-net#10224, sonic-net#10923, sonic-net#12164, sonic-net#12775, sonic-net#12796, sonic-net#12806, sonic-net#12808

…ration of lazy packages (#12164) Why I did it The current lazy installer relies on a filename sort for both unpack and configuration steps. When systemd services are configured [started] by multiple packages the order is by filename not by the declared package dependencies. This can cause the start order of services to differ between first-boot and subsequent boots. Declared systemd service dependencies further exacerbate the issue (e.g. blocking the first-boot script). The current installer leaves packages un-configured if the package dependency order does not match the filename order. This also fixes a trivial bug in [Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923 where externally downloaded dependencies are duplicated across lazy package device directories. How I did it Changed the staging and first-boot scripts to use apt-get: dpkg -i /host/image-$SONIC_VERSION/platform/$platform/*.deb becomes apt-get -y install /host/image-$SONIC_VERSION/platform/$platform/*.deb when dependencies are detected during image staging. How to verify it Apt-get critical rules Add a Depends= to the control information of a package. Grep the syslog for rc.local between images and observe the configuration order of packages change.

…ed in sonic-net#10923

xumia force-pushed the fix-onie-unzip-symlink-2 branch from 4bfa11a to fcdcc93 Compare May 26, 2022 06:16

xumia changed the title ~~[Build]: Use the platform.tar.gz as a replacement for platform folder~~ [Build]: Support to use symbol links in platform folder to reduce the image size May 27, 2022

xumia changed the title ~~[Build]: Support to use symbol links in platform folder to reduce the image size~~ [Build]: Support to use symbol links for lazy installation targets to reduce the image size May 27, 2022

xumia requested a review from abdosi May 29, 2022 06:50

xumia marked this pull request as ready for review May 29, 2022 06:50

xumia requested review from qiluo-msft and lguohan as code owners May 29, 2022 06:50

lguohan reviewed May 30, 2022

View reviewed changes

xumia requested a review from Staphylo June 6, 2022 01:16

Staphylo reviewed Jun 8, 2022

View reviewed changes

xumia added 3 commits June 8, 2022 10:53

Revert "Revert "Reduce image size for lazy installation packages (son…

d6cc8e6

…ic-net#10775)" (sonic-net#10916)" This reverts commit f0dfd39.

Use the platform.tar.gz as a replacement for platform folder

96c8fba

Reduce the requird disk size to unzip platform.tar.gz

677c5bf

xumia force-pushed the fix-onie-unzip-symlink-2 branch from 758b99a to 677c5bf Compare June 8, 2022 10:54

xumia added 2 commits June 8, 2022 10:57

Fix a type issue

b6039a7

Change to extract tar files in memory

dff8acb

Staphylo approved these changes Jun 17, 2022

View reviewed changes

qiluo-msft approved these changes Jun 21, 2022

View reviewed changes

xumia merged commit fdef1f0 into sonic-net:master Jun 21, 2022

xumia deleted the fix-onie-unzip-symlink-2 branch June 21, 2022 05:03

xumia added Request for 202012 Branch Request for 202106 Branch labels Jun 27, 2022

xumia added Request for 202111 Branch For PRs being requested for 202111 branch Request for 202205 Branch labels Jun 27, 2022

yxieca added the Included in 202205 Branch label Jun 28, 2022

qiluo-msft added the Included in 202012 Branch label Jul 5, 2022

lornelong mentioned this pull request Sep 22, 2022

[Build] Use apt-get to predictably support dependency ordered configuration of lazy packages #12164

Merged

5 tasks

oleksandrivantsiv added a commit to oleksandrivantsiv/sonic-buildimage that referenced this pull request Apr 11, 2023

[blufield] Align installer implementation with infra changes introduc…

4f30596

…ed in sonic-net#10923

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923

[Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923

xumia commented May 25, 2022 •

edited

Loading

lguohan commented May 27, 2022

xumia commented May 27, 2022 •

edited

Loading

lguohan May 30, 2022

xumia May 31, 2022 •

edited

Loading

lguohan commented May 30, 2022

xumia commented May 30, 2022

xumia commented Jun 6, 2022 •

edited

Loading

xumia commented Jun 6, 2022

abdosi commented Jun 7, 2022

Staphylo Jun 8, 2022

xumia Jun 8, 2022

Staphylo Jun 8, 2022

xumia Jun 8, 2022

Staphylo Jun 8, 2022

xumia Jun 8, 2022 •

edited

Loading

Staphylo Jun 9, 2022

xumia commented Jun 14, 2022

Staphylo left a comment

[Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923

[Build]: Support to use symbol links for lazy installation targets to reduce the image size #10923

Conversation

xumia commented May 25, 2022 • edited Loading

Why I did it

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

lguohan commented May 27, 2022

xumia commented May 27, 2022 • edited Loading

lguohan May 30, 2022

Choose a reason for hiding this comment

xumia May 31, 2022 • edited Loading

Choose a reason for hiding this comment

lguohan commented May 30, 2022

xumia commented May 30, 2022

xumia commented Jun 6, 2022 • edited Loading

xumia commented Jun 6, 2022

abdosi commented Jun 7, 2022

Staphylo Jun 8, 2022

Choose a reason for hiding this comment

xumia Jun 8, 2022

Choose a reason for hiding this comment

Staphylo Jun 8, 2022

Choose a reason for hiding this comment

xumia Jun 8, 2022

Choose a reason for hiding this comment

Staphylo Jun 8, 2022

Choose a reason for hiding this comment

xumia Jun 8, 2022 • edited Loading

Choose a reason for hiding this comment

Staphylo Jun 9, 2022

Choose a reason for hiding this comment

xumia commented Jun 14, 2022

Staphylo left a comment

Choose a reason for hiding this comment

xumia commented May 25, 2022 •

edited

Loading

xumia commented May 27, 2022 •

edited

Loading

xumia May 31, 2022 •

edited

Loading

xumia commented Jun 6, 2022 •

edited

Loading

xumia Jun 8, 2022 •

edited

Loading