[Reclaim buffer] Reclaim unused buffers by applying zero buffer profiles

[stephenxs]

Why I did it

Support zero buffer profiles

1. Add buffer profiles and pool definition for zero buffer profiles
2. Support applying zero profiles on INACTIVE PORTS
3. Enable dynamic buffer manager to load zero pools and profiles from a JSON file

Dependency: It depends on sonic-net/sonic-swss#1910 and submodule advancing PR once the former merged.

Signed-off-by: Stephen Sun stephens@nvidia.com

How I did it

1. Add buffer profiles and pool definition for zero buffer profiles

   • If the buffer model is static:
     • Apply normal buffer profiles to admin-up ports
     • Apply zero buffer profiles to admin-down ports
   • If the buffer model is dynamic:
     • Apply normal buffer profiles to all ports
     • buffer manager will take care when a port is shut down

2. Update buffers_config.j2 to support INACTIVE PORTS by extending the existing macros to generate the various buffer objects, including PGs, queues, ingress/egress profile lists

   • Originally, all the macros to generate the above buffer objects took active ports only as an argument
   • Now that buffer items need to be generated on inactive ports as well, an extra argument representing the inactive ports need to be added
   • To be backward compatible, a new series of macros are introduced to take both active and inactive ports as arguments
   • The original version (with active ports only) will be checked first. If it is not defined, then the extended version will be called
   • Only vendors who support zero profiles need to change their buffer templates

3. Enable buffer manager to load zero pools and profiles from a JSON file:

   • The JSON file is provided on a per-platform basis
   • It is copied from platform/<vendor> folder to /usr/share/sonic/temlates folder in compiling time and rendered when the swss container is being created.

4. To make code clean and reduce redundant code, extract common macros from buffer_defaults_t{0,1}.j2 of all SKUs to two common files:

   • One in Mellanox-SN2700-D48C8 for single ingress pool mode
   • The other in ACS-MSN2700 for double ingress pool mode

   Those files of all other SKUs will be symbol link to the above files

5. Update sonic-cfggen test accordingly:

   • Adjust example output file of JSON template for unit test
   • Add unit test in for Mellanox's new buffer templates.

How to verify it

1. Regression test.
2. Unit test in sonic-cfggen

Run regression test and manually test.

Which release branch to backport (provide reason below if selected)

• 201811
• 201911
• 202006
• 202012
• 202106

Description for the changelog

A picture of a cute animal (not mandatory but encouraged)

[stephenxs]

Detailed change:
device/mellanox/<platform>/<SKU>/buffers_defaults_objects.j2 - link to the same file in 2700 folder

device/mellanox/x86_64-mlnx_msn2700-r0/ACS-MSN2700/buffers_defaults_objects.j2

• move most of the macros to generate pools, profiles, profile lists, queues, and priority groups to this file to avoid redundancy
• extend the macros with support of inactive ports for the purpose of applying zero profiles on admin-down ports

device/mellanox/x86_64-mlnx_msn2700-r0/Mellanox-SN2700-D48C8/buffers_defaults_objects.j2

• same as that in ACS-MSN2700 but for single ingress pool mode

device/mellanox///buffers_defaults_t{0,1}.j2

• move the macros to buffers_defaults_objects.j2

files/build_templates/buffers_config.j2

• generate inactive port as well
• move the code to generate active and inactive port list ahead for include buffers_default_t0/1. by doing so the active and inactive port list is accessible from buffers_default_t0/1
• update to logic of calling macros generate_profile_lists, generate_pg_profils, and generate_queue_buffers:
  • if the original macros exist, call them
  • otherwise, call the _with_inactive_ports version.
  • no affections to vendors doesn't support reclaiming buffer

files/build_templates/docker_image_ctl.j2 and files/build_templates/sonic_debian_extension.j2

• make it possible to generate zero_profile.json during compiling time.
• this is the same way we generate asic_table.json

platform/mellanox/zero_profiles.j2

• zero_profiles for mellanox platform

platform/vs/docker-sonic-vs/buffermgrd.sh

• make it possible to run zero profile in vs image

src/sonic-config-engine/tests/*

• files for unit test

[neethajohn]

As discussed, let's try to make this PR into 2 smaller prs

[neethajohn]

@stephenxs, this PR still contains the code from #9133

[stephenxs]

@stephenxs, this PR still contains the code from #9133

Yes for now. Will remove them after that one is merged otherwise there is dependencies

[stephenxs]

It depends on sonic-net/sonic-swss#1910 and submodule advancing PR once the former merged.