…t#6275)

Tgen API 0.0.42 has many problems. We have fixed them in 0.0.70.

Remove the unused codes addressed in sonic-net#5891 on Dx010 platform.

Y* profile is the name pattern for p4 programs that developed for the current platform. The difference between them is features enabled and resource reservation.

For this platform, it is expected to work on any Y profile. but after the latest changes, the first Y profile is always used.

…t-versions info (sonic-net#6289)

- Why I did it
scripts/collect_host_image_version_files.sh fails with below error:

scripts/collect_host_image_version_files.sh target ./fsroot
/usr/sbin/chroot: failed to run command 'post_run_buildinfo': No such file or directory
/bin/cp: cannot stat './fsroot/usr/local/share/buildinfo/post-versions': No such file or directory

- How I did it
Issues is because qemu-arm-static is removed before this step. So, I moved the cleanup step to the end.

Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>

[routeorch] more build error fixes when compiling for armhf (32-bit) (sonic-net#1563)

Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>

…#5979)

Add support to start fdbsyncd when swss docker starts. 
New demon is added to sync MAC from Kernel to DB and vise versa.

Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

Co-authored-by: Andriy Kokhan <andriyx.kokhan@intel.com>

Signed-off-by: Guohan Lu <lguohan@gmail.com>

example:

DOCKER_DATA_ROOT_FOR_MULTIARCH=/data/march/docker make configure \
    PLATFORM=centec-arm64 PLATFORM_ARCH=arm64

Signed-off-by: Guohan Lu <lguohan@gmail.com>

for exmaple, for arm64, the sonic-slave docker image name
is sonic-slave-arm64-$(USER)

for amd64, the docker image is kept as it is

Signed-off-by: Guohan Lu <lguohan@gmail.com>

print out sonic-slave name and tag

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…ython 2 (sonic-net#6162)

**- Why I did it**

As part of migrating SONiC codebase from Python 2 to Python 3

**- How I did it**

- No longer install Python 2 in docker-base-buster or docker-config-engine-buster.
- Install Python 2 and pip2 in the following containers until we can completely eliminate it there:
    - docker-platform-monitor
    - docker-sonic-mgmt-framework
    - docker-sonic-vs
- Pin pip2 version <21 where it is still temporarily needed, as pip version 21 will drop support for Python 2
- Also preform some other cleanup, ensuring that pip3, setuptools and wheel packages are installed in docker-base-buster, and then removing any attempts to re-install them in derived containers

…et#6300)

- add config option ENABLE_DOCKER_BASE_PULL to pull sonic-slave docker from registry
- use REGISTRY_PORT, REGISTRY_SERVER to specify docker registry

Signed-off-by: Guohan Lu <lguohan@gmail.com>

Introduce tunnel manager daemon. Start the process as part of swss container

Submodule update for swss:
9ed3026 - 2020-12-24 : [NAT] ACL Rule with DO_NOT_NAT action is getting failed. (sonic-net#1502) [Akhilesh Samineni]
c39a4b1 - 2020-12-23 : Mux/IPTunnel orchagent changes (sonic-net#1497) [Prince Sunny]
bc8df0e - 2020-12-23 : Add support for headroom pool watermark (sonic-net#1567) [Neetha John]

Signed-off-by: Guohan Lu <lguohan@gmail.com>

unit tests are running for sonic utilitie repo

Signed-off-by: Guohan Lu <lguohan@gmail.com>

… fixes (sonic-net#6282)

submodule sonic-utilities in sonic-buildimage has not been updated for a while and there were many fixes needed to be picked up to ease debug/development in master branch.

Here are the list of changes that this update will bring in:

* de14939 2020-12-27 | Revert "[pytest][qos][config] Added pytests for "config qos reload" commands (sonic-net#1266)" (HEAD -> master, origin/master, origin/HEAD) [Guohan Lu]
* ef3370d 2020-12-24 | [fast-reboot]: Stop sflow during fast-reboot (sonic-net#1296) [jangidrk]
* ff1db4d 2020-12-24 | [fast-reboot]: Update fast-reboot-dump.py to handle invalid parsing of genetlink type hostif object id attribute (sonic-net#1318) [jangidrk]
* d63d11d 2020-12-23 | Added show ip/v6 route summary support for multi-asic platform (sonic-net#1320) [gechiang]
* 6202a81 2020-12-23 | [pytest][qos][config] Added pytests for "config qos reload" commands (sonic-net#1266) [Mahesh Maddikayala]
* f555e9e 2020-12-23 | [CRM]Add support for snat, dnat and ipmc crm resources (sonic-net#1258) [Prabhu Sreenivasan]
* f44e57e 2020-12-22 | Retain fgnhg state db table entry during warm reboot (sonic-net#1315) [weixchen1215]
* 5e18bf5 2020-12-23 | Fix exception handling in python3 (sonic-net#1324) [bingwang-ms]
* b6221f4 2020-12-22 | [config] Add unit tests for 'config interface breakout' command (sonic-net#1223) [Praveen Chaudhary]
* 150642e 2020-12-22 | [dropcounters] Fix clear for non-root users (sonic-net#1253) [Danny Allen]
* becf5b5 2020-12-21 | Kubernetes support commands update (sonic-net#1133) [Renuka Manavalan]
* aad2c38 2020-12-19 | VXLAN config and show utilities (sonic-net#870) [Rajesh Sankaran]
* 9419627 2020-12-17 | [Multi-asic] Enhanced Feature Table configuration for multi-asic platforms (sonic-net#1152) [abdosi]
* 155f6d5 2020-12-17 | [config vlan]: Remove `-t` flag from docker exec command (sonic-net#1317) [Lawrence Lee]
* 281b157 2020-12-17 | [fast-reboot-dump] Fix exception in struct.pack (sonic-net#1309) [bingwang-ms]
* 269c317 2020-12-16 | Mgmt vrf/SNMP agent validations and bug fixes (sonic-net#1289) [Suman Kumar]
* 394b202 2020-12-16 | [Dynamic buffer calc] Support dynamic buffer calculation (sonic-net#973) [Stephen Sun]
* 6dd5d31 2020-12-14 | show tech with platform dump option (sonic-net#1158) [Sujin Kang]
* 6255384 2020-12-13 | [kdump]: Parse sonic_platform kernel command line parameter to read the platform identifier string (sonic-net#1291) [rajendra-dendukuri]
* 0603c0b 2020-12-13 | [pcieutil] Remove 'pcie-' prefix from arguments (sonic-net#1297) [Joe LeVeque]
* 10ffc28 2020-12-13 | Added 'detailed' option for 'show interface counters' command (sonic-net#1299) [Akhilesh Samineni]
* 980ea0d 2020-12-11 | Fix show ip route summary on pizzabox platforms (sonic-net#1302) [gechiang]
* af1bb47 2020-12-10 | [acl_loader] Fix default DENY rule for V6 dataplane ACLs (sonic-net#1281) [Danny Allen]
* 57a0b41 2020-12-09 | Add show and clear commands for headroom pool watermark  (sonic-net#1144) [Neetha John]
* d414970 2020-12-08 | [unit test][CLI][pfcwd] Added pfcwd config tests for single and multi ASIC platform. (sonic-net#1248) [Mahesh Maddikayala]
* 2b4a58c 2020-12-08 | [sflow] Fix traceback seen for show sflow interface (sonic-net#1282) [Garrick He]
* e22980f 2020-12-09 | [config/console][consutil] Support enable/disable console switch (sonic-net#1275) [Blueve]
* 2f263c4 2020-12-08 | [fast-reboot] Fix fast-reboot when NDP entries are present (sonic-net#1295) [shlomibitton]
* 326e534 2020-12-07 | Fast-reboot: add a new flag to ignore ASIC config checksum verification failures (sonic-net#1292) [Vaibhav Hemant Dixit]
* 042254e 2020-12-04 | Kdump improvements (sonic-net#1284) [rajendra-dendukuri]

**- Why I did it**

python2 is end of life and SONiC is going to support python3. This PR is going to support:

1. Build pmon daemons with python3
2. Install and run python3 version pmon daemons

**- How I did it**

1. Change pmon daemons make files to build bothe python2 and python3 whl
2. Change docker-platform-monitor make files to install both python2 and python3 whl
3. Change pmon docker startup files to start pmon daemons according to the supported platform API version

…#6291)

swss/teamd/syncd services were changed to always enabled
in commit fad481e as a workaround
for not letting hostcfgd start service during the bootup process.

commit 317a4b3 introduce
wait till full system bootup before updating feature states in hostcfgd.

Thus, workaround introduced in commit fad481e can be removed

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…e prefix

frr does not advertise route if local route is not reachable, as a result
loopback route /64 is not advertised to the neighbors. Add static route
allows frr to advertise the route to its peers

Signed-off-by: Guohan Lu <lguohan@gmail.com>

* 321291a 2020-12-26 | [loopback]: skip route pointing to loopback interface (sonic-net#1570) (HEAD, origin/master, origin/HEAD) [lguohan]
* 41fd2c4 2020-12-26 | [CRM] Safety check for division by 0 (sonic-net#1569) [Prince Sunny]

Signed-off-by: Guohan Lu <lguohan@gmail.com>

* 7924bd1 2020-12-28 | [route_check]: ignore routes pointing to Loopback interface (sonic-net#1337) (HEAD, origin/master, origin/HEAD) [lguohan]

Signed-off-by: Guohan Lu <lguohan@gmail.com>

This PR is in preparation to move from snmp.yml to configdb. This will more closely align with other commands in sonic and use configdb as the source of truth for snmp configuration.
Note: This is the first of 2 PR's to enable this. This PR will not change any functionality but will allow the snmp.yml file info to be put into the configdb.
Created a script that takes the snmp.yml variables and converts them to the configdb format.
Added file to dockerfile.j2 so that file is copied in the container.
Updated start.sh file to automatically run the python conversion script each time the docker container is restarted.

…orms (sonic-net#6304)

Mellanox already supports multiple destination IPs in IPinIP tunnel configuration, thus removing mellanox
exception for IPinIP configuration.

- How I did it

Removed "dst_ip" field generation in mellanox platform condition.
Sorted the "dst_ip" list, so that it is easier to test against sample configuration in unit tests.
Aligned unit test sample.

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>

instruct azure pipeline to use node.js in the docker.

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops#bring-your-own-nodejs

Signed-off-by: Guohan Lu <lguohan@gmail.com>

Build libyang1 which is required for frr 7.5

…et#6301)

- No longer install Python 2 packages in host:
    - libpython2.7-dev
    - docker
    - ipaddress
    - netifaces
    - azure-storage
    - watchdog
    - futures

- Install Python 3 versions of the following packages in host:
    - docker
    - azure-storage
    - watchdog
    - redis
    - swsssdk (install unconditionally)

- Convert system-health scripts to Python 3
- Build and install system-health as a Python 3 wheel
- Also convert newlines from DOS to UNIX

use bash -t test flag to check if input device is tty or not

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…onic-net#6319)

sonic-slave tag only allows all lower case. In case the user
name is mixed case, we need to change user name to all lower case.

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…rk (sonic-net#6315)

Signed-off-by: Nazarii Hnydyn <nazariig@nvidia.com>

Depending on the performance characteristics of a given hardware platform, it's possible to exceed the default 120 second kernel timeout during I/O intensive operations like image installation. This can cause a kernel panic like so:

kernel:[ 852.441781] Kernel panic - not syncing: hung_task: blocked tasks

If this happens during image installation, it's possible for the install to become corrupted and leave the device in an unreachable state that requires a power cycle to resolve. This risk increases as image size continues to increase. So, we need to increase the timeout so that we don't encounter kernel panics on devices with lower disk throughput.

Signed-off-by: Danny Allen <daall@microsoft.com>

- Why I did it
Make EEPROM platform APIs Python3 compliant in Nokia platform.

- How I did it
Handle bytearray type returned by read_eeprom and read_eeprom_bytes methods.

- How to verify it
Boot Nokia ixs7215 and verify PMON docker running and show platform syseeprom

Co-authored-by: Carl Keene <keene@nokia.com>

 Why I did it
To support FG_ECMP  scenarios
- How I did it
Modified minigraph parser to parse ECMP fields in the case they are present in minigraph
- How to verify it
Loaded ensuing config_db file on a DUT to verify the fields are parsed and configure device correctly

…or hardware_reboot_cause (sonic-net#6325)

What: Modify unit test to not call any platform dependent api in test_find_hardware_reboot_cause.

- Why I did it
MELLANOX build is failing for the recent PRs. The errors are due to platform library being invoked in a unit test for determine-reboot-cause script.

Verified by running unit tests and a successful Mellanox build.

Co-authored-by: Vaibhav Hemant Dixit <vadixit@microsoft.com>

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…sonic-net#6298)

Enable the notify mode of rsyslogd imfile module used for supervisord logs in docker container

Including commits in sonic-swss-common repo:

b423b9c Add support for hexists call (sonic-net#432) [Tony Titus]
0982996 Remove extension of tableNameSeparatorMap (sonic-net#430) [Qi Luo]
d16cc76 [build]: add azure pipeline build badge (sonic-net#429) [lguohan]
f2aaf55 Set up CI with Azure Pipelines (sonic-net#428) [lguohan]

…net#6333)

Features:
    Spectrum-3 | Systems | Added GA-level support for SN4700 A0 system
    All | Shared headroom | Added GA-level support for Shared headroom between PGs

  Bugs fixes:
    All | Counters | Sent traffic in certain size is wrongly increase to a smaller size counter, because port extended counter has a counter for sent traffic per packet-size range
    All | Shared buffer | Configuring shared buffer on the fly may, on occasion, cause the chip to get stuck
    Spectrum-2 | Modules | On occasion, link down is experienced with INPHI COLORZ PAM4 100G optic cables on SN3700 systems

Accton util applies lsmod to check if drivers are installed.
But lsmod may return error on startup and skip module installation.

Signed-off-by: Brandon Chuang <brandon_chuang@edge-core.com>

Signed-off-by: Guohan Lu <lguohan@gmail.com>

fix a bug when there are multiple debian packages to be uninstalled

Signed-off-by: Guohan Lu <lguohan@gmail.com>

in case conflicting packages have same name but different version,
the install step needs to wait till package with the conflicting
version is uninstalled

Signed-off-by: Guohan Lu <lguohan@gmail.com>

no instalation of frr during the build process

Signed-off-by: Guohan Lu <lguohan@gmail.com>

frr build requires libyang 1.0.184 which conflicts with
libyang 1.0.73. Solution here is to compile frr and libyang 1.0.184
first, and then uninstall libyang 1.0.184 after frr build.
Then, compile libyang 1.0.73 and all packages depend on it later.

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…ion (sonic-net#6309)

* [sonic-py-common] add an API to get file path containing SONiC version so that the API can be mocked for unit tests.

…on (sonic-net#6339)

the control file version is 4.2.1.5-7

sonic$ sudo dpkg -i target/debs/buster/libsaibcm-dev_4.2.1.5-8_amd64.deb
(Reading database ... 175880 files and directories currently installed.)
Preparing to unpack .../libsaibcm-dev_4.2.1.5-8_amd64.deb ...
Unpacking libsaibcm-dev (4.2.1.5-7) over (4.2.1.5-7) ...
Setting up libsaibcm-dev (4.2.1.5-7) ...
lgh@491d842369cf:/sonic$ sudo dpkg -i target/debs/buster/libsaibcm_4.2.1.5-8_amd64.deb
(Reading database ... 175880 files and directories currently installed.)
Preparing to unpack .../libsaibcm_4.2.1.5-8_amd64.deb ...
Unpacking libsaibcm (4.2.1.5-7) over (4.2.1.5-7) ...
Setting up libsaibcm (4.2.1.5-7) ...
Processing triggers for libc-bin (2.28-10) ...

Signed-off-by: Guohan Lu <lguohan@gmail.com>

… image (sonic-net#6340)

build frr dbg image force to install frr in the build process
which breaks the current build and is uneccessary.

Signed-off-by: Guohan Lu <lguohan@gmail.com>

* Add changes to allow migration of bfn syncd to buster

* Update BFN packages for Debian 10

Signed-off-by: Denys Petryshyn <denysx.petryshyn@intel.com>

In the emulated armhf environment, the function readdir()returns NULL on a ext4 file system directory. When running the libyang1 test cases, it will require to load the plugins from the files (such as metadata.so), because the readdir() is failing, the plugins can’t be loaded in the emulated armhf environment, so it causes libyang1 test error. This error is a combination of the following reasons.
• Emulation of a 32-bit target from a 64-bit host –> qemu from x86_64 to armhf
• Glibc version > 2.27 – Debian buster is using glibc 2.28

- How I did it
Enabled large file support by setting _FILE_OFFSET_BITS=64 for libyang1.

Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>

Fix syntax errors during xcvrd start with Python 3 daemons

Signed-off-by: Jing Kan jika@microsoft.com

…ause (sonic-net#6322)

Add a systemd dependency to make platform-modules service as a prerequisite for determine-reboot-cause service to ensure platform initialization is complete before determine-reboot-cause.service executes.

…net#6350)

combine multiple same operation into one operation to reduce
the build steps. this is to avoid max depth exceeded issue
in the build.

Signed-off-by: Guohan Lu <lguohan@gmail.com>

The details are as follows:

    1. 0010-Ported-fix-for-length-exceeded-from-lldp-community.patch

    Patch taken from lldpd/lldpd@7824347

    lib: remove limit on system description length

    The limit was introduced in 9c49ced while fixing a memory leak.
    The state data is used to ensure we don't interleave operations. We
    need to handle the case where the value is truncated because it is
    larger than the allocated size.

    Fix issue lldpd/lldpd#408

    2. 0011-fix-med-location-len.patch
    
    Patch taken from lldpd/lldpd@5c34794

    lib: fix LLDP-MED location parsing in liblldpctl

    Some bounds were not checked correctly when parsing LLDP-MED civic
    location fields. This triggers out-of-bound reads (no write) in
    lldpcli, ultimately leading to a crash.

    Fix lldpd/lldpd#420

Signed-off-by: sudhanshukumar22 <sudhanshu.kumar@broadcom.com>

sonic-net#6348)

…nic-net#5911)

Fix the After first bootup(onie-install), the FEATURE table is not present in CONFIG_DB. 
Fix is done by calling config reload.

…M_API_PY3 (sonic-net#6344)

Remove the build time dependency on SONIC_PLATFORM_API_PY2 and SONIC_PLATFORM_API_PY3 from xcvrd make rule

…6349)

- Why I did it - Reboot cause prints "Non-Hardware (N/A)" instead of showing the software reboot cause.
The issue is mishandling of hardware reboot cause in determine-reboot-cause script.

- How I did it
Fixed the handling for Non Hardware reboot cause. Ignore if Non-Hardware is present in the hardware_reboot_cause output. Added some code refactoring for simplicity.

- How to verify it - With fix, the hardware reboot cause is ignored (if it is non hw):

…#6134)

**- Why I did it**
I'm updating the jinja2 template to support getting SNMP information from the redis configdb. 
I'm using the format approved here: 
sonic-net/SONiC#718

This will pave the way for us to decrement using the snmp.yml in the future.  
Right now we will still be using both the snmp.yml and configdb to get variable information in order to create the snmpd.conf via the sonic-cfggen tool. 

**- How I did it**
I first updated the SNMP Schema in PR sonic-net#718 to get that approved as a standardized format. 
Then I verified I could add snmp configs to the configdb using this standard schema.  Once the configs were added to the configdb then I updated the snmpd.conf.j2 file to support the updates via the configdb while still using the variables in the snmp.yml file in parallel.  This way we will have backward compatibility until we can fully migrate to the configdb only. 

By updating the snmpd.conf.j2 template and running the sonic-cfggen tool the snmpd.conf gets generated with using the values in both the configdb and snmp.yml file. 

Co-authored-by: trvanduy <trvanduy@microsoft.com>

…onic-net#6351)

Enable the notify mode of rsyslogd imfile module used for supervisord
logs in docker container. 

Setup the mode="inotify" when loading imfile, made sure we are are getting 
supervisord logs in host immediately.

Signed-off-by: Abhishek Dosi <abdosi@microsoft.com>

To incldue following changes:

- [ci]: add build for arm64 and armhf (sonic-net#757)
- Use template hgetall, because we will tune the return types of library functions (sonic-net#759)
- [syncd] Fix bulk multi attrs for same key db update (sonic-net#761)

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

- Dynamically change EEPROM driver based on media type.
- Otherwise, EEPROM INFO and DOM INFO might not be fetched properly and will result in erroneous output.

…ebian (sonic-net#6114)

Sonic devices advertise meaningful system description along with Debian package information.

before the fix:

-------------
admin@sonic:~$ show lldp neighbors
-------------------------------------------------------------------------------
LLDP neighbors:
-------------------------------------------------------------------------------
Interface: Ethernet0, via: LLDP, RID: 3, Time: 0 day, 16:36:30
SysName: sonic
SysDescr: Debian GNU/Linux 9 (stretch) Linux 4.9.0-11-2-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64
-------------------------------------------------------------------------------

After the fix:

root@sonic:~# show lldp neighbors Ethernet16
-------------------------------------------------------------------------------
LLDP neighbors:
-------------------------------------------------------------------------------
Interface:    Ethernet16, via: LLDP, RID: 10, Time: 0 day, 00:01:00
    SysName:      sonic
    SysDescr:     SONiC Software Version: SONiC.sonic_upstream_1.0_daily_201130_1501_62-dirty-20201130.203529 - HwSku: Accton-AS7816-64X - Distribution: Debian 10.6 - Kernel: 4.19.0-9-2-amd64
-------------------------------------------------------------------------------

Signed-off-by: sudhanshukumar22 <sudhanshu.kumar@broadcom.com>

…nic-net#6352)

src/sonic-platform-common 9935fca...8664efc (2):

Make sonic_sfp Python2 and Python3 compatible (sonic-net#157)
[sffbase.py] Fix to make Python 3-compatible (sonic-net#156)

src/sonic-platform-daemons e6c786b...81318f7 (1):

[psud] Fix issue where PSU Fan info is not updated in State DB (sonic-net#137)

Fixes sonic-net#6341

…onic-net#6364)

* [platform][Nokia]: Add buffers and qos files for config qos reload

   - providing required files

* [platform][armhf]: remove hardcoded disable for Telemetry on armhf

Co-authored-by: Carl Keene <keene@nokia.com>

Updated imfile configuration for supervisord logs for stretch and buster.

…xpected queue causing _brcm_sai_switch_assert () after warm reboot (sonic-net#6374)

Starting from build (master) 176 the warm reboot on BRCM Platform started to experience syncd crash. Upon further debug by Ying it was determined that the crash was related to the following new change:
[Dynamic buffer calc] Support dynamic buffer calculation (sonic-net#1338)

Ying also debugged further and found The crash was caused by buffer pool profile setting operation SAI_BUFFER_PROFILE_ATTR_SHARED_DYNAMIC_TH

A case has filed with BRCM while a potential fix was tried by Ying that seems to have addressed this issue and we are making this change available in master branch so that it will allow further feature validation/testing especially in the warm reboot area.
Once an official fix is provided by BRCM, we will then remove this in house fix and apply the official fix.

- How to verify it
Just perform warm reboot with any master code 175 or above you should see this issue or issue the following cmd will also cause the crash: "mmuconfig -p egress_lossy_profile -a 0"

Pass universal_newlines=True parameter to subprocess.Popen(); no longer use .encode('utf-8') on resulting stdout.
This was missed in sonic-net#5886

Note: I would prefer to use text=True instead of universal_newlines=True, as the former is an alias only available in Python 3 and is more understandable than the latter. However, Even though the setup.py file for this package only specifies Python 3, the LGTM tool finds other Python 2 code in the repo and validates the code as Python 2 code and alerts that text=True is an invalid parameter. Will stick with universal_newlines=True for now. Once all Python code in the repo has been converted to Python 3, I will change all universal_newlines=True to text=True.

…t#6373)

Signed-off-by: Danny Allen <daall@microsoft.com>

- Make PDDF code compliant with both Python 2 and Python 3
- Align code with PEP8 standards using autopep8
- Build and install both Python 2 and Python 3 PDDF packages

…ade (sonic-net#6285)

In scenario where upgrade gets config from minigraph, it could miss tacacs credentials as they are not in minigraph. Hence restore explicitly upon load-minigraph, if present.

- Why I did it
Upon boot, when config migration is required, the switch could load config from minigraph. The config-load from minigraph would wipe off TACACS key and disable login via TACACS, which would disable all remote user access. This change, would re-configure the TACACS if there is a saved copy available.

- How I did it
When config is loaded from minigraph, look for a TACACS credentials back up (tacacs.json) under /etc/sonic/old_config. If present, load the credentials into running config, before config-save is called.

- How to verify it
Remove /etc/sonic/config_db.json and do an image update. Upon reboot, w/o this change, you would not be able ssh in as remote user. You may login as admin and check out, "show tacacs" & "show aaa" to verify that tacacs-key is missing and login is not enabled for tacacs.
With this change applied, remove /etc/sonic/config_db.json, but save tacacs & aaa credentials as tacacs.json in /etc/sonic/. Upon reboot, you should see remote user access possible.

[skip ci]

**- Why I did it**
This PR aims to monitor the running status of each container. Currently the auto-restart feature was enabled. If a critical process exited unexpected, the container will be restarted. If the container was restarted 3 times during 20 minutes, then it will not run anymore unless we cleared the flag using the command `sudo systemctl reset-failed <container_name>` manually. 

**- How I did it**
We will employ Monit to monitor a script. This script will generate the expected running container list and compare it with the current running containers. If there are containers which were expected to run but were not running, then an alerting message will be written into syslog.

**- How to verify it**
I tested this feature on a lab device `str-a7050-acs-3` which has single ASIC and `str2-n3164-acs-3` which has a Multi-ASIC. First I manually stopped a container by running the command `sudo systemctl stop <container_name>`, then I checked whether there was an alerting message in the syslog.

Signed-off-by: Yong Zhao <yozhao@microsoft.com>

…et#6370)

* Use 20 and 30 route-map entries instead of 2 and 3 for TSA

* Added support for dynamic "Allow list" default action.

Co-authored-by: Pavel Shirshov <pavel.contrib@gmail.com>

Includes sonic-swss-common commits:
```
71dc350 2021-01-07 | Lower the log level for outdated key for SubscriberStateTable notification (sonic-net#441) [Qi Luo]
7e40582 2021-01-08 | Add boost dependencies (sonic-net#442) [Ze Gan]
30a8ddf 2021-01-05 | Change DBConnector::hgetall return type from map to unordered_map (sonic-net#440) [Qi Luo]
021108d 2021-01-02 | MCLAG Enhancements per HLD sonic-net/SONiC#596 (sonic-net#405) [Praveen-Brcm]
54996fc 2021-01-02 | Implement ConfigDBConnector and ConfigDBPipeConnector in C++ (sonic-net#437) [Qi Luo]
8286525 2020-12-27 | Simply refactor DBConnector hgetall() [Qi Luo]
6d1d33b 2020-12-27 | Fix RedisTransactioner: handle empty deque [Qi Luo]
624e0b8 2020-12-26 | Move complex class constructor as explicit, and fix several mistaken copy constructor usage [Qi Luo]
3b983f9 2020-12-30 | [ci]: add timeout to 180 minutes for arm build (sonic-net#439) [lguohan]
f2e4210 2020-12-29 | Add utility for string and redis (sonic-net#434) [Ze Gan]
7a885fd 2020-12-29 | [build]: add build check for arm64 and armhf (sonic-net#436) [lguohan]
47bccc4 2020-12-24 | Add missed vector header to rediscommand.h (sonic-net#435) [Ze Gan]
```

…6385)

Bumps [lxml](https://github.com/lxml/lxml) from 4.6.1 to 4.6.2.
- [Release notes](https://github.com/lxml/lxml/releases)
- [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt)
- [Commits](lxml/lxml@lxml-4.6.1...lxml-4.6.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…try on ARM64 (sonic-net#6386)

* Enable telemetry for ARM64 by default

* [Centec]Upgrade Centec syncd docker to buster; libjemalloc2 have been installed in docker-base-buster, remove libjemalloc1 from docker-syncd-centec's Dockerfile.j2

Co-authored-by: Gu Xianghong <xgu@centecnetworks.com>

 - Cleanup and Refactor of library internals, logic mostly unchanged.
 - Enhance debugability with `arista dump` and `arista diag` commands.
 - Fix power supply detection issue.

…-net#5700)

The HLD about MACsec feature is at :

https://github.com/Azure/SONiC/blob/master/doc/macsec/MACsec_hld.md

- How to verify it
This PR doesn't set MACsec container automatically start, You should manually start the container by docker run docker-macsec
wpa_supplicant binary can be found at MACsec container.
This PR depends on the PR, WPA_SUPPLICANT, and The MACsec container will be set as automatically start by later PR.

Signed-off-by: zegan <zegan@microsoft.com>

- (HEAD, github/master) [storyteller] adding a grep wrapper with predefined scenarios (sonic-net#1349)
- Adding global-timeout, individual command timeout, log files collection (sonic-net#1249)
- Add FW dump with new SAI implementation (sonic-net#1338)
- [unit test][pfcwd] Fix tests that require sudo access (sonic-net#1340)

Signed-off-by: Ying Xie <ying.xie@microsoft.com>

…t#6405)

To make the peer switch hostname easily accessible from config DB. Add peer_switch field to DEVICE_METADATA table

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>

Need to add the follow redirection option -L when downloading the package with redirection.

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…net#6408)

Fix sonic-net#119

when parallel build is enable, multiple dpkg-buildpackage
instances are running at the same time. /var/lib/dpkg is shared
by all instances and the /var/lib/dpkg/updates could be corrupted
and cause the build failure.

the fix is to use overlay fs to mount separate /var/lib/dpkg
for each dpkg-buildpackage instance so that they are not affecting
each other.

Signed-off-by: Guohan Lu <lguohan@gmail.com>

…t#6404)

…onic-net#6418)

Server IPv4 loopbacks do not always arrive with /32 prefix, which is a requirement for the MUX_CABLE table in config DB

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>

…debian control dependency (sonic-net#6420)

Remove executable bit from the service files

… before assignment” error (sonic-net#6419)

In rare case can see that xcvrd failed due to "UnboundLocalError: local variable 'label_port' referenced before assignment"

Init "label_port" as None at the beginning of the function, to avoid the case that "label_port" not assigned.

…-saiserver-centec and platform-modules (sonic-net#6423)

Centec syncd have beend upgraded to buster, docker-syncd-centec-rpc do not need generate stretch based docker.

Co-authored-by: Xianghong Gu <xgu@centecnetworks.com>