Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[config] parse optional fields from hwsku #7

Open
wants to merge 143 commits into
base: master
Choose a base branch
from

Conversation

dmytroxshevchuk
Copy link
Owner

@dmytroxshevchuk dmytroxshevchuk commented Dec 8, 2020

- Why I did it
For now hwsku.json and platform.json dont support optional fields. For example no way to add fec or autoneg field using platform.json and hwsku.json.

- How I did it
Added parsing of optional fields from hwsku.json.

- How to verify it
Add optional field to hwsku.json. After first boot will be generated new config_db.json or you can generate it using sonic-cfggen command. In this file must be optional field from hwsku.json or check using command redis-cli hgetall PORT_TABLE:Ethernet0
Example of hwsku.json, that must be parsed:

{
    "interfaces": {
        "Ethernet0": {
            "default_brkout_mode": "1x100G[40G]",
            "fec": "rs",
            "autoneg": "0"
        },
...
}

Example of generated config_db.json:

    "PORT": {
        "Ethernet0": {
            "alias": "Ethernet0",
            "lanes": "0,1,2,3",
            "speed": "100000",
            "index": "1",
            "admin_status": "up",
            "fec": "rs",
            "autoneg": "0",
            "mtu": "9100"
        },

So, we can see this entries in redis db:

admin@sonic:~$ redis-cli hgetall PORT_TABLE:Ethernet0

 1) "alias"
 2) "Ethernet0"
 3) "lanes"
 4) "0,1,2,3"
 5) "speed"
 6) "100000"
 7) "index"
 8) "1"
 9) "admin_status"
10) "up"
11) "fec"
12) "rs"
13) "autoneg"
14) "0"
15) "mtu"
16) "9100"
17) "description"
18) ""
19) "oper_status"
20) "up"

Also its way to fix show interface status, FEC field but also need add FEC field to hwsku.json.
Before:

admin@sonic:~$ show interfaces status
  Interface            Lanes    Speed    MTU    FEC        Alias    Vlan    Oper    Admin             Type    Asym PFC
-----------  ---------------  -------  -----  -----  -----------  ------  ------  -------  ---------------  ----------
  Ethernet0          0,1,2,3     100G   9100     N/A    Ethernet0  routed      up       up  QSFP28 or later         N/A

After:

admin@sonic:~$ show interfaces status
  Interface            Lanes    Speed    MTU    FEC        Alias    Vlan    Oper    Admin             Type    Asym PFC
-----------  ---------------  -------  -----  -----  -----------  ------  ------  -------  ---------------  ----------
  Ethernet0          0,1,2,3     100G   9100     rs    Ethernet0  routed      up       up  QSFP28 or later         N/A

- Which release branch to backport (provide reason below if selected)

  • 201811
  • 201911
  • 202006

- Description for the changelog

- A picture of a cute animal (not mandatory but encouraged)

dmytroxshevchuk pushed a commit that referenced this pull request Dec 10, 2020
This update brings in the following commits.

86c1108 Enable arm architecture to build in addition to amd64 (sonic-net#37)
4acb2c3 fix bugs and enhance Transformer (sonic-net#35)
49e5a22 ygot related enhancements and fixes (sonic-net#34)
51224de Fix ietf yang search path for cvl schema builds (sonic-net#32)
3c6cdb3 CVL Changes #8: 'must' and 'when' expression evaluation (sonic-net#31)
dabf231 CVL Changes #7: 'leafref' evaluation (sonic-net#28)
6f9535f CVL Changes #6: Customized Xpath Engine integration (sonic-net#27)
5e2466b DB-Layer fixes/enhancements (sonic-net#26)
9a27302 CVL Changes #4: Implementation of new CVL APIs (sonic-net#22)
dbf1093 Translib support for authorization, yang versioning and Delete flag (sonic-net#21)
80f369e CVL Changes #5: YParser enhancement (sonic-net#23)
904ce18 CVL Changes #3: Multi-db instance support (sonic-net#20)
9d24a34 CVL Changes #2:  YValidator infra changes for evaluating xpath expression (sonic-net#19)
f3fc40f CVL Changes #1: Initial CVL code reorganization and common infra changes (sonic-net#18)
4922601 Bulk and RPC API support in translib (sonic-net#16)
1d730df RFC7895 yang module library implementation (#15)
baiwei0427 and others added 28 commits December 24, 2020 01:53
…t#6275)

Tgen API 0.0.42 has many problems. We have fixed them in 0.0.70.
Remove the unused codes addressed in sonic-net#5891 on Dx010 platform.
Y* profile is the name pattern for p4 programs that developed for the current platform. The difference between them is features enabled and resource reservation.

For this platform, it is expected to work on any Y profile. but after the latest changes, the first Y profile is always used.
…t-versions info (sonic-net#6289)

- Why I did it
scripts/collect_host_image_version_files.sh fails with below error:

scripts/collect_host_image_version_files.sh target ./fsroot
/usr/sbin/chroot: failed to run command 'post_run_buildinfo': No such file or directory
/bin/cp: cannot stat './fsroot/usr/local/share/buildinfo/post-versions': No such file or directory

- How I did it
Issues is because qemu-arm-static is removed before this step. So, I moved the cleanup step to the end.

Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
[routeorch] more build error fixes when compiling for armhf (32-bit) (sonic-net#1563)

Signed-off-by: Sabareesh Kumar Anandan <sanandan@marvell.com>
…#5979)

Add support to start fdbsyncd when swss docker starts. 
New demon is added to sync MAC from Kernel to DB and vise versa.
Signed-off-by: Andriy Kokhan <andriyx.kokhan@intel.com>

Co-authored-by: Andriy Kokhan <andriyx.kokhan@intel.com>
example:

DOCKER_DATA_ROOT_FOR_MULTIARCH=/data/march/docker make configure \
    PLATFORM=centec-arm64 PLATFORM_ARCH=arm64

Signed-off-by: Guohan Lu <lguohan@gmail.com>
for exmaple, for arm64, the sonic-slave docker image name
is sonic-slave-arm64-$(USER)

for amd64, the docker image is kept as it is

Signed-off-by: Guohan Lu <lguohan@gmail.com>
print out sonic-slave name and tag

Signed-off-by: Guohan Lu <lguohan@gmail.com>
…ython 2 (sonic-net#6162)

**- Why I did it**

As part of migrating SONiC codebase from Python 2 to Python 3

**- How I did it**

- No longer install Python 2 in docker-base-buster or docker-config-engine-buster.
- Install Python 2 and pip2 in the following containers until we can completely eliminate it there:
    - docker-platform-monitor
    - docker-sonic-mgmt-framework
    - docker-sonic-vs
- Pin pip2 version <21 where it is still temporarily needed, as pip version 21 will drop support for Python 2
- Also preform some other cleanup, ensuring that pip3, setuptools and wheel packages are installed in docker-base-buster, and then removing any attempts to re-install them in derived containers
…et#6300)

- add config option ENABLE_DOCKER_BASE_PULL to pull sonic-slave docker from registry
- use REGISTRY_PORT, REGISTRY_SERVER to specify docker registry

Signed-off-by: Guohan Lu <lguohan@gmail.com>
Introduce tunnel manager daemon. Start the process as part of swss container

Submodule update for swss:
9ed3026 - 2020-12-24 : [NAT] ACL Rule with DO_NOT_NAT action is getting failed. (sonic-net#1502) [Akhilesh Samineni]
c39a4b1 - 2020-12-23 : Mux/IPTunnel orchagent changes (sonic-net#1497) [Prince Sunny]
bc8df0e - 2020-12-23 : Add support for headroom pool watermark (sonic-net#1567) [Neetha John]
Signed-off-by: Guohan Lu <lguohan@gmail.com>
unit tests are running for sonic utilitie repo

Signed-off-by: Guohan Lu <lguohan@gmail.com>
… fixes (sonic-net#6282)

submodule sonic-utilities in sonic-buildimage has not been updated for a while and there were many fixes needed to be picked up to ease debug/development in master branch.

Here are the list of changes that this update will bring in:

* de14939 2020-12-27 | Revert "[pytest][qos][config] Added pytests for "config qos reload" commands (sonic-net#1266)" (HEAD -> master, origin/master, origin/HEAD) [Guohan Lu]
* ef3370d 2020-12-24 | [fast-reboot]: Stop sflow during fast-reboot (sonic-net#1296) [jangidrk]
* ff1db4d 2020-12-24 | [fast-reboot]: Update fast-reboot-dump.py to handle invalid parsing of genetlink type hostif object id attribute (sonic-net#1318) [jangidrk]
* d63d11d 2020-12-23 | Added show ip/v6 route summary support for multi-asic platform (sonic-net#1320) [gechiang]
* 6202a81 2020-12-23 | [pytest][qos][config] Added pytests for "config qos reload" commands (sonic-net#1266) [Mahesh Maddikayala]
* f555e9e 2020-12-23 | [CRM]Add support for snat, dnat and ipmc crm resources (sonic-net#1258) [Prabhu Sreenivasan]
* f44e57e 2020-12-22 | Retain fgnhg state db table entry during warm reboot (sonic-net#1315) [weixchen1215]
* 5e18bf5 2020-12-23 | Fix exception handling in python3 (sonic-net#1324) [bingwang-ms]
* b6221f4 2020-12-22 | [config] Add unit tests for 'config interface breakout' command (sonic-net#1223) [Praveen Chaudhary]
* 150642e 2020-12-22 | [dropcounters] Fix clear for non-root users (sonic-net#1253) [Danny Allen]
* becf5b5 2020-12-21 | Kubernetes support commands update (sonic-net#1133) [Renuka Manavalan]
* aad2c38 2020-12-19 | VXLAN config and show utilities (sonic-net#870) [Rajesh Sankaran]
* 9419627 2020-12-17 | [Multi-asic] Enhanced Feature Table configuration for multi-asic platforms (sonic-net#1152) [abdosi]
* 155f6d5 2020-12-17 | [config vlan]: Remove `-t` flag from docker exec command (sonic-net#1317) [Lawrence Lee]
* 281b157 2020-12-17 | [fast-reboot-dump] Fix exception in struct.pack (sonic-net#1309) [bingwang-ms]
* 269c317 2020-12-16 | Mgmt vrf/SNMP agent validations and bug fixes (sonic-net#1289) [Suman Kumar]
* 394b202 2020-12-16 | [Dynamic buffer calc] Support dynamic buffer calculation (sonic-net#973) [Stephen Sun]
* 6dd5d31 2020-12-14 | show tech with platform dump option (sonic-net#1158) [Sujin Kang]
* 6255384 2020-12-13 | [kdump]: Parse sonic_platform kernel command line parameter to read the platform identifier string (sonic-net#1291) [rajendra-dendukuri]
* 0603c0b 2020-12-13 | [pcieutil] Remove 'pcie-' prefix from arguments (sonic-net#1297) [Joe LeVeque]
* 10ffc28 2020-12-13 | Added 'detailed' option for 'show interface counters' command (sonic-net#1299) [Akhilesh Samineni]
* 980ea0d 2020-12-11 | Fix show ip route summary on pizzabox platforms (sonic-net#1302) [gechiang]
* af1bb47 2020-12-10 | [acl_loader] Fix default DENY rule for V6 dataplane ACLs (sonic-net#1281) [Danny Allen]
* 57a0b41 2020-12-09 | Add show and clear commands for headroom pool watermark  (sonic-net#1144) [Neetha John]
* d414970 2020-12-08 | [unit test][CLI][pfcwd] Added pfcwd config tests for single and multi ASIC platform. (sonic-net#1248) [Mahesh Maddikayala]
* 2b4a58c 2020-12-08 | [sflow] Fix traceback seen for show sflow interface (sonic-net#1282) [Garrick He]
* e22980f 2020-12-09 | [config/console][consutil] Support enable/disable console switch (sonic-net#1275) [Blueve]
* 2f263c4 2020-12-08 | [fast-reboot] Fix fast-reboot when NDP entries are present (sonic-net#1295) [shlomibitton]
* 326e534 2020-12-07 | Fast-reboot: add a new flag to ignore ASIC config checksum verification failures (sonic-net#1292) [Vaibhav Hemant Dixit]
* 042254e 2020-12-04 | Kdump improvements (sonic-net#1284) [rajendra-dendukuri]
**- Why I did it**

python2 is end of life and SONiC is going to support python3. This PR is going to support:

1. Build pmon daemons with python3
2. Install and run python3 version pmon daemons

**- How I did it**

1. Change pmon daemons make files to build bothe python2 and python3 whl
2. Change docker-platform-monitor make files to install both python2 and python3 whl
3. Change pmon docker startup files to start pmon daemons according to the supported platform API version
…#6291)

swss/teamd/syncd services were changed to always enabled
in commit fad481e as a workaround
for not letting hostcfgd start service during the bootup process.

commit 317a4b3 introduce
wait till full system bootup before updating feature states in hostcfgd.

Thus, workaround introduced in commit fad481e can be removed

Signed-off-by: Guohan Lu <lguohan@gmail.com>
…e prefix

frr does not advertise route if local route is not reachable, as a result
loopback route /64 is not advertised to the neighbors. Add static route
allows frr to advertise the route to its peers

Signed-off-by: Guohan Lu <lguohan@gmail.com>
* 321291a 2020-12-26 | [loopback]: skip route pointing to loopback interface (sonic-net#1570) (HEAD, origin/master, origin/HEAD) [lguohan]
* 41fd2c4 2020-12-26 | [CRM] Safety check for division by 0 (sonic-net#1569) [Prince Sunny]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
* 7924bd1 2020-12-28 | [route_check]: ignore routes pointing to Loopback interface (sonic-net#1337) (HEAD, origin/master, origin/HEAD) [lguohan]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
This PR is in preparation to move from snmp.yml to configdb. This will more closely align with other commands in sonic and use configdb as the source of truth for snmp configuration.
Note: This is the first of 2 PR's to enable this. This PR will not change any functionality but will allow the snmp.yml file info to be put into the configdb.
Created a script that takes the snmp.yml variables and converts them to the configdb format.
Added file to dockerfile.j2 so that file is copied in the container.
Updated start.sh file to automatically run the python conversion script each time the docker container is restarted.
…orms (sonic-net#6304)

Mellanox already supports multiple destination IPs in IPinIP tunnel configuration, thus removing mellanox
exception for IPinIP configuration.

- How I did it

Removed "dst_ip" field generation in mellanox platform condition.
Sorted the "dst_ip" list, so that it is easier to test against sample configuration in unit tests.
Aligned unit test sample.

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
Build libyang1 which is required for frr 7.5
Wirut Getbamrung and others added 27 commits January 15, 2021 10:20
…onic-net#6089)

**- Why I did it**
- The thermalctld daemon on the Pmon docker requires support from the thermal manager API.

**- How I did it**
- Removed the old function for detecting a faulty fan.
- Removed the old function for detecting excess temperature.
- Implement thermal_manager APIs based on ThermalManagerBase
- Implement thermal_conditions APIs based on ThermalPolicyConditionBase
- Implement thermal_actions APIs based on ThermalPolicyActionBase
- Implement thermal_info APIs based on ThermalPolicyInfoBase
- Add thermal_policy.json
…ent service (sonic-net#6465)

**- Why I did it**

On the Mellanox platform, reboot cause is fetched from some certain sysfs which is created by the hw-management service. So determine-reboot-cause service shall start after hw-management, otherwise it could fail due to the related sysfs is not available yet.

**- How I did it**

Add a patch to the hw-management service to make sure determine-reboot-cause service should start after it.

Signed-off-by: Kebo Liu <kebol@nvidia.com>
When building the sonic-slave-buster docker container, the node.js package is
installed to meet the requirements of the Azure DevOPs pipleline
build. Recently this install of node.js has been failing.

This commit fixes that build break by upgrading the
sonic-slave-buster build to install version 14.x of node.js which is the
current LTS version for buster.
…cks (sonic-net#6457)

Avoid sonic-cfggen crashing when a server does not have a configured loopback address in the minigraph

Signed-off-by: Lawrence Lee <lawlee@microsoft.com>
…ms speeds requirements (sonic-net#6279)

[DPB][MLNX][YANG] fixed range of max speed

- Why I did it
All Mellanox platforms require DPB modes with a specific set of speeds example

- How I did it
Extended regex pattern inside YANG model.
Supported platforms: SN2010, SN2100, SN2410, SN2700, SN3420, SN3700, SN3700C, SN3800, SN4600C, SN4410, SN4700

- How to verify it
Manually tested DPB CLI on all platform with all modes

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…t#6003)

[DPB] added capability files for SN2700 platform

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN2700

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

NOTE: breakout to 4 is currently not available as of missing functionality in DPB implementation.

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…t#6013)

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN3800

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…t#6015)

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN2010

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…t#6016)

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN2100

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…t#6017)

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN2410

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

NOTE: breakout to 4 is currently not available as of missing functionality in DPB implementation.

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…t#6021)

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN3700

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…t#6059)

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN4410

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
…et#6061)

- Why I did it
platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

- How I did it
Created capability files according to platform specification SN4600C

- How to verify it
Full qualification requires bugs fixes reported under sonic-buildimage

NOTE: breakout to 4 is currently not available as of missing functionality in DPB implementation.

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
dump netstat info on error

Signed-off-by: Guohan Lu <lguohan@gmail.com>
use dpkg-buildpackage build with fakeroot

Signed-off-by: Guohan Lu <lguohan@gmail.com>
Signed-off-by: Guohan Lu <lguohan@gmail.com>
…#6478)

Fix sonic-net#5026

There is a race condition between zebra server accepts connections and bgpd tries to connect. Bgpd has a chance to try to connect before zebra is ready. In this scenario, bgpd will try again after 10 seconds and operate as normal within these 10 seconds. As a consequence, whatever bgpd tries to sent to zebra will be missing in the 10 seconds. To avoid such a scenario, bgpd should start after zebra is ready to accept connections.
…t#6020)

platform.json and hwsku.json files are required for a feature called Dynamic Port Breakout

Signed-off-by: Vadym Hlushko <vadymh@nvidia.com>
* 3b330db4a 2021-01-18 | [build]: Fix build error when compiling for armhf (32-bit) (sonic-net#30) (HEAD, origin/master, origin/HEAD, master) [dflynn-Nokia]
* 56aaa225b 2021-01-16 | [ci]: add pipeline for armhf and arm64 (sonic-net#29) [lguohan]
* 90da6141c 2021-01-12 | [ci]: propagate the correct error code the next step (sonic-net#27) [lguohan]

Signed-off-by: Guohan Lu <lguohan@gmail.com>
[ci]: download artifacts from master branch (sonic-net#768)
Do not create fabric port if mapping is not available (sonic-net#769)
[syncd] Comparison logic log also current attr value on set operation (sonic-net#763)
Add fabric port test to vslib (sonic-net#737)
[ci]: use sonicbld pool (sonic-net#766)
[tests] Remove exit command blocking all tests to run (sonic-net#765)
[vslib]: adapt macsec sai 1.7.1 (sonic-net#755)
Add support for SAI_SWITCH_ATTR_AVAILABLE_IPMC_ENTRY needed by CRM (sonic-net#756)

Signed-off-by: Danny Allen <daall@microsoft.com>
- [route_check.py] - update includes checks on subscriptions (sonic-net/sonic-utilities#1344)
- Validations checks while adding a member to PortChannel and removing a member from a Portchannel (sonic-net/sonic-utilities#1328)
- [show] Add subcommand to show midplane status for modular chassis (sonic-net/sonic-utilities#1267)
- [pytest][qos][config] Added pytests for "config qos reload" commands" (sonic-net/sonic-utilities#1346)
- Drop explict 3 seconds pause between two object updates/deletes. (sonic-net/sonic-utilities#1359)
- [show]fix for show muxcable status by replacing "hostname" to "peer_switch" for deriving tor ipv4_address (sonic-net/sonic-utilities#1360)
- [PFCWD] Fix 'start' pfcwd command (sonic-net/sonic-utilities#1345)

Signed-off-by: Shlomi Bitton <shlomibi@nvidia.com>
…6406)

**- Why I did it**
swsssdk will be deprecated. Migrate sonic-config-engine to use libswsscommon library instead

**- How to verify it**
Unit test
…th prefix to fpmsyncd. (sonic-net#4806)

Currently FRR is send Prefix with VNI information to FPMSYNCD. This PR allows FRR to send RMAC with EVPN Type5 prefix to fpmsyncd. This is a temp fix. This patch will be removed once neighorch is ready to handle the Prefix and ARP (containing RMAC) separately.
…t#5774)

* support reproduceable build for git clone

Signed-off-by: shilongliu <shilongliu@microsoft.com>

* fix

Co-authored-by: shilongliu <shilongliu@microsoft.com>
Changes in this update:
37695c8 [show]: Use TCP Connection For Muxcable Commands (sonic-net#1371)
8119ba2 Validations checks while creating and deleting a Portchannel (sonic-net#1326)
3df267e [config] Fix Breakout mode option and BREAKOUT_CFG table check method (sonic-net#1270)
9bd709b [show] Fix show arp in case with FDB entries, linked to default VLAN (sonic-net#1357)
bc2d27e [generate_dump]: fix syntax error

signed-of-by: Tamer Ahmed <tamer.ahmed@microsoft.com>
@dmytroxshevchuk dmytroxshevchuk force-pushed the optional_fields_hwsku branch 2 times, most recently from b7088b3 to 427903e Compare January 21, 2021 15:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.