From a1243c38385e4ccd3229cca31c1b710b5f1b584b Mon Sep 17 00:00:00 2001 From: bbinxie Date: Sun, 12 Jul 2020 17:23:11 +0800 Subject: [PATCH] 201911 (#199) * [brcmsai]: Updated BRCM SAI Debina package to 3.7.5.1-2 (#4916) Fix for Copp Rules not having Policer Rate-Limit applied. Signed-off-by: Abhishek Dosi * [nephos]: upgrade Nephos SAI version to c749df (#4814) Verified with Nephos nps8365 based platform Accton AS7116-54x. * "[config]: Multi ASIC loopback changes (#4895) Resubmitting the changes for (#4825) with fixes for sonic-bgpcdgd test failures Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan * [mellanox]: Support warm reboot on MSN4700 (#4910) * [Submodule Update] Sonic-platform-common [sfp_base] Update return value documentation of channel-specific methods (#98) [SfpBase] Fix key name typo in docstring (#99) [sfp] Tweak key names of some transceiver info fields (#97) [sfputil] Make SfpUtilHelper.get_physical_to_logical noexcept as in SfpUtilBase (#96) * [Submodule update] sonic-platform-daemons [xcvrd] Update key names in 'get_media_settings_value()' (#63) [xcvrd] Tweak some transceiver info key names (#62) * [Submodule update] sonic-utilities [sfpshow][mock_state_db] Tweak key names of some transceiver info fields (#958) [config] Fix syntax error (#966) [config] Fix indentation level in _get_disabled_services_list() (#965) * [Submodule Update] sonic-swss [aclorch] Use IPv6 Next Header internally for protocol number on MLNX platform (#1343) * [Submodule Update] Add support for attribute capability query in lua script (#362) * Cherry-pick was not clean. Fixing it. Signed-off-by: Abhishek Dosi * [telemetry] Call sonic-cfggen Once (#4901) sonic-cfggen call is slow and this is taking place in the SONiC boot up process. The change uses templates to assemble all required vars into single template file. With this change, telemetry now calls once into sonic-cfggen. signed-off-by: Tamer Ahmed * [mgmt docker] move pycryptodome installation to the end of the docker building (#4917) * [mgmt docker] move pycryptodome installation to the end of the docker building Signed-off-by: Ying Xie * pin down the version to current: 3.9.8 * comment * Add support for bcmsh and bcmcmd utlitites in multi ASIC devices (#4926) Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan This PR has changes to support accessing the bcmsh and bcmcmd utilities on multi ASIC devices Changes done - move the link of /var/run/sswsyncd from docker-syncd-brcm.mk to docker_image_ctl.j2 - update the bcmsh and bcmcmd scripts to take -n [ASIC_ID] as an argument on multi ASIC platforms * [caclmgrd] Improve code reuse (#4931) Improve code reuse in `generate_block_ip2me_traffic_iptables_commands()` function. * [Submodule Update] sonic-utilities Intf table migration for APP_DB entries during warmboot (#980) [Multi NPU] Time Improvements to the config reload/load_minigraph commands (#917) * [Submodule Update] sonic-py-swssdk [MultiDB]: use python class composition to avoid confusion in base class (#74) * [Submodule update] sonic-snmpagent. Movent to 201911 Branch with with following PR's : Implement cbgpPeer2State in CiscoBgp4MIB (#119) Fix index nodes in LLDP tables whose access right is not-accessible. (#112) Fix quagga/FRR parser on IPv6 BGP sessions (#122) [lint] Fix some syntax errors or warnings (#127) Update README.md: Add lgtm badges (#128) [Multi-asic]: Support multi-asic platform (#126) Simplify test code (#132) [Multi-asic]: Namespace support for LLDP and Sensor tables (#131) Fix undefined variable and warning message (#134) Fix SNMP AgentX socket connection timeout when using Namespace.get_all() (#140) [Namespace] Fix interfaces counters in InterfacesMIB RFC 2863 (#141) Fix LGTM reported alert of PR#141 (#142) * [bgpcfgd] - Fix a key error during delete (#4946) Co-authored-by: abdosi <58047199+abdosi@users.noreply.github.com> Co-authored-by: gracelicd <39251567+gracelicd@users.noreply.github.com> Co-authored-by: arlakshm <55814491+arlakshm@users.noreply.github.com> Co-authored-by: Stephen Sun <5379172+stephenxs@users.noreply.github.com> Co-authored-by: Abhishek Dosi Co-authored-by: Tamer Ahmed Co-authored-by: Ying Xie Co-authored-by: Joe LeVeque Co-authored-by: Prince Sunny --- .../ACS-MSN4700/sai_4700.xml | 3 + dockers/docker-base-stretch/Dockerfile.j2 | 4 +- .../docker-fpm-frr/frr/bgpd/bgpd.main.conf.j2 | 16 ++- dockers/docker-sonic-mgmt/Dockerfile.j2 | 5 +- dockers/docker-sonic-telemetry/Dockerfile.j2 | 2 +- dockers/docker-sonic-telemetry/telemetry.sh | 35 +++--- .../docker-sonic-telemetry/telemetry_vars.j2 | 5 + files/build_templates/docker_image_ctl.j2 | 5 + files/image_config/caclmgrd/caclmgrd | 108 +++++------------- platform/broadcom/docker-syncd-brcm.mk | 2 +- .../base_image_files/bcm_common | 40 +++++++ .../docker-syncd-brcm/base_image_files/bcmcmd | 7 +- .../docker-syncd-brcm/base_image_files/bcmsh | 7 +- platform/broadcom/sai.mk | 8 +- platform/nephos/rules.mk | 2 +- platform/nephos/sai.mk | 15 +-- src/sonic-bgpcfgd/bgpcfgd | 2 +- src/sonic-config-engine/minigraph.py | 32 ++++-- src/sonic-config-engine/sonic-cfggen | 3 + src/sonic-config-engine/sonic_device_util.py | 14 ++- .../tests/multi_npu_data/sample-minigraph.xml | 40 ++++++- .../tests/test_multinpu_cfggen.py | 29 +++++ src/sonic-platform-common | 2 +- src/sonic-platform-daemons | 2 +- src/sonic-py-swsssdk | 2 +- src/sonic-snmpagent | 2 +- src/sonic-swss | 2 +- src/sonic-swss-common | 2 +- src/sonic-utilities | 2 +- 29 files changed, 256 insertions(+), 142 deletions(-) create mode 100644 dockers/docker-sonic-telemetry/telemetry_vars.j2 create mode 100644 platform/broadcom/docker-syncd-brcm/base_image_files/bcm_common diff --git a/device/mellanox/x86_64-mlnx_msn4700-r0/ACS-MSN4700/sai_4700.xml b/device/mellanox/x86_64-mlnx_msn4700-r0/ACS-MSN4700/sai_4700.xml index 177a79d13d6f..e3d0e4ea723d 100644 --- a/device/mellanox/x86_64-mlnx_msn4700-r0/ACS-MSN4700/sai_4700.xml +++ b/device/mellanox/x86_64-mlnx_msn4700-r0/ACS-MSN4700/sai_4700.xml @@ -5,6 +5,9 @@ 00:02:03:04:05:00 + + 1 + 32 diff --git a/dockers/docker-base-stretch/Dockerfile.j2 b/dockers/docker-base-stretch/Dockerfile.j2 index 49b6aa16ab57..2cb52d8ef57f 100644 --- a/dockers/docker-base-stretch/Dockerfile.j2 +++ b/dockers/docker-base-stretch/Dockerfile.j2 @@ -55,7 +55,9 @@ RUN apt-get update && \ libjemalloc1 \ liblua5.1-0 \ lua-bitop \ - lua-cjson + lua-cjson \ +# for processing json files in bash environment + jq {% if CONFIGURED_ARCH == "armhf" %} # ip and ifconfig utility missing in docker for armhf diff --git a/dockers/docker-fpm-frr/frr/bgpd/bgpd.main.conf.j2 b/dockers/docker-fpm-frr/frr/bgpd/bgpd.main.conf.j2 index ef98dcf9f16a..5fe7a2b01051 100644 --- a/dockers/docker-fpm-frr/frr/bgpd/bgpd.main.conf.j2 +++ b/dockers/docker-fpm-frr/frr/bgpd/bgpd.main.conf.j2 @@ -15,7 +15,7 @@ ipv6 prefix-list PL_LoopbackV6 permit {{ get_ipv6_loopback_address(LOOPBACK_INTE {% endif %} ! ! -{% if DEVICE_METADATA['localhost']['sub_role'] == 'FrontEnd' %} +{% if DEVICE_METADATA['localhost']['sub_role'] == 'FrontEnd' or DEVICE_METADATA['localhost']['sub_role'] == 'BackEnd' %} route-map HIDE_INTERNAL permit 10 set community local-AS ! @@ -38,16 +38,30 @@ router bgp {{ DEVICE_METADATA['localhost']['bgp_asn'] }} {% endif %} ! {# set router-id #} +{% if multi_asic() %} + bgp router-id {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback4096") | ip }} +{% else %} bgp router-id {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | ip }} +{% endif %} ! {# advertise loopback #} network {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | ip }}/32 +{% if multi_asic() %} + network {{ get_ipv4_loopback_address(LOOPBACK_INTERFACE, "Loopback4096") | ip }}/32 route-map HIDE_INTERNAL +{% endif %} ! {% if get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback0") != 'None' %} address-family ipv6 network {{ get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback0") | ip }}/64 exit-address-family {% endif %} +{% if multi_asic() %} +{% if get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback4096") != 'None' %} + address-family ipv6 + network {{ get_ipv6_loopback_address(LOOPBACK_INTERFACE, "Loopback4096") | ip }}/64 route-map HIDE_INTERNAL + exit-address-family +{% endif %} +{% endif %} {% endblock bgp_init %} ! {% block vlan_advertisement %} diff --git a/dockers/docker-sonic-mgmt/Dockerfile.j2 b/dockers/docker-sonic-mgmt/Dockerfile.j2 index 3c2bbff2504e..67142242293c 100644 --- a/dockers/docker-sonic-mgmt/Dockerfile.j2 +++ b/dockers/docker-sonic-mgmt/Dockerfile.j2 @@ -49,7 +49,6 @@ RUN pip install cffi==1.10.0 \ prettytable \ psutil \ pyasn1==0.1.9 \ - pycryptodome \ pyfiglet \ pylint==1.8.1 \ pyro4 \ @@ -169,3 +168,7 @@ RUN ~/lib/azure-cli/bin/python -m pip install azure-keyvault==0.3.7 -U # Install Virtual Environment RUN python -m virtualenv --system-site-packages env-201811 RUN env-201811/bin/pip install ansible==2.0.0.2 + +# NOTE: There is an ordering dependency for pycryptodome. Leaving this at +# the end until we figure that out. +RUN pip install pycryptodome==3.9.8 diff --git a/dockers/docker-sonic-telemetry/Dockerfile.j2 b/dockers/docker-sonic-telemetry/Dockerfile.j2 index e94441b4f066..470dbe9048ee 100644 --- a/dockers/docker-sonic-telemetry/Dockerfile.j2 +++ b/dockers/docker-sonic-telemetry/Dockerfile.j2 @@ -26,7 +26,7 @@ RUN apt-get clean -y && \ apt-get autoremove -y && \ rm -rf /debs -COPY ["start.sh", "telemetry.sh", "dialout.sh", "/usr/bin/"] +COPY ["start.sh", "telemetry.sh", "dialout.sh", "telemetry_vars.j2", "/usr/bin/"] COPY ["supervisord.conf", "/etc/supervisor/conf.d/"] COPY ["files/supervisor-proc-exit-listener", "/usr/bin"] COPY ["critical_processes", "/etc/supervisor"] diff --git a/dockers/docker-sonic-telemetry/telemetry.sh b/dockers/docker-sonic-telemetry/telemetry.sh index b8f7fffb3ebb..18356e0945e2 100755 --- a/dockers/docker-sonic-telemetry/telemetry.sh +++ b/dockers/docker-sonic-telemetry/telemetry.sh @@ -2,36 +2,38 @@ # Try to read telemetry and certs config from ConfigDB. # Use default value if no valid config exists -X509=`sonic-cfggen -d -v "DEVICE_METADATA['x509']"` -gnmi=`sonic-cfggen -d -v "TELEMETRY['gnmi']"` -certs=`sonic-cfggen -d -v "TELEMETRY['certs']"` +TELEMETRY_VARS=$(sonic-cfggen -d -t telemetry_vars.j2) +TELEMETRY_VARS=${TELEMETRY_VARS//[\']/\"} +X509=$(echo $TELEMETRY_VARS | jq -r '.x509') +GNMI=$(echo $TELEMETRY_VARS | jq -r '.gnmi') +CERTS=$(echo $TELEMETRY_VARS | jq -r '.certs') TELEMETRY_ARGS=" -logtostderr" export CVL_SCHEMA_PATH=/usr/sbin/schema -if [ -n "$certs" ]; then - SERVER_CRT=`sonic-cfggen -d -v "TELEMETRY['certs']['server_crt']"` - SERVER_KEY=`sonic-cfggen -d -v "TELEMETRY['certs']['server_key']"` +if [ -n "$CERTS" ]; then + SERVER_CRT=$(echo $CERTS | jq -r '.server_crt') + SERVER_KEY=$(echo $CERTS | jq -r '.server_key') if [ -z $SERVER_CRT ] || [ -z $SERVER_KEY ]; then TELEMETRY_ARGS+=" --insecure" else TELEMETRY_ARGS+=" --server_crt $SERVER_CRT --server_key $SERVER_KEY " fi - CA_CRT=`sonic-cfggen -d -v "TELEMETRY['certs']['ca_crt']"` + CA_CRT=$(echo $CERTS | jq -r '.ca_crt') if [ ! -z $CA_CRT ]; then TELEMETRY_ARGS+=" --ca_crt $CA_CRT" fi elif [ -n "$X509" ]; then - SERVER_CRT=`sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_crt']"` - SERVER_KEY=`sonic-cfggen -d -v "DEVICE_METADATA['x509']['server_key']"` + SERVER_CRT=$(echo $X509 | jq -r '.server_crt') + SERVER_KEY=$(echo $X509 | jq -r '.server_key') if [ -z $SERVER_CRT ] || [ -z $SERVER_KEY ]; then TELEMETRY_ARGS+=" --insecure" else TELEMETRY_ARGS+=" --server_crt $SERVER_CRT --server_key $SERVER_KEY " fi - CA_CRT=`sonic-cfggen -d -v "DEVICE_METADATA['x509']['ca_crt']"` + CA_CRT=$(echo $X509 | jq -r '.ca_crt') if [ ! -z $CA_CRT ]; then TELEMETRY_ARGS+=" --ca_crt $CA_CRT" fi @@ -40,19 +42,20 @@ else fi # If no configuration entry exists for TELEMETRY, create one default port -if [ -z "$gnmi" ]; then - sonic-db-cli CONFIG_DB hset "TELEMETRY|gnmi" port 8080 +if [ -z "$GNMI" ]; then + PORT=8080 + sonic-db-cli CONFIG_DB hset "TELEMETRY|gnmi" port $PORT +else + PORT=$(echo $GNMI | jq -r '.port') fi - -PORT=`sonic-cfggen -d -v "TELEMETRY['gnmi']['port']"` TELEMETRY_ARGS+=" --port $PORT" -CLIENT_AUTH=`sonic-cfggen -d -v "TELEMETRY['gnmi']['client_auth']"` +CLIENT_AUTH=$(echo $GNMI | jq -r '.client_auth') if [ -z $CLIENT_AUTH ] || [ $CLIENT_AUTH == "false" ]; then TELEMETRY_ARGS+=" --allow_no_client_auth" fi -LOG_LEVEL=`sonic-cfggen -d -v "TELEMETRY['gnmi']['log_level']"` +LOG_LEVEL=$(echo $GNMI | jq -r '.log_level') if [ ! -z $LOG_LEVEL ]; then TELEMETRY_ARGS+=" -v=$LOG_LEVEL" else diff --git a/dockers/docker-sonic-telemetry/telemetry_vars.j2 b/dockers/docker-sonic-telemetry/telemetry_vars.j2 new file mode 100644 index 000000000000..687781e1b9b2 --- /dev/null +++ b/dockers/docker-sonic-telemetry/telemetry_vars.j2 @@ -0,0 +1,5 @@ +{ + "certs": "{% if "certs" in TELEMETRY.keys() %}{{ TELEMETRY["certs"] }}{% endif %}", + "gnmi" : "{% if "gnmi" in TELEMETRY.keys() %}{{ TELEMETRY["gnmi"] }}{% endif %}", + "x509" : "{% if "x509" in DEVICE_METADATA.keys() %}{{ DEVICE_METADATA["x509"] }}{% endif %}" +} diff --git a/files/build_templates/docker_image_ctl.j2 b/files/build_templates/docker_image_ctl.j2 index e8cc666779d8..4a0fd7750bd9 100644 --- a/files/build_templates/docker_image_ctl.j2 +++ b/files/build_templates/docker_image_ctl.j2 @@ -299,6 +299,11 @@ start() { --tmpfs /tmp \ {%- endif %} {%- endif %} +{%- if sonic_asic_platform == "broadcom" %} +{%- if docker_container_name == "syncd" %} + -v /var/run/docker-syncd$DEV:/var/run/sswsyncd \ +{%- endif %} +{%- endif %} {%- if docker_container_name == "bgp" %} -v /etc/sonic/frr/$DEV:/etc/frr:rw \ {%- endif %} diff --git a/files/image_config/caclmgrd/caclmgrd b/files/image_config/caclmgrd/caclmgrd index 37a86418967e..b6d5cbe71398 100755 --- a/files/image_config/caclmgrd/caclmgrd +++ b/files/image_config/caclmgrd/caclmgrd @@ -134,89 +134,37 @@ class ControlPlaneAclManager(object): return tcp_flags_str def generate_block_ip2me_traffic_iptables_commands(self): - LOOPBACK_INTERFACE_TABLE_NAME = "LOOPBACK_INTERFACE" - MGMT_INTERFACE_TABLE_NAME = "MGMT_INTERFACE" - VLAN_INTERFACE_TABLE_NAME = "VLAN_INTERFACE" - PORTCHANNEL_INTERFACE_TABLE_NAME = "PORTCHANNEL_INTERFACE" - INTERFACE_TABLE_NAME = "INTERFACE" + INTERFACE_TABLE_NAME_LIST = [ + "LOOPBACK_INTERFACE", + "MGMT_INTERFACE", + "VLAN_INTERFACE", + "PORTCHANNEL_INTERFACE", + "INTERFACE" + ] block_ip2me_cmds = [] - # Add iptables rules to drop all packets destined for loopback interface IP addresses - loopback_iface_table = self.config_db.get_table(LOOPBACK_INTERFACE_TABLE_NAME) - if loopback_iface_table: - for key, _ in loopback_iface_table.iteritems(): - if not _ip_prefix_in_key(key): - continue - iface_name, iface_cidr = key - ip_ntwrk = ipaddress.ip_network(iface_cidr, strict=False) - if isinstance(ip_ntwrk, ipaddress.IPv4Network): - block_ip2me_cmds.append("iptables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - elif isinstance(ip_ntwrk, ipaddress.IPv6Network): - block_ip2me_cmds.append("ip6tables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - else: - log_warning("Unrecognized IP address type on interface '{}': {}".format(iface_name, ip_ntwrk)) - - # Add iptables rules to drop all packets destined for management interface IP addresses - mgmt_iface_table = self.config_db.get_table(MGMT_INTERFACE_TABLE_NAME) - if mgmt_iface_table: - for key, _ in mgmt_iface_table.iteritems(): - if not _ip_prefix_in_key(key): - continue - iface_name, iface_cidr = key - ip_ntwrk = ipaddress.ip_network(iface_cidr, strict=False) - if isinstance(ip_ntwrk, ipaddress.IPv4Network): - block_ip2me_cmds.append("iptables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - elif isinstance(ip_ntwrk, ipaddress.IPv6Network): - block_ip2me_cmds.append("ip6tables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - else: - log_warning("Unrecognized IP address type on interface '{}': {}".format(iface_name, ip_ntwrk)) - - # Add iptables rules to drop all packets destined for our VLAN interface gateway IP addresses - vlan_iface_table = self.config_db.get_table(VLAN_INTERFACE_TABLE_NAME) - if vlan_iface_table: - for key, _ in vlan_iface_table.iteritems(): - if not _ip_prefix_in_key(key): - continue - iface_name, iface_cidr = key - ip_ntwrk = ipaddress.ip_network(iface_cidr, strict=False) - first_host = next(ip_ntwrk.hosts()) - if isinstance(ip_ntwrk, ipaddress.IPv4Network): - block_ip2me_cmds.append("iptables -A INPUT -d {}/{} -j DROP".format(first_host, ip_ntwrk.max_prefixlen)) - elif isinstance(ip_ntwrk, ipaddress.IPv6Network): - block_ip2me_cmds.append("ip6tables -A INPUT -d {}/{} -j DROP".format(first_host, ip_ntwrk.max_prefixlen)) - else: - log_warning("Unrecognized IP address type on interface '{}': {}".format(iface_name, ip_ntwrk)) - - # Add iptables rules to drop all packets destined for point-to-point interface IP addresses - # (All portchannel interfaces and configured front-panel interfaces) - portchannel_iface_table = self.config_db.get_table(PORTCHANNEL_INTERFACE_TABLE_NAME) - if portchannel_iface_table: - for key, _ in portchannel_iface_table.iteritems(): - if not _ip_prefix_in_key(key): - continue - iface_name, iface_cidr = key - ip_ntwrk = ipaddress.ip_network(iface_cidr, strict=False) - if isinstance(ip_ntwrk, ipaddress.IPv4Network): - block_ip2me_cmds.append("iptables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - elif isinstance(ip_ntwrk, ipaddress.IPv6Network): - block_ip2me_cmds.append("ip6tables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - else: - log_warning("Unrecognized IP address type on interface '{}': {}".format(iface_name, ip_ntwrk)) - - iface_table = self.config_db.get_table(INTERFACE_TABLE_NAME) - if iface_table: - for key, _ in iface_table.iteritems(): - if not _ip_prefix_in_key(key): - continue - iface_name, iface_cidr = key - ip_ntwrk = ipaddress.ip_network(iface_cidr, strict=False) - if isinstance(ip_ntwrk, ipaddress.IPv4Network): - block_ip2me_cmds.append("iptables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - elif isinstance(ip_ntwrk, ipaddress.IPv6Network): - block_ip2me_cmds.append("ip6tables -A INPUT -d {}/{} -j DROP".format(ip_ntwrk.network_address, ip_ntwrk.max_prefixlen)) - else: - log_warning("Unrecognized IP address type on interface '{}': {}".format(iface_name, ip_ntwrk)) + # Add iptables rules to drop all packets destined for peer-to-peer interface IP addresses + for iface_table_name in INTERFACE_TABLE_NAME_LIST: + iface_table = self.config_db.get_table(iface_table_name) + if iface_table: + for key, _ in iface_table.iteritems(): + if not _ip_prefix_in_key(key): + continue + + iface_name, iface_cidr = key + ip_ntwrk = ipaddress.ip_network(iface_cidr, strict=False) + + # For VLAN interfaces, the IP address we want to block is the default gateway (i.e., + # the first available host IP address of the VLAN subnet) + ip_addr = next(ip_ntwrk.hosts()) if iface_table_name == "VLAN_INTERFACE" else ip_ntwrk.network_address + + if isinstance(ip_ntwrk, ipaddress.IPv4Network): + block_ip2me_cmds.append("iptables -A INPUT -d {}/{} -j DROP".format(ip_addr, ip_ntwrk.max_prefixlen)) + elif isinstance(ip_ntwrk, ipaddress.IPv6Network): + block_ip2me_cmds.append("ip6tables -A INPUT -d {}/{} -j DROP".format(ip_addr, ip_ntwrk.max_prefixlen)) + else: + log_warning("Unrecognized IP address type on interface '{}': {}".format(iface_name, ip_ntwrk)) return block_ip2me_cmds diff --git a/platform/broadcom/docker-syncd-brcm.mk b/platform/broadcom/docker-syncd-brcm.mk index d3a6d67c5cbc..d23141481915 100644 --- a/platform/broadcom/docker-syncd-brcm.mk +++ b/platform/broadcom/docker-syncd-brcm.mk @@ -12,8 +12,8 @@ $(DOCKER_SYNCD_BASE)_DBG_DEPENDS += $(SYNCD_DBG) \ $(LIBSAIREDIS_DBG) $(DOCKER_SYNCD_BASE)_RUN_OPT += -v /host/warmboot:/var/warmboot -$(DOCKER_SYNCD_BASE)_RUN_OPT += -v /var/run/docker-syncd:/var/run/sswsyncd $(DOCKER_SYNCD_BASE)_BASE_IMAGE_FILES += bcmcmd:/usr/bin/bcmcmd $(DOCKER_SYNCD_BASE)_BASE_IMAGE_FILES += bcmsh:/usr/bin/bcmsh +$(DOCKER_SYNCD_BASE)_BASE_IMAGE_FILES += bcm_common:/usr/bin/bcm_common $(DOCKER_SYNCD_BASE)_BASE_IMAGE_FILES += monit_syncd:/etc/monit/conf.d diff --git a/platform/broadcom/docker-syncd-brcm/base_image_files/bcm_common b/platform/broadcom/docker-syncd-brcm/base_image_files/bcm_common new file mode 100644 index 000000000000..1b560a1a1522 --- /dev/null +++ b/platform/broadcom/docker-syncd-brcm/base_image_files/bcm_common @@ -0,0 +1,40 @@ +#!/bin/bash + +function help() +{ + echo "Usage: $0 -n [0 to $(($NUM_ASIC-1))]" 1>&2; exit 1; + +} + + +DEV="" + +PLATFORM=`sonic-cfggen -H -v DEVICE_METADATA.localhost.platform` + +# Parse the device specific asic conf file, if it exists + +ASIC_CONF=/usr/share/sonic/device/$PLATFORM/asic.conf +if [ -f "$ASIC_CONF" ]; then + source $ASIC_CONF +fi + + +if [[ ($NUM_ASIC -gt 1) ]]; then + OPTIND=1 + + while getopts ":n:h:" opt; do + case "${opt}" in + h) help + exit 0 + ;; + n) DEV=${OPTARG} + [ $DEV -lt $NUM_ASIC -a $DEV -ge 0 ] || help + ;; + esac + done + shift "$((OPTIND-1))" + + if [ -z "${DEV}" ]; then + help + fi +fi diff --git a/platform/broadcom/docker-syncd-brcm/base_image_files/bcmcmd b/platform/broadcom/docker-syncd-brcm/base_image_files/bcmcmd index 7903db6ed6a3..76362fc64804 100755 --- a/platform/broadcom/docker-syncd-brcm/base_image_files/bcmcmd +++ b/platform/broadcom/docker-syncd-brcm/base_image_files/bcmcmd @@ -1,3 +1,8 @@ #!/bin/bash -docker exec -i syncd bcmcmd "$@" +BCM_COMMON=/usr/bin/bcm_common +if [ -f "$BCM_COMMON" ]; then + source $BCM_COMMON +fi +docker exec -i syncd$DEV bcmcmd "$@" + diff --git a/platform/broadcom/docker-syncd-brcm/base_image_files/bcmsh b/platform/broadcom/docker-syncd-brcm/base_image_files/bcmsh index 3bb78b0da796..3cb2aad7afb6 100755 --- a/platform/broadcom/docker-syncd-brcm/base_image_files/bcmsh +++ b/platform/broadcom/docker-syncd-brcm/base_image_files/bcmsh @@ -1,3 +1,8 @@ #!/bin/bash -docker exec -it syncd bcmsh "$@" +BCM_COMMON=/usr/bin/bcm_common +if [ -f "$BCM_COMMON" ]; then + source $BCM_COMMON +fi + +docker exec -it syncd$DEV bcmsh "$@" diff --git a/platform/broadcom/sai.mk b/platform/broadcom/sai.mk index d92305d4240a..a65f5ac4fbb8 100644 --- a/platform/broadcom/sai.mk +++ b/platform/broadcom/sai.mk @@ -1,8 +1,8 @@ -BRCM_SAI = libsaibcm_3.7.5.1-1_amd64.deb -$(BRCM_SAI)_URL = "https://sonicstorage.blob.core.windows.net/packages/bcmsai/3.7/libsaibcm_3.7.5.1-1_amd64.deb?sv=2015-04-05&sr=b&sig=cxmXsJ%2BjcnR9ckFRbMigIbkzOncYkiV04weL%2FVPKBmk%3D&se=2034-03-06T00%3A30%3A30Z&sp=r" -BRCM_SAI_DEV = libsaibcm-dev_3.7.5.1-1_amd64.deb +BRCM_SAI = libsaibcm_3.7.5.1-2_amd64.deb +$(BRCM_SAI)_URL = "https://sonicstorage.blob.core.windows.net/packages/bcmsai/3.7/libsaibcm_3.7.5.1-2_amd64.deb?sv=2015-04-05&sr=b&sig=NMXmDm7ME%2BDN9n4kw6wXgIVmIjRifu%2FWV0UbLU9qllw%3D&se=2034-03-17T05%3A53%3A29Z&sp=r" +BRCM_SAI_DEV = libsaibcm-dev_3.7.5.1-2_amd64.deb $(eval $(call add_derived_package,$(BRCM_SAI),$(BRCM_SAI_DEV))) -$(BRCM_SAI_DEV)_URL = "https://sonicstorage.blob.core.windows.net/packages/bcmsai/3.7/libsaibcm-dev_3.7.5.1-1_amd64.deb?sv=2015-04-05&sr=b&sig=LVgghAv75VG4idW6xfpId%2FlrvPBja7uBQeTbjZsR3CA%3D&se=2034-03-06T00%3A31%3A30Z&sp=r" +$(BRCM_SAI_DEV)_URL = "https://sonicstorage.blob.core.windows.net/packages/bcmsai/3.7/libsaibcm-dev_3.7.5.1-2_amd64.deb?sv=2015-04-05&sr=b&sig=3Q8S5fwg7WV%2BCKVwMALrf8dpQWK2cSD4J4zxbVht%2BT8%3D&se=2034-03-17T05%3A54%3A05Z&sp=r" SONIC_ONLINE_DEBS += $(BRCM_SAI) $(BRCM_SAI_DEV)_DEPENDS += $(BRCM_SAI) diff --git a/platform/nephos/rules.mk b/platform/nephos/rules.mk index 5c115eeaf8d0..188dc6856f4f 100644 --- a/platform/nephos/rules.mk +++ b/platform/nephos/rules.mk @@ -23,7 +23,7 @@ SONIC_ONLINE_FILES += $(NPX_DIAG) $(WARM_VERIFIER) $(DSSERVE) SONIC_ALL += $(SONIC_ONE_IMAGE) $(DOCKER_FPM) # Inject nephos sai into sairedis -$(LIBSAIREDIS)_DEPENDS += $(NEPHOS_SAI) $(NEPHOS_SAI_DEV) +$(LIBSAIREDIS)_DEPENDS += $(NEPHOS_SAI) ifeq ($(ENABLE_SYNCD_RPC),y) $(LIBSAIREDIS)_DEPENDS += $(LIBSAITHRIFT_DEV) endif diff --git a/platform/nephos/sai.mk b/platform/nephos/sai.mk index 5f7c4a23ae95..0a49b408a8a0 100644 --- a/platform/nephos/sai.mk +++ b/platform/nephos/sai.mk @@ -1,6 +1,6 @@ SDK_VERSION = 3.0.0 -SAI_VERSION = 1.5.0 -SAI_COMMIT_ID = 06a67d +SAI_VERSION = 1.5.1 +SAI_COMMIT_ID = c749df # Place here URL where SAI deb exist NEPHOS_SAI_DEB_LOCAL_URL = @@ -19,17 +19,10 @@ else $(NEPHOS_SAI)_URL = "https://github.com/NephosInc/SONiC/raw/master/sai/libsainps_$(SDK_VERSION)_sai_$(SAI_VERSION)_$(SAI_COMMIT_ID)_amd64.deb" endif -NEPHOS_SAI_DEV = libsainps-dev_$(SDK_VERSION)_sai_$(SAI_VERSION)_$(SAI_COMMIT_ID)_amd64.deb -$(eval $(call add_derived_package,$(NEPHOS_SAI),$(NEPHOS_SAI_DEV))) -ifeq ($(SAI_FROM_LOCAL), y) -$(NEPHOS_SAI_DEV)_PATH = $(NEPHOS_SAI_DEB_LOCAL_URL) -else -$(NEPHOS_SAI_DEV)_URL = "https://github.com/NephosInc/SONiC/raw/master/sai/libsainps-dev_$(SDK_VERSION)_sai_$(SAI_VERSION)_$(SAI_COMMIT_ID)_amd64.deb" -endif ifeq ($(SAI_FROM_LOCAL), y) -SONIC_COPY_DEBS += $(NEPHOS_SAI) $(NEPHOS_SAI_DEV) +SONIC_COPY_DEBS += $(NEPHOS_SAI) else -SONIC_ONLINE_DEBS += $(NEPHOS_SAI) $(NEPHOS_SAI_DEV) +SONIC_ONLINE_DEBS += $(NEPHOS_SAI) endif $(NEPHOS_SAI_DEV)_DEPENDS += $(NEPHOS_SAI) diff --git a/src/sonic-bgpcfgd/bgpcfgd b/src/sonic-bgpcfgd/bgpcfgd index 0fbe44602a22..38d7f0671f89 100755 --- a/src/sonic-bgpcfgd/bgpcfgd +++ b/src/sonic-bgpcfgd/bgpcfgd @@ -647,7 +647,7 @@ class BGPPeerMgrBase(Manager): ret_code = self.apply_op(cmd, vrf) if ret_code: log_info("Peer '(%s|%s)' has been removed" % (vrf, nbr)) - self.peers.remove(key) + self.peers.remove(peer_key) else: log_err("Peer '(%s|%s)' hasn't been removed" % (vrf, nbr)) diff --git a/src/sonic-config-engine/minigraph.py b/src/sonic-config-engine/minigraph.py index 2c079f1b472b..6d4ea6a1ad65 100644 --- a/src/sonic-config-engine/minigraph.py +++ b/src/sonic-config-engine/minigraph.py @@ -251,6 +251,15 @@ def parse_asic_png(png, asic_name, hostname): devices[name] = device_data return (neighbors, devices, port_speeds) +def parse_loopback_intf(child): + lointfs = child.find(str(QName(ns, "LoopbackIPInterfaces"))) + lo_intfs = {} + for lointf in lointfs.findall(str(QName(ns1, "LoopbackIPInterface"))): + intfname = lointf.find(str(QName(ns, "AttachTo"))).text + ipprefix = lointf.find(str(QName(ns1, "PrefixStr"))).text + lo_intfs[(intfname, ipprefix)] = {} + return lo_intfs + def parse_dpg(dpg, hname): aclintfs = None mgmtintfs = None @@ -269,7 +278,6 @@ def parse_dpg(dpg, hname): """ if mgmtintfs is None and child.find(str(QName(ns, "ManagementIPInterfaces"))) is not None: mgmtintfs = child.find(str(QName(ns, "ManagementIPInterfaces"))) - hostname = child.find(str(QName(ns, "Hostname"))) if hostname.text.lower() != hname.lower(): continue @@ -290,12 +298,7 @@ def parse_dpg(dpg, hname): ipprefix = ipintf.find(str(QName(ns, "Prefix"))).text intfs[(intfname, ipprefix)] = {} - lointfs = child.find(str(QName(ns, "LoopbackIPInterfaces"))) - lo_intfs = {} - for lointf in lointfs.findall(str(QName(ns1, "LoopbackIPInterface"))): - intfname = lointf.find(str(QName(ns, "AttachTo"))).text - ipprefix = lointf.find(str(QName(ns1, "PrefixStr"))).text - lo_intfs[(intfname, ipprefix)] = {} + lo_intfs = parse_loopback_intf(child) mvrfConfigs = child.find(str(QName(ns, "MgmtVrfConfigs"))) mvrf = {} @@ -445,6 +448,13 @@ def parse_dpg(dpg, hname): return intfs, lo_intfs, mvrf, mgmt_intf, vlans, vlan_members, pcs, pc_members, acls, vni return None, None, None, None, None, None, None, None, None, None +def parse_host_loopback(dpg, hname): + for child in dpg: + hostname = child.find(str(QName(ns, "Hostname"))) + if hostname.text.lower() != hname.lower(): + continue + lo_intfs = parse_loopback_intf(child) + return lo_intfs def parse_cpg(cpg, hname): bgp_sessions = {} @@ -818,6 +828,7 @@ def parse_xml(filename, platform=None, port_config_file=None, asic_name=None): cloudtype = None hostname = None linkmetas = {} + host_lo_intfs = None # hostname is the asic_name, get the asic_id from the asic_name if asic_name is not None: @@ -859,6 +870,7 @@ def parse_xml(filename, platform=None, port_config_file=None, asic_name=None): else: if child.tag == str(QName(ns, "DpgDec")): (intfs, lo_intfs, mvrf, mgmt_intf, vlans, vlan_members, pcs, pc_members, acls, vni) = parse_dpg(child, asic_name) + host_lo_intfs = parse_host_loopback(child, hostname) elif child.tag == str(QName(ns, "CpgDec")): (bgp_sessions, bgp_asn, bgp_peers_with_range, bgp_monitors) = parse_cpg(child, asic_name) enable_internal_bgp_session(bgp_sessions, filename, asic_name) @@ -922,6 +934,12 @@ def parse_xml(filename, platform=None, port_config_file=None, asic_name=None): for lo_intf in lo_intfs: results['LOOPBACK_INTERFACE'][lo_intf] = lo_intfs[lo_intf] results['LOOPBACK_INTERFACE'][lo_intf[0]] = {} + + if host_lo_intfs is not None: + for host_lo_intf in host_lo_intfs: + results['LOOPBACK_INTERFACE'][host_lo_intf] = host_lo_intfs[host_lo_intf] + results['LOOPBACK_INTERFACE'][host_lo_intf[0]] = {} + results['MGMT_VRF_CONFIG'] = mvrf phyport_intfs = {} diff --git a/src/sonic-config-engine/sonic-cfggen b/src/sonic-config-engine/sonic-cfggen index c449bb1d17b4..0a94b810a189 100755 --- a/src/sonic-config-engine/sonic-cfggen +++ b/src/sonic-config-engine/sonic-cfggen @@ -46,6 +46,7 @@ from sonic_device_util import get_platform_info from sonic_device_util import get_system_mac from sonic_device_util import get_npu_id_from_name from sonic_device_util import get_npu_device_id +from sonic_device_util import is_multi_npu from config_samples import generate_sample_config from config_samples import get_available_config from swsssdk import SonicV2Connector, ConfigDBConnector, SonicDBConfig @@ -343,6 +344,8 @@ def main(): env.filters['ip_network'] = ip_network for attr in ['ip', 'network', 'prefixlen', 'netmask']: env.filters[attr] = partial(prefix_attr, attr) + # Pass the is_multi_npu function as global + env.globals['multi_asic'] = is_multi_npu template = env.get_template(template_file) print(template.render(sort_data(data))) diff --git a/src/sonic-config-engine/sonic_device_util.py b/src/sonic-config-engine/sonic_device_util.py index 1bdf33162dac..006d3f064dea 100644 --- a/src/sonic-config-engine/sonic_device_util.py +++ b/src/sonic-config-engine/sonic_device_util.py @@ -68,14 +68,20 @@ def get_npu_device_id(npu_id): device_id = tokens[1].strip() return device_id +def get_asic_conf_file_path(platform): + asic_conf_path_candidates = [] + asic_conf_path_candidates.append(os.path.join('/usr/share/sonic/platform', ASIC_CONF_FILENAME)) + if platform is not None: + asic_conf_path_candidates.append(os.path.join(SONIC_DEVICE_PATH, platform, ASIC_CONF_FILENAME)) + for asic_conf_file_path in asic_conf_path_candidates: + if os.path.isfile(asic_conf_file_path): + return asic_conf_file_path return None def get_num_npus(): platform = get_platform_info(get_machine_info()) - if not platform: - return 1 - asic_conf_file_path = os.path.join(SONIC_DEVICE_PATH, platform, ASIC_CONF_FILENAME) - if not os.path.isfile(asic_conf_file_path): + asic_conf_file_path = get_asic_conf_file_path(platform) + if asic_conf_file_path is None: return 1 with open(asic_conf_file_path) as asic_conf_file: for line in asic_conf_file: diff --git a/src/sonic-config-engine/tests/multi_npu_data/sample-minigraph.xml b/src/sonic-config-engine/tests/multi_npu_data/sample-minigraph.xml index 8ca17925c6ec..412262315b4e 100644 --- a/src/sonic-config-engine/tests/multi_npu_data/sample-minigraph.xml +++ b/src/sonic-config-engine/tests/multi_npu_data/sample-minigraph.xml @@ -387,12 +387,20 @@ LoopbackInterface HostIP - Loopback0 + Loopback4096 8.0.0.0/32 8.0.0.0/32 + + HostIP1 + Loopback4096 + + FD00:1::32/128 + + FD00:1::32/128 + @@ -457,12 +465,20 @@ LoopbackInterface HostIP - Loopback0 + Loopback4096 8.0.0.1/32 8.0.0.1/32 + + HostIP1 + Loopback4096 + + FD00:2::32/128 + + FD00:2::32/128 + @@ -526,12 +542,20 @@ LoopbackInterface HostIP - Loopback0 + Loopback4096 8.0.0.4/32 8.0.0.4/32 + + HostIP1 + Loopback4096 + + FD00:3::32/128 + + FD00:3::32/128 + @@ -580,12 +604,20 @@ LoopbackInterface HostIP - Loopback0 + Loopback4096 8.0.0.5/32 8.0.0.5/32 + + HostIP1 + Loopback4096 + + FD00:4::32/128 + + FD00:4::32/128 + diff --git a/src/sonic-config-engine/tests/test_multinpu_cfggen.py b/src/sonic-config-engine/tests/test_multinpu_cfggen.py index c3307b482b79..7511ce9bcdd1 100644 --- a/src/sonic-config-engine/tests/test_multinpu_cfggen.py +++ b/src/sonic-config-engine/tests/test_multinpu_cfggen.py @@ -245,3 +245,32 @@ def test_back_end_asic_acl(self): argument = "-m {} -p {} -n asic3 --var-json \"ACL_TABLE\"".format(self.sample_graph, self.port_config[3]) output = json.loads(self.run_script(argument)) self.assertDictEqual(output, {}) + + def test_loopback_intfs(self): + argument = "-m {} --var-json \"LOOPBACK_INTERFACE\"".format(self.sample_graph) + output = json.loads(self.run_script(argument)) + self.assertDictEqual(output, {\ + "Loopback0": {}, + "Loopback0|10.1.0.32/32": {}, + "Loopback0|FC00:1::32/128": {}}) + + # The asic configuration should have 2 loopback interfaces + argument = "-m {} -n asic0 --var-json \"LOOPBACK_INTERFACE\"".format(self.sample_graph) + output = json.loads(self.run_script(argument)) + self.assertDictEqual(output, { \ + "Loopback0": {}, + "Loopback4096": {}, + "Loopback0|10.1.0.32/32": {}, + "Loopback0|FC00:1::32/128": {}, + "Loopback4096|8.0.0.0/32": {}, + "Loopback4096|FD00:1::32/128": {}}) + + argument = "-m {} -n asic3 --var-json \"LOOPBACK_INTERFACE\"".format(self.sample_graph) + output = json.loads(self.run_script(argument)) + self.assertDictEqual(output, {\ + "Loopback0": {}, + "Loopback4096": {}, + "Loopback0|10.1.0.32/32": {}, + "Loopback0|FC00:1::32/128": {}, + "Loopback4096|8.0.0.5/32": {}, + "Loopback4096|FD00:4::32/128": {}}) diff --git a/src/sonic-platform-common b/src/sonic-platform-common index 5d7e8aad7e4c..7ba711d7a530 160000 --- a/src/sonic-platform-common +++ b/src/sonic-platform-common @@ -1 +1 @@ -Subproject commit 5d7e8aad7e4c5220206613c0fda04070502628d5 +Subproject commit 7ba711d7a530e6c1d7841a92ae049fd69ca5b4fe diff --git a/src/sonic-platform-daemons b/src/sonic-platform-daemons index 74ffbd6c115c..afd16ceb0d1e 160000 --- a/src/sonic-platform-daemons +++ b/src/sonic-platform-daemons @@ -1 +1 @@ -Subproject commit 74ffbd6c115c345d59909189604e3c02fb72edf3 +Subproject commit afd16ceb0d1eb09d0e72bdf31d21b13636ca1c41 diff --git a/src/sonic-py-swsssdk b/src/sonic-py-swsssdk index 1e7573229b51..51c9d1fe2d6d 160000 --- a/src/sonic-py-swsssdk +++ b/src/sonic-py-swsssdk @@ -1 +1 @@ -Subproject commit 1e7573229b518a6050d903c90db31f7fbff20a68 +Subproject commit 51c9d1fe2d6dba6349c7beb959c4cdd0b89928ed diff --git a/src/sonic-snmpagent b/src/sonic-snmpagent index 862e51ab85d4..6429835af558 160000 --- a/src/sonic-snmpagent +++ b/src/sonic-snmpagent @@ -1 +1 @@ -Subproject commit 862e51ab85d48290082adfcbb801bfbbe3a95bf3 +Subproject commit 6429835af558c26afb8750aa630ab2d0eb8b1c14 diff --git a/src/sonic-swss b/src/sonic-swss index a153622725d4..7ecc849e6dda 160000 --- a/src/sonic-swss +++ b/src/sonic-swss @@ -1 +1 @@ -Subproject commit a153622725d4518155a24d2807c31dbcdc85d317 +Subproject commit 7ecc849e6ddaa3dd43caaa8d3b8992ad2efa7156 diff --git a/src/sonic-swss-common b/src/sonic-swss-common index 9a1e2b727cf2..8102f7ff8aaf 160000 --- a/src/sonic-swss-common +++ b/src/sonic-swss-common @@ -1 +1 @@ -Subproject commit 9a1e2b727cf208c70fa96562757b41aee433f797 +Subproject commit 8102f7ff8aaf4102fe90195f5230544d6b13b4f7 diff --git a/src/sonic-utilities b/src/sonic-utilities index 1149792b054d..7f02429f0da6 160000 --- a/src/sonic-utilities +++ b/src/sonic-utilities @@ -1 +1 @@ -Subproject commit 1149792b054d3cf37f0fb034473db7f1b59c8380 +Subproject commit 7f02429f0da63b3b4ebea6e06fe696103f76de31