Merge branch 'github-1803' --5/12

[installer]: Suppress tar xz warning about time stamp in the future, if date is not correctly set (sonic-net#1562)

\[sonic-platform-common\] Update submodule (sonic-net#1563)

\- Includes the following commits:
    \- \[bcmshell.py\] Match extra whitespace before prompt in regex (#3)
    \- add support for qsfp28 eeprom (#2)
\[baseimage\]: bring down eth0 before restart networking (sonic-net#1555)

cfggen generates new eth0 configuration. Need to first
clean existing configuration on eth0 before bring up
new configuration on eth0. Thus, we need to first bring
down eth0 before putting new configuration into /etc/network/
interfaces

\[mellanox\]: Update MLNX SAI pointer (sonic-net#1557)

\[minigraph.py\] Add support to parse tacacs server information (sonic-net#1549)

\* \[minigraph.py\] Add support to parse tacacs server information
\[router advertiser\] Only start radvd process if device role is 'ToRRouter' (sonic-net#1569)

\[submodules\]: update sonic-swss (sonic-net#1570)

\[submodules\]: update sonic-utilities (sonic-net#1571)

\[cfggen\]: ignore acl when its type is not defined (sonic-net#1568)

\[installer\]: Umount before delete partition (sonic-net#1575)

Use eth0 interface only to generate lldpd SystemId (sonic-net#1577)

Allow one Service ACL to bind to multiple services (sonic-net#1576)

\* \[caclmgrd\] Also ignore IP protocol if found in rule; we will only use our predefined protocols
\[snmp\]: Bind snmpd to all ip addresses (sonic-net#1587)

\[device\] Update Arista driver submodule (sonic-net#1585)

Watchdog timeout increased
\[devices\]: Fix type for qos.json in 7060 and S6100 (sonic-net#1582)

\[minigraph\]: ignore minigraph ports which are not in port_config.ini (sonic-net#1593)

\[minigraph\] Fix parser on PNG DeviceInterfaceLink Bandwidth (sonic-net#1592)

\* \[minigraph\] Fix parser on PNG DeviceInterfaceLink Bandwidth

\[Broadcom SAI\] upgrade Broadcom SAI to version 3.1.3.4-10 (sonic-net#1591)

\* \[Broadcom SAI\] upgrade Broadcom SAI to version 3.1.3.4-9

Includes configuration files for following devices:

\- Quanta 1X1B-32X
\- Dell Z9264F
\- Inventec D7054Q28B and D7032Q28B

\* \[bcm sai\] upgrade sai version to 3.1.3.4-10

include configuration change to 7060 T0.

50G support for Arista 7060 (sonic-net#1580)

\* 50G SKU for Arista 7060

Marvell's updates for SONiC 201803 over SAI v1.2 (sonic-net#1588)

\[Mellanox\] Add support for a new platform LS-SN2700

\[devices\]: Merge ingress service pools of lossless and lossy traffic for TD2 (sonic-net#1578)

\[sonic-utilities\] add pfcstat and queuestat tool (sonic-net#1606)

Add support for S6100 switchport LEDs (sonic-net#1610)

\[ip-in-ip\]: Fix config template to apply correct platform depended values (sonic-net#1619)

\[platform-common\]: Update sonic-platform-common submodule (sonic-net#1620)

\[sfputilbase\]: Add logic to parse the title of port_config.ini file

\[sonic-cfggen\] Be case insensitive to hostname in minigraph (sonic-net#1614)

\[bugfix\]: pass correct port name to led_control.py in ledd

\[cfggen\]: Fix build by fixing pyangbind version (sonic-net#1633)

\[swss\]: update sonic-swss submodule

\* ea34b92 2018-04-24 | Fix tables handling race condition in buffermgr (sonic-net#484) (HEAD -> 201803, origin/201803) \[Andriy Moroz\]
\* 53831be 2018-04-19 | \[pfcwd\]: create PFCWD acl instead of L3 ACL (sonic-net#479) \[sihuihan88\]

\[radvd\] Ensure at least one interface is specified in radvd.conf before starting radvd (sonic-net#1636)

\[updategraph\]: Keep updategraph service active after start (sonic-net#1651)

\[docker-lldpd\]: Various fixes (sonic-net#1650)

\* We don't need configure anything until we have interfaces created

\* Don't run lldpcli for a port, until a port is up and running

\* Remove lldpd socket before starting lldpd

\* Fix sample files for lldpd configuration

\* Another attempt to make the test working

\* Quick fix for lldpd paused after start bug

\[submodules\]: update swss and utilities modules

swss:

\* e34104e 2018-04-13 | \[pfcwd\]: support BIG_RED_SWITCH mode (sonic-net#467) (HEAD, origin/201803) \[sihuihan88\]
\* 1f857d5 2018-04-25 | \[buffermgr\]: remove the item from consumer queue if invalid (sonic-net#489) \[sihuihan88\]

utilities:

\* 0b9bb2b 2018-04-26 | Stop services before pushing new config during "load_minigraph" (sonic-net#247) (HEAD, origin/201803) \[Prince Sunny\]
\* dc119c9 2018-04-18 | \[show logging\] For following, change 'tail -f' to 'tail -F' in order to retry in the case log is rotated (sonic-net#240) \[Joe LeVeque\]
\* 08da428 2018-04-16 | \[pfcwd\]: add cli to enable/disable BIG_RED_SWITCH mode (sonic-net#237) \[sihuihan88\]

\[snmp\]: Fix a race between snmpd-config-updater and snmpd (sonic-net#1628)

There is a small window in which snmpd might not have registered a
callback for SIGHUP and which will result in its death if
snmpd-config-updater send this signal meant for a config reload.
\[snmpd\]: Fix typo in is_platform_arista (sonic-net#1634)

\[mellanox\]: Update SAI version to 1.11.4 and SDK to 4.2.7303 (sonic-net#1655)

\[docker-dhcp\]: Fix the sonic build issue (sonic-net#1659)

Install the built version of isc-dhcp-client in docker-dhcp-relay
\[swss\]: update swss

118b3f0 2018-05-01 | Populate existing interface cache, bring down before configDone

\[zebra.conf\] Fix template issue with multiple lo addresses (sonic-net#1662)

\* \[zebra.conf\] Fix template issue with multiple lo addresses

\* Add unitest for Loopback1

\[swss\]: Change the hash seed to 0 for ToR and 10 for Leaf routers (sonic-net#1667)

Due to some ASIC platform limitations, the hash seed range is from 0
to 15. Thus the switch.json.j2 template is updated so that ToRRouter
is using hash seed 0 and LeafRouter is using hash seed 10.

\[snmp\]: Stop spamming logs with statfs permission denied log message (sonic-net#1668)

\[broadcom\]: update broadcom SAI to 3.1.3.4-11 (sonic-net#1670)

Provide better ECMP load-balancing via hash seed

\[sonic-cfggen\]: fix bgpd and zebra template for sonic-cfggen test

I took the original patch (bebb7a0) into 201803 branch need to also
adapt the patch since we do not have commit (d423841) in 201803 branch.

\[swss\]: update sonic-swss module

\[201803 d57f9a1\] \[lua\]: use not to check whether the field exists (sonic-net#492)

\[device\] Update arista driver submodule (sonic-net#1674)

\[submodule\]: Update submodule sonic-snmpagent (sonic-net#1642)

sonic-cfggen supports hwsku parameter (sonic-net#1631) \*Note\*: tuned test data during cherry-pick

\[device\] Add PSU utility for platform ly1200 of MiTAC (sonic-net#1673)

\[platform/broadcom/sonic-platform-modules-mitac\] Install acpi package for daemon and adjust i2c sequence
\[mellanox\]: Update MLNX SAI pointer (sonic-net#1684)

40G profile for Arista 7060 (sonic-net#1677)

Update buffers config for Mellanox 27xx devices (sonic-net#1649)

\* Update buffers config for Mellanox 27xx devices

\* Remove buffers template test for msn27xx

\[submodule\]: Update submodule sonic-snmpagent: Improve mib fundamental classes (sonic-net#1689)

\[sonic-utilities\]: update sonic utilities submodule

\* 951633b 2018-05-04 |  \[generate_dump\]: fix a saidump file copy bug (sonic-net#248) (HEAD, origin/201803) \[Kebo Liu\]
\* 69baff7 2018-05-03 | \[acl_loader\]: Missing one colon (sonic-net#252) \[Shuotian Cheng\]
\* 557248d 2018-05-02 | \[acl-loader\]: Add --table_name option to update full operation (sonic-net#249) \[Shuotian Cheng\]
\* a8aadee 2018-04-30 | \[acl-loader\]: Change the header from Rule ID to Table (sonic-net#250) \[Shuotian Cheng\]

\[swss\]: update sonic-swss

\* b57c376 2018-05-10 | \[teamsyncd\]: Add team_ifindex2ifname return value check (sonic-net#500) (HEAD, origin/201803) \[Shuotian Cheng\]
\* 236843f 2018-05-07 | Fix Crm Acl used counter update (sonic-net#496) \[Nadiya\]

\[swss\]: update sonic-swss

c374357 2018-04-23 | Fix ZeroBufferProfile parameters (sonic-net#485) (HEAD -> 201803) \[Andriy Moroz\]

\[platform\]: Fixed Cavium platform modules build. (sonic-net#1694)

\[submodule\]: Update submodule sonic-snmpagent: Improve mib fundamental classes: retry after reinit_data() throws (sonic-net#1700)

Merge branch 'github-1803'

 Conflicts:
	dockers/docker-router-advertiser/start.sh
	platform/broadcom/sai.mk
	platform/mellanox/mlnx-sai.mk
	src/sonic-config-engine/sonic-cfggen
	src/sonic-config-engine/tests/sample_output/ports.json
	src/sonic-config-engine/tests/test_cfggen.py
	src/sonic-platform-daemons
	src/sonic-snmpagent
	src/sonic-swss
	src/sonic-utilities

\[baseimage\]: Disable DAD for eth0 explicitly (sonic-net#1701)

\[quagga\]: update quagga submodule (sonic-net#1698)

\* \[quagga\]: update quagga submodule

0bc6bd6 2018-05-11 | ignore nexthop attribute when NLRI is present (#18) (HEAD, origin/debian/0.99.24.1, origin/HEAD) \[lguohan\]

\* add vs bgp test

Fix the build error

Revert "\[sonic-cfggen\]: fix bgpd and zebra template for sonic-cfggen test"

This reverts commit b29d835.

Fix the build issue for sonic-cfggen test

Merge branch 'github-1803' --5/12

 Conflicts:
	src/sonic-quagga
	src/sonic-snmpagent

    RB=1312391
    G=lnos-reviewers
    R=pchaudha,pmao,rmolina,zxu
    A=

build_debian.sh

@@ -305,6 +305,7 @@ set /files/etc/sysctl.conf/net.ipv6.conf.eth0.forwarding 0
 
 set /files/etc/sysctl.conf/net.ipv6.conf.default.accept_dad 0
 set /files/etc/sysctl.conf/net.ipv6.conf.all.accept_dad 0
+set /files/etc/sysctl.conf/net.ipv6.conf.eth0.accept_dad 0
 
 set /files/etc/sysctl.conf/net.ipv6.conf.eth0.accept_ra_defrtr 0
 

platform/vs/README.md

@@ -44,7 +44,7 @@ sw-srv0 (id: 5)
 2. Start sonic virtual switch docker
 
 ```
-$ docker run --privileged --network container:sw -d docker-sonic-vs
+$ docker run --privileged --network container:sw --name vs -d docker-sonic-vs
 ```
 
 3. Setup IP in the virtual switch docker

platform/vs/tests/README.md

@@ -0,0 +1,4 @@
+Requirements:
+
+- Enable IPv6 for docker engine
+- pip install exabgp

platform/vs/tests/bgp/files/bgpd.conf

@@ -0,0 +1,7 @@
+router bgp 65501
+ bgp router-id 1.1.1.1
+ no bgp default ipv4-unicast
+ neighbor fc00::2 remote-as 65502
+ address-family ipv6
+ neighbor fc00::2 activate
+ exit-address-family

platform/vs/tests/bgp/files/invalid_nexthop.conf

@@ -0,0 +1,19 @@
+neighbor fc00::1 {
+ router-id 1.2.3.4;
+ local-address fc00::2;
+ local-as 65502;
+ peer-as 65501;
+ group-updates false;
+
+ family {
+ ipv4 unicast;
+ ipv6 unicast;
+ }
+
+ static {
+ route 3333::0/64 {
+ next-hop 0.0.0.0;
+ next-hop fc00::2;
+ }
+ }
+}

platform/vs/tests/bgp/test_invalid_nexthop.py

@@ -0,0 +1,32 @@
+from swsscommon import swsscommon
+import os
+import re
+import time
+import json
+
+def test_InvalidNexthop(dvs):
+
+ dvs.copy_file("/etc/quagga/", "bgp/files/bgpd.conf")
+ dvs.runcmd("supervisorctl start bgpd")
+ dvs.runcmd("ip addr add fc00::1/126 dev Ethernet0")
+ dvs.runcmd("ifconfig Ethernet0 up")
+
+ dvs.servers[0].runcmd("ip addr add fc00::2/126 dev eth0")
+ dvs.servers[0].runcmd("ifconfig eth0 up")
+
+ time.sleep(5)
+
+ print dvs.runcmd("supervisorctl status")
+
+ p = dvs.servers[0].runcmd_async("exabgp -d bgp/files/invalid_nexthop.conf")
+
+ time.sleep(10)
+
+ output = dvs.runcmd(["vtysh", "-c", "show ipv6 bgp"])
+
+ p.terminate()
+ p = p.wait()
+
+ print output
+
+ assert "3333::/64" in output

platform/vs/tests/conftest.py

@@ -0,0 +1,231 @@
+import os
+import os.path
+import re
+import time
+import docker
+import pytest
+import commands
+import tarfile
+import StringIO
+import subprocess
+from swsscommon import swsscommon
+
+def pytest_addoption(parser):
+ parser.addoption("--dvsname", action="store", default=None,
+ help="dvs name")
+
+class AsicDbValidator(object):
+ def __init__(self, dvs):
+ self.adb = swsscommon.DBConnector(1, dvs.redis_sock, 0)
+
+ # get default dot1q vlan id
+ atbl = swsscommon.Table(self.adb, "ASIC_STATE:SAI_OBJECT_TYPE_VLAN")
+
+ keys = atbl.getKeys()
+ assert len(keys) == 1
+ self.default_vlan_id = keys[0]
+
+ # build port oid to front port name mapping
+ self.portoidmap = {}
+ self.portnamemap = {}
+ self.hostifoidmap = {}
+ self.hostifnamemap = {}
+ atbl = swsscommon.Table(self.adb, "ASIC_STATE:SAI_OBJECT_TYPE_HOSTIF")
+ keys = atbl.getKeys()
+
+ assert len(keys) == 32
+ for k in keys:
+ (status, fvs) = atbl.get(k)
+
+ assert status == True
+
+ for fv in fvs:
+ if fv[0] == "SAI_HOSTIF_ATTR_OBJ_ID":
+ port_oid = fv[1]
+ elif fv[0] == "SAI_HOSTIF_ATTR_NAME":
+ port_name = fv[1]
+
+ self.portoidmap[port_oid] = port_name
+ self.portnamemap[port_name] = port_oid
+ self.hostifoidmap[k] = port_name
+ self.hostifnamemap[port_name] = k
+
+ # get default acl table and acl rules
+ atbl = swsscommon.Table(self.adb, "ASIC_STATE:SAI_OBJECT_TYPE_ACL_TABLE")
+ keys = atbl.getKeys()
+
+ assert len(keys) == 1
+ self.default_acl_table = keys[0]
+
+ atbl = swsscommon.Table(self.adb, "ASIC_STATE:SAI_OBJECT_TYPE_ACL_ENTRY")
+ keys = atbl.getKeys()
+
+ assert len(keys) == 2
+ self.default_acl_entries = keys
+
+class VirtualServer(object):
+ def __init__(self, ctn_name, pid, i):
+ self.nsname = "%s-srv%d" % (ctn_name, i)
+ self.vifname = "vEthernet%d" % (i * 4)
+ self.cleanup = True
+
+ # create netns
+ if os.path.exists("/var/run/netns/%s" % self.nsname):
+ self.cleanup = False
+ else:
+ os.system("ip netns add %s" % self.nsname)
+
+ # create vpeer link
+ os.system("ip link add %s type veth peer name %s" % (self.nsname[0:12], self.vifname))
+ os.system("ip link set %s netns %s" % (self.nsname[0:12], self.nsname))
+ os.system("ip link set %s netns %d" % (self.vifname, pid))
+
+ # bring up link in the virtual server
+ os.system("ip netns exec %s ip link set dev %s name eth0" % (self.nsname, self.nsname[0:12]))
+ os.system("ip netns exec %s ip link set dev eth0 up" % (self.nsname))
+ os.system("ip netns exec %s ethtool -K eth0 tx off" % (self.nsname))
+
+ # bring up link in the virtual switch
+ os.system("nsenter -t %d -n ip link set dev %s up" % (pid, self.vifname))
+
+ def __del__(self):
+ if self.cleanup:
+ os.system("ip netns delete %s" % self.nsname)
+
+ def runcmd(self, cmd):
+ os.system("ip netns exec %s %s" % (self.nsname, cmd))
+
+ def runcmd_async(self, cmd):
+ return subprocess.Popen("ip netns exec %s %s" % (self.nsname, cmd), shell=True)
+
+class DockerVirtualSwitch(object):
+ def __init__(self, name=None):
+ self.pnames = ['fpmsyncd',
+ 'intfmgrd',
+ 'intfsyncd',
+ 'neighsyncd',
+ 'orchagent',
+ 'portsyncd',
+ 'redis-server',
+ 'rsyslogd',
+ 'syncd',
+ 'teamsyncd',
+ 'vlanmgrd',
+ 'zebra']
+ self.mount = "/var/run/redis-vs"
+ self.redis_sock = self.mount + '/' + "redis.sock"
+ self.client = docker.from_env()
+
+ self.ctn = None
+ self.cleanup = True
+ if name != None:
+ # get virtual switch container
+ for ctn in self.client.containers.list():
+ if ctn.name == name:
+ self.ctn = ctn
+ (status, output) = commands.getstatusoutput("docker inspect --format '{{.HostConfig.NetworkMode}}' %s" % name)
+ ctn_sw_id = output.split(':')[1]
+ self.cleanup = False
+ if self.ctn == None:
+ raise NameError("cannot find container %s" % name)
+
+ # get base container
+ for ctn in self.client.containers.list():
+ if ctn.id == ctn_sw_id or ctn.name == ctn_sw_id:
+ ctn_sw_name = ctn.name
+
+ (status, output) = commands.getstatusoutput("docker inspect --format '{{.State.Pid}}' %s" % ctn_sw_name)
+ self.ctn_sw_pid = int(output)
+
+ # create virtual servers
+ self.servers = []
+ for i in range(32):
+ server = VirtualServer(ctn_sw_name, self.ctn_sw_pid, i)
+ self.servers.append(server)
+
+ self.restart()
+ else:
+ self.ctn_sw = self.client.containers.run('debian:jessie', privileged=True, detach=True,
+ command="bash", stdin_open=True)
+ (status, output) = commands.getstatusoutput("docker inspect --format '{{.State.Pid}}' %s" % self.ctn_sw.name)
+ self.ctn_sw_pid = int(output)
+
+ # create virtual server
+ self.servers = []
+ for i in range(32):
+ server = VirtualServer(self.ctn_sw.name, self.ctn_sw_pid, i)
+ self.servers.append(server)
+
+ # create virtual switch container
+ self.ctn = self.client.containers.run('docker-sonic-vs', privileged=True, detach=True,
+ network_mode="container:%s" % self.ctn_sw.name,
+ volumes={ self.mount: { 'bind': '/var/run/redis', 'mode': 'rw' } })
+
+ self.ctn.exec_run("sysctl -w net.ipv6.conf.all.disable_ipv6=0")
+ self.check_ready()
+ self.init_asicdb_validator()
+
+ def destroy(self):
+ if self.cleanup:
+ self.ctn.remove(force=True)
+ self.ctn_sw.remove(force=True)
+ for s in self.servers:
+ del(s)
+
+ def check_ready(self, timeout=30):
+ '''check if all processes in the dvs is ready'''
+
+ re_space = re.compile('\s+')
+ process_status = {}
+ ready = False
+ started = 0
+ while True:
+ # get process status
+ out = self.ctn.exec_run("supervisorctl status")
+ for l in out.split('\n'):
+ fds = re_space.split(l)
+ if len(fds) < 2:
+ continue
+ process_status[fds[0]] = fds[1]
+
+ # check if all processes are running
+ ready = True
+ for pname in self.pnames:
+ try:
+ if process_status[pname] != "RUNNING":
+ ready = False
+ except KeyError:
+ ready = False
+
+ if ready == True:
+ break
+
+ started += 1
+ if started > timeout:
+ raise ValueError(out)
+
+ time.sleep(1)
+
+ def restart(self):
+ self.ctn.restart()
+
+ def init_asicdb_validator(self):
+ self.asicdb = AsicDbValidator(self)
+
+ def runcmd(self, cmd):
+ return self.ctn.exec_run(cmd)
+
+ def copy_file(self, path, filename):
+ tarstr = StringIO.StringIO()
+ tar = tarfile.open(fileobj=tarstr, mode="w")
+ tar.add(filename, os.path.basename(filename))
+ tar.close()
+ self.ctn.put_archive(path, tarstr.getvalue())
+ tarstr.close()
+
+@pytest.yield_fixture(scope="module")
+def dvs(request):
+ name = request.config.getoption("--dvsname")
+ dvs = DockerVirtualSwitch(name)
+ yield dvs
+ dvs.destroy()