[multi-dut] Support for gen-mg to work with multi-dut where VM's conn…

…ect to only a single host and supervisor card (#2700)

What is the motivation for this PR?
For SONiC Chassis support gen-mg has the following limitations:
* for multi-dut it assumes each VM is connected to all the DUT's (similar to the dualtor topology)
* assumes that all interfaces connected to a PortChannel are sequential.

It also needs the following enhancements:
* In a SONiC Chassis we have a 'supervisor' card which has no ports and thus no BGP/loopback/IpInterfaces/ACL's etc. The minigraph should not include info about these.
* We need to introduce 'T3' router type as 'CoreRouter'

How did you do it?
For VMs to connect to a single DUT:
* In vm_topo_config, the fields like interface_indexes, intfs, ip_intf etc would be either empty or null. Add checks for these where they are being used in gen-mg (config_sonic_basedon_testbed.yml and the minigraph templates).

For not all PortChannel ports being sequential:
* In config_sonic_based_on_testbed.yml, changed interface_to_vms structure to be compatable to allow using ansible's 'with_subelements' loop to get the intf_names

For SONiC Chassis support:
* For 'supervisor' card, there are no BGP peers, interfaces etc. required.
  * A 'supervisor' card is identified by having 'type' field in the inventory with value 'supervisor'.
  * Add this check to skip generation of portions not valid for supervisor card
* Added 'CoreRouter' as the DUT type for T3 VM's that connect to 'T2' DUT.

How did you verify/test it?
* Ran gen-mg against all the testbeds defined in vtestbed.csv with latest github master code, and then with our code changes. Verified that there are no differences in the generated minigraph files.
* Ran against topo_t2.yml proposed in PR #2638 where we have 3 linecards with 24 VM's each (8 on 2 port LAG and 16 on single port) and 1 supervisor card.
  * Validated generated minigraph files for the linecards are the same as those generated by running gen-mg against individual linecard testbed (with only 1 DUT).
  * Validate the generated minigraph file for supervisor card is acceptable to sonic-cfggen and the generated config_db.json is valid for the supervisor card.

ansible/config_sonic_basedon_testbed.yml

@@ -134,7 +134,7 @@
 
   - name: find all interface indexes mapping connecting to VM
     set_fact:
-      interface_to_vms: "{{ interface_to_vms|default({}) | combine({ item.key: item.value['interface_indexes'][dut_index|int] }) }}"
+      interface_to_vms: "{{ interface_to_vms|default([]) + [ {'name': item.key, 'ports': item.value['interface_indexes'][dut_index|int] }] }}"
     with_dict: "{{ vm_topo_config['vm'] }}"
 
   - name: find all interface indexes connecting to VM
@@ -144,8 +144,10 @@
 
   - name: find all interface names
     set_fact:
-      intf_names: "{{ intf_names | default({}) | combine({item.key: port_alias[item.value[0]|int:item.value[-1]|int+1] }) }}"
-    with_dict: "{{ interface_to_vms }}"
+      intf_names: "{{ intf_names | default({}) | combine({item.0.name:  intf_names[item.0.name]|default([]) + [ port_alias[item.1]] }) }}"
+    with_subelements:
+      - "{{ interface_to_vms }}"
+      - "ports"
 
   - name: create minigraph file in ansible minigraph folder
     template: src=templates/minigraph_template.j2

ansible/library/topo_facts.py

@@ -153,15 +153,19 @@ def get_topo_config(self, topo_name):
                     ip = ipaddress.ip_address(ipstr.decode('utf8'))
                     for dut_index in range(0, dut_num):
                         if ip.version == 4:
-                            ipsubnet_str = vmconfig[vm]['peer_ipv4'][dut_index]+'/'+vmconfig[vm]['ipv4mask'][dut_index]
-                            ipsubnet = ipaddress.ip_interface(ipsubnet_str.decode('utf8'))
-                            if ip in ipsubnet.network:
-                                vmconfig[vm]['bgp_ipv4'][dut_index] = ipstr.upper()
+                            # Each VM might not be connected to all the DUT's, so check if this VM is a peer to DUT at dut_index
+                            if vmconfig[vm]['peer_ipv4'][dut_index]:
+                                ipsubnet_str = vmconfig[vm]['peer_ipv4'][dut_index]+'/'+vmconfig[vm]['ipv4mask'][dut_index]
+                                ipsubnet = ipaddress.ip_interface(ipsubnet_str.decode('utf8'))
+                                if ip in ipsubnet.network:
+                                    vmconfig[vm]['bgp_ipv4'][dut_index] = ipstr.upper()
                         elif ip.version == 6:
-                            ipsubnet_str = vmconfig[vm]['peer_ipv6'][dut_index]+'/'+vmconfig[vm]['ipv6mask'][dut_index]
-                            ipsubnet = ipaddress.ip_interface(ipsubnet_str.decode('utf8'))
-                            if ip in ipsubnet.network:
-                                vmconfig[vm]['bgp_ipv6'][dut_index] = ipstr.upper()
+                            # Each VM might not be connected to all the DUT's, so check if this VM is a peer to DUT at dut_index
+                            if vmconfig[vm]['peer_ipv6'][dut_index]:
+                                ipsubnet_str = vmconfig[vm]['peer_ipv6'][dut_index]+'/'+vmconfig[vm]['ipv6mask'][dut_index]
+                                ipsubnet = ipaddress.ip_interface(ipsubnet_str.decode('utf8'))
+                                if ip in ipsubnet.network:
+                                    vmconfig[vm]['bgp_ipv6'][dut_index] = ipstr.upper()
 
             vm_topo_config['vm'] = vmconfig
 

ansible/templates/minigraph_cpg.j2

@@ -29,17 +29,20 @@
 {% endfor %}
     </PeeringSessions>
     <Routers xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution">
+{% if type is not defined or type != 'supervisor' %}
       <a:BGPRouterDeclaration>
         <a:ASN>{{ vm_topo_config['dut_asn'] }}</a:ASN>
         <a:Hostname>{{ inventory_hostname }}</a:Hostname>
         <a:Peers>
 {% for index in range(vms_number)  %}
+{% if vm_topo_config['vm'][vms[index]]['peer_ipv4'][dut_index|int] %}
           <BGPPeer>
             <Address>{{ vm_topo_config['vm'][vms[index]]['peer_ipv4'][dut_index|int] }}</Address>
             <RouteMapIn i:nil="true"/>
             <RouteMapOut i:nil="true"/>
             <Vrf i:nil="true"/>
           </BGPPeer>
+{% endif %}
 {% endfor %}
 {% if 'tor' in vm_topo_config['dut_type'] | lower %}
           <BGPPeer i:type="a:BGPPeerPassive">
@@ -65,12 +68,15 @@
         <a:RouteMaps/>
       </a:BGPRouterDeclaration>
 {% for index in range( vms_number) %}
+{% if vm_topo_config['vm'][vms[index]]['intfs'][dut_index|int]|length > 0 %}
       <a:BGPRouterDeclaration>
         <a:ASN>{{ vm_topo_config['vm'][vms[index]]['bgp_asn'] }}</a:ASN>
         <a:Hostname>{{ vms[index] }}</a:Hostname>
         <a:RouteMaps/>
       </a:BGPRouterDeclaration>
+{% endif %}
 {% endfor %}
+{% endif %}
     </Routers>
   </CpgDec>
 

ansible/templates/minigraph_device.j2

@@ -2,6 +2,7 @@
     <DeviceInfo>
       <AutoNegotiation>true</AutoNegotiation>
       <EthernetInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution">
+{% if type is not defined or type != 'supervisor' %}
 {% set num_of_intf = port_alias | length %}
 {% for index in range(num_of_intf) %}
         <a:EthernetInterface>
@@ -23,6 +24,7 @@
 {% endif %}
         </a:EthernetInterface>
 {% endfor %}
+{% endif %}
       </EthernetInterfaces>
       <FlowControl>true</FlowControl>
       <Height>0</Height>

ansible/templates/minigraph_dpg.j2

@@ -2,6 +2,7 @@
     <DeviceDataPlaneInfo>
       <IPSecTunnels/>
       <LoopbackIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution">
+{% if type is not defined or type != 'supervisor' %}
         <a:LoopbackIPInterface>
           <Name>HostIP</Name>
           <AttachTo>Loopback0</AttachTo>
@@ -39,6 +40,7 @@
         </a:LoopbackIPInterface>
       {%- endfor -%}
       {%- endif -%}
+{% endif %}
       </LoopbackIPInterfaces>
       <ManagementIPInterfaces xmlns:a="http://schemas.datacontract.org/2004/07/Microsoft.Search.Autopilot.Evolution">
         <a:ManagementIPInterface>
@@ -106,9 +108,10 @@
       </VlanInterfaces>
       <IPInterfaces>
 {% for index in range(vms_number) %}
+{% if vm_topo_config['vm'][vms[index]]['ip_intf'][dut_index|int] is not none %}
         <IPInterface>
           <Name i:nil="true"/>
-{% if 'port-channel' in vm_topo_config['vm'][vms[index]]['ip_intf']|lower %}
+{% if 'port-channel' in vm_topo_config['vm'][vms[index]]['ip_intf'][dut_index|int]|lower %}
           <AttachTo>PortChannel{{ ((index+1) |string).zfill(4) }}</AttachTo>
 {% else %}
           <AttachTo>{{ port_alias[vm_topo_config['vm'][vms[index]]['interface_indexes'][dut_index|int][0]] }}</AttachTo>
@@ -117,13 +120,14 @@
         </IPInterface>
         <IPInterface>
           <Name i:Name="true"/>
-{% if 'port-channel' in (vm_topo_config['vm'][vms[index]]['ip_intf']|lower) %}
+{% if 'port-channel' in vm_topo_config['vm'][vms[index]]['ip_intf'][dut_index|int]|lower %}
           <AttachTo>PortChannel{{ ((index+1) |string).zfill(4) }}</AttachTo>
 {% else %}
           <AttachTo>{{ port_alias[vm_topo_config['vm'][vms[index]]['interface_indexes'][dut_index|int][0]] }}</AttachTo>
 {% endif %}
           <Prefix>{{ vm_topo_config['vm'][vms[index]]['bgp_ipv6'][dut_index|int] }}/{{ vm_topo_config['vm'][vms[index]]['ipv6mask'][dut_index|int] }}</Prefix>
         </IPInterface>
+{% endif %}
 {% endfor %}
 {% if 'tor' in vm_topo_config['dut_type'] | lower %}
 {% for vlan, vlan_param in vlan_configs.items() %}
@@ -146,6 +150,7 @@
       </IPInterfaces>
       <DataAcls/>
       <AclInterfaces>
+{% if type is not defined or type != 'supervisor' %}
         <AclInterface>
           <InAcl>SNMP_ACL</InAcl>
           <AttachTo>SNMP</AttachTo>
@@ -169,20 +174,27 @@
 {% if enable_data_plane_acl|default('true')|bool %}
         <AclInterface>
           <AttachTo>
+{%- set acl_intfs = [] -%}
 {%- for index in range(vms_number) %}
 {% if 'port-channel' in vm_topo_config['vm'][vms[index]]['ip_intf'][dut_index|int]|lower %}
-PortChannel{{ ((index+1) |string).zfill(4) }}{% if not loop.last %};{% endif %}
+{% set a_intf = 'PortChannel' + ((index+1) |string).zfill(4) %}
+{{- acl_intfs.append(a_intf) -}}
 {% endif %}
 {% endfor %}
 {% for index in range(vms_number) %}
 {% if 'port-channel' not in vm_topo_config['vm'][vms[index]]['ip_intf']|lower %}
-{{ port_alias[vm_topo_config['vm'][vms[index]]['interface_indexes'][dut_index|int][0]] }}{% if not loop.last %};{% endif %}
+{% if vm_topo_config['vm'][vms[index]]['intfs'][dut_index|int]|length %}
+{% set a_intf = port_alias[vm_topo_config['vm'][vms[index]]['interface_indexes'][dut_index|int][0]] %}
+{{- acl_intfs.append(a_intf) -}}
+{% endif %}
 {% endif %}
 {% endfor -%}
+{{- acl_intfs|join(';') -}}
           </AttachTo>
           <InAcl>DataAcl</InAcl>
           <Type>DataPlane</Type>
         </AclInterface>
+{% endif %}
 {% endif %}
       </AclInterfaces>
       <DownstreamSummaries/>

ansible/templates/minigraph_png.j2

@@ -113,10 +113,13 @@
 {% endif %}
 {% if VM_topo %}
 {% for dev in neighbor_eosvm_mgmt %}
+{% if vm_topo_config['vm'][dev]['intfs'][dut_index|int]|length %}
 {% if 'T1' in dev %}
 {% set dev_type = 'LeafRouter' %}
 {% elif 'T2' in dev %}
 {% set dev_type = 'SpineRouter' %}
+{% elif 'T3' in dev %}
+{% set dev_type = 'CoreRouter' %}
 {% elif 'T0' in dev %}
 {% set dev_type = 'ToRRouter' %}
 {% else %}
@@ -129,6 +132,7 @@
          </ManagementAddress>
          <HwSku>Arista-VM</HwSku>
       </Device>
+{% endif %}
 {% endfor %}
 {% endif %}
     </Devices>

ansible/templates/minigraph_template.j2

@@ -2,10 +2,14 @@
 {% set vms=vm_topo_config['vm'].keys() | sort %}
 {% set vms_number = vms | length %}
 {% if 'loopback' in vm_topo_config['DUT'] %}
+{% if type is not defined or type != 'supervisor' %}
 {% set lp_ipv4 = vm_topo_config['DUT']['loopback']['ipv4'][dut_index|int] %}
 {% set lp_ipv4_addr = lp_ipv4.split('/')[0] %}
+{% endif %}
+{% if type is not defined or type != 'supervisor' %}
 {% set lp_ipv6 = vm_topo_config['DUT']['loopback']['ipv6'][dut_index|int] %}
 {% set lp_ipv6_addr = lp_ipv6.split('/')[0] %}
+{% endif %}
 {% else %}
 {% set lp_ipv4 = '10.1.0.32/32' %}
 {% set lp_ipv4_addr = '10.1.0.32' %}