Multi ASIC HLD #644

arlakshm · 2020-07-01T20:41:02Z

No description provided.

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>

Signed-off-by: arlakshm <55814491+arlakshm@users.noreply.github.com>

dzhangalibaba · 2020-07-01T21:43:31Z

doc/multi_asic/SONiC_multi_asic_hld.md

+ 192.168.199.193 proto 186 src 172.16.132.64 metric 20
+ nexthop via 10.0.107.12 dev PortChannel4007 weight 1
+```
+ - A packet with destination ip as 192.168.199.193


confused here, could you review this part and make the name/ip/asicX correct? Looks the description and the picture are not matching.

Signed-off-by: arlakshm <55814491+arlakshm@users.noreply.github.com>

dev-aviz · 2020-07-15T02:38:32Z

doc/multi_asic/SONiC_multi_asic_hld.md

+
+#### 2.3.2.1. BGP on backend ASICs
+ In this approach:
+ - BGP instance is running on all the ASICs.


Don't this proposal consume more CPU and memory resources?

dev-aviz · 2020-07-15T02:40:42Z

doc/multi_asic/SONiC_multi_asic_hld.md

+ ```
+ (( N * (N-1))/2) * M 
+ ```
+#### 2.3.2.3. Comparison of both approaches


Which proposal does this design consider to implemement? or do you provide both?

dev-aviz · 2020-07-15T02:42:07Z

doc/multi_asic/SONiC_multi_asic_hld.md

+This creates a separate linux network stack including routes and network devices for every ASIC.
+The interfaces for a given ASIC is linked to its corresponding namespace.
+
+In a multi-ASIC system, very commonly the ASICs are physically connected in a clos fabric topology. With sonic container dockers running as separate namespaces for each ASIC in a multi-ASIC system, we can model and configure the system as if there is a spine-leaf network topology within the box.


How to configure multi-asic ? does it have a CLI? how does it configure different ASICs

dev-aviz · 2020-07-15T02:42:38Z

doc/multi_asic/SONiC_multi_asic_hld.md

+This creates a separate linux network stack including routes and network devices for every ASIC.
+The interfaces for a given ASIC is linked to its corresponding namespace.
+
+In a multi-ASIC system, very commonly the ASICs are physically connected in a clos fabric topology. With sonic container dockers running as separate namespaces for each ASIC in a multi-ASIC system, we can model and configure the system as if there is a spine-leaf network topology within the box.


Multi-ASIC ?? Does it support multi vendor integration ? ex: MLNX & BRCM etc.

dev-aviz · 2020-07-15T02:44:36Z

doc/multi_asic/SONiC_multi_asic_hld.md

+- Introduced ASIC element type for device internal chip
+- There is an internal view from the minigraph file. It gives chipset, internal connections and internal routing logic setup for SONiC multi-asic devices.
+
+ In a Multi ASIC platform, configuration is generated per ASIC.


How does this design handle warm-reboot?

dev-aviz · 2020-07-15T02:46:18Z

doc/multi_asic/SONiC_multi_asic_hld.md

+ - Backend ASIC, which have only internal links. These ASICs only connect to other Frontend ASICs. In a chassis system, the fabric ASIC is considered as a Backend ASIC.
+
+### 2.3.2. Control Plane
+ Within a Multi ASIC SONiC system, internal control plane can be setup in following ways


Do you have any scaling numbers ? do you see any perfromance (latency & memory foot print) issues?

kannankvs

Where is the requirement/design for "downward compatibility" for single ASIC platforms explained? For example, if there is a platform with single ASIC with single "config_db.json" (and frr.conf) running previous version of SONiC and when it is upgraded with the new version that supports multi-asic, can we assume that no changes are required for such platforms? Same thing applies for all config and show commands that might have been heavily used in automation.

msreddypaluru · 2020-07-21T15:45:41Z

doc/multi_asic/SONiC_multi_asic_hld.md

+
+#### 2.4.3.1. Minigraph 
+
+minigraph.xml is used to generate the initial configuration of a multi asic platform. There is a single minigraph.xml consisting of internal ASIC connectivity view, where each ASIC is modelled as a device.


Is minigraph mandatory to configure multi-asic?

msreddypaluru · 2020-07-21T15:48:44Z

doc/multi_asic/SONiC_multi_asic_hld.md

+ - The database container running in the linux host will be called _globalDB_ container in a multi asic platform. The database tables populated here will be system wide attributes like SYSLOG, AAA, TACACS, Mgmt interface and resources like fan, psu, thermal.
+ - The ***database{n}*** container will be started in each namespace. _n_ here denotes the ASIC_ID, These databases will be used and updated by applications in that namespace.
+
+Each database container will have it's own **"/var/run/redis{n}/"** directory which contains the database_config.json file, redis unix socket etc. The database_config.json will no longer be a static build time file.


As SONiC point of view, why don't use single database container to configure multi-asic ? what is the rational behind several database containers to configure multi-asic system?

msreddypaluru · 2020-07-21T15:50:13Z

doc/multi_asic/SONiC_multi_asic_hld.md

+}
+
+```
+**Sample database_config.json for ASIC0**


Does the design consider multi db support? is this template consider multi db instances ?

rck-innovium · 2020-07-21T16:05:01Z

doc/multi_asic/SONiC_multi_asic_hld.md

+ <neighbor_ip> applies to either internal/external BGP sessions as per the user input.
+ * config interface : Added an optional argument to specify the namespace [ -n namespace ]. In Multi-ASIC devices the namespace could either be taken as a user input or if not provided will be derived based on the interface name. 
+ * config vlan : Added an optional argument to specify the namespace [ -n namespace ]. In Multi-ASIC devices namespace parameter is mandatory for (add/del) of vlan and member interface.
+ * config portchannel : Added an optional argument to specify the namespace [ -n namespace ]. In Multi-ASIC devices namesapce parameter is mandatory for (add/del) of portchannel and member interface.


Does all members of a portchannel need to be from the same ASIC? i.e. can a port-channel have member ports from ASIC0 and ASIC1?
Please document it.

rck-innovium · 2020-07-21T16:09:40Z

doc/multi_asic/SONiC_multi_asic_hld.md

+ * config bgp : No additional namespace argument added to bgp commands. The bgp startup/shutdown all commands are applied on the external bgp sessions ( where BGP neighbors are external routers ). The commands like bgp <startup/shutdown/remove>
+ <neighbor_ip> applies to either internal/external BGP sessions as per the user input.
+ * config interface : Added an optional argument to specify the namespace [ -n namespace ]. In Multi-ASIC devices the namespace could either be taken as a user input or if not provided will be derived based on the interface name. 
+ * config vlan : Added an optional argument to specify the namespace [ -n namespace ]. In Multi-ASIC devices namespace parameter is mandatory for (add/del) of vlan and member interface.


Can the VLAN member ports of a given VLAN be from different ASICs?

If yes, then, say we need to do an ARP request on the VLAN interface-- how will this be achieved?

Ext representing internal or external port respectively. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

PR sonic-net/SONiC#644 introduced the HLD to support multi ASIC. In the future, multi DUT or Chassis will be supported by SONiC as well. The test infrastructure and some of the customized ansible modules need to be updated to support testing of the upcoming new architectures. This PR is implementation of PR 2347 which tried to propose how to improve the current test infrastructure to support multi-DUT and multi-ASIC systems. The target is to ensure that the existing test scripts are not broken and we can update the tests in incremental way. This change is the implementation of PR 2347 - Add proposal for multi-DUT and multi-ASIC testing support - Added the classes described in the PR: - SonicAsic - represents an asic, and implements the asic/namespace related operations to hide the complexity of handling the asic/namespace specific details. - For now, have added bgp_facts as an example to add 'instance_id' to the bgp_facts module call on a SonicHost. - MutliAsicSonicHost - a host with one or more SonicAsics. - DutHosts - represents all the DUT's in a testbed. - has 'nodes' list to represent each DUT in the testbed. - Update duthosts fixture to return an instance of DutHosts instead of a list of SonicHosts - Modify duthost fixture to return a MultiAsicSonicHost from duthosts.nodes

arlakshm and others added 5 commits June 26, 2020 12:46

Multi ASIC HLD

99d30fd

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>

fix snmp table

39e3331

Signed-off-by: Arvindsrinivasan Lakshmi Narasimhan <arlakshm@microsoft.com>

minor edits

5c0eba5

Signed-off-by: arlakshm <55814491+arlakshm@users.noreply.github.com>

some more minor edits

7dc2161

Signed-off-by: arlakshm <55814491+arlakshm@users.noreply.github.com>

some more minor edits

d911de9

Signed-off-by: arlakshm <55814491+arlakshm@users.noreply.github.com>

dzhangalibaba reviewed Jul 1, 2020

View reviewed changes

rlhui and others added 2 commits July 1, 2020 23:35

Update architecture doc

ebe4f4b

some more minor edits

2f32043

Signed-off-by: arlakshm <55814491+arlakshm@users.noreply.github.com>

dev-aviz reviewed Jul 15, 2020

View reviewed changes

kannankvs reviewed Jul 21, 2020

View reviewed changes

msreddypaluru reviewed Jul 21, 2020

View reviewed changes

rck-innovium reviewed Jul 21, 2020

View reviewed changes

judyjoseph mentioned this pull request Aug 3, 2020

PMON in multi-asic platforms - an approach note. #655

Merged

wangxin mentioned this pull request Sep 16, 2020

[multi-asic]: Add multi-asic virtual switch testbed sonic-net/sonic-mgmt#2168

Merged

3 tasks

Update value of role in port_config.ini. Role can be Int or

d10dbb9

Ext representing internal or external port respectively. Signed-off-by: SuvarnaMeenakshi <sumeenak@microsoft.com>

sanmalho-git mentioned this pull request Oct 27, 2020

Implementation of multi-DUT and multi-ASIC as per PR 2347 sonic-net/sonic-mgmt#2417

Merged

3 tasks

rlhui approved these changes Dec 15, 2020

View reviewed changes

rlhui merged commit 2f89785 into sonic-net:master Dec 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi ASIC HLD #644

Multi ASIC HLD #644

arlakshm commented Jul 1, 2020

dzhangalibaba Jul 1, 2020

dev-aviz Jul 15, 2020

dev-aviz Jul 15, 2020

dev-aviz Jul 15, 2020

dev-aviz Jul 15, 2020

dev-aviz Jul 15, 2020

dev-aviz Jul 15, 2020

kannankvs left a comment

msreddypaluru Jul 21, 2020

msreddypaluru Jul 21, 2020

msreddypaluru Jul 21, 2020

rck-innovium Jul 21, 2020

rck-innovium Jul 21, 2020


		#### 2.4.3.1. Minigraph

		minigraph.xml is used to generate the initial configuration of a multi asic platform. There is a single minigraph.xml consisting of internal ASIC connectivity view, where each ASIC is modelled as a device.

Multi ASIC HLD #644

Multi ASIC HLD #644

Conversation

arlakshm commented Jul 1, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kannankvs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment