NAS L3

The NAS routing subsystem provides functionality for Layer 3 routing, and database for forwarding data to program routing information into the NPU via the SAI and SDKs.

NAS L3 routing components include:

• Northbound interface with CPS APIs
• Core NAS L3 routing functionality
• L3 network device interface (NDI) layer in lower HAL
• Southbound interface to SAI APIs

NAS routing functionality

• Handles the generic routing forwarding information base (FIB)
• Maintains direct routes, host routes, and recursive routes and their dependencies
• Resolves routes based on ARP resolution, host routes, and recursive dependencies
• Adjusts the FIB actions for ACTION_DROP, TRAPCPU, and ACTION_FORWARD-based route resolution
• Layer 3 ECMP load balancing (max: 64 paths, def: 16 paths) along with Layer 2 weighted ECMP provided by other modules.
• Provides route and next-hop reachability information to other interested applications
• Interfaces with the NDI layer
• Handles the CPS object messages for get/set, pub/sub from the CPS interface for routes and next-hop information, ARP/NDP information from user applications, and provides a notification service
• Handles the CPS object messages from any route table manager, and programs them to the Linux kernel using native Linux netlink APIs

The NAS routing module is dedicated to handling the FIB-related functionality in the hold daemon. The NAS routing module obtains Layer 3 route information (IP address to next-hop information using the CPS model).

The NAS is notified about different types of routes and host route updates of address resolution protocol (ARP) related netlink messages from the kernel via the CPS API model (which get the events from the kernel using Linux database calls), maintains the forwarding information base (FIB) database, and updates the hardware host route table entries via the SAI routing API interface.

SAI routing performs the functionality of interfacing with the NPU/SDK and hardware table management.

NAS routing also performs L3 FIB functionality related to recursive-resolution, equal-cost-multiple-path (ECMP), failover, configuration, and objects collection via the CPS interface for clearing of hardware tables, and show and debug functions.

The NAS routing thread receives routes and next-hop information from the CPS event service.

CPS object attribute messages list

• Route add
• Route delete
• Next-hop add
• Next-hop delete
• Clear routes

The NAS routing thread receives ARP resolution information from ARP/NDP netlink messages via the CPS event service.

CPS object attributes list

• ARP resolved
• ARP resolving
• ARP unresolved
• ARP deleted

The NAS routing module builds complete information of all route entry information called the forwarding table database for programming hardware by merging information from the FIBs local unicast route database and ARP database.

For certain routes, the next-hop given by the netlink database interface may have corresponding next-hop information received from the database via the kernel ARP table (for certain routes it may not).

For each route entry, if the next-hop does not directly map to an ARP table node, the NAS routing thread performs a lookup of the next-hop IP address in the unicast route database to find the best route. If the next-hop of the best route maps to an ARP node, recursive resolution is successful else the next-hop IP address of the best route is looked up recursively to maximum recursive depth (or until a resolved ARP entry is found).

NAS routing module logical sub-task functions

• Route resolution sub-task which handles loose next-hops and explicit next-hops (host routes with L2 information and port information from ARP information)
• ARP/NDP management subtask
• Programming NPU via SAI APIs
• Performing configuration/show commands using an object model

NAS routing functionality is implemented using a few threads.

Main thread

• Handles the CPS object messages from the CPS interface for routes and next-hop information, and ARP/NDP information. This information is fed to the CPS event service via the Linux netlink interface.
• Handles northbound routing service requests via the CPS inteface for applications such as PBR, ERSPAN, mirror, and sFlow. These applications invoke CPS APIs to perform get/set and publish/subscribe to the NAS routing.
• Performs route lookups, next-hop lookup, route/next-hop object tracking, configuration/show commands (FIB table entries, next-hop entries, ECMP count) and bookkeeping tasks.

Next-hop thread

• The main function of this thread is to walk through the route database (radix tree of the route) and resolve recursive and simple routes based on next-hop resolution.
• Host routes are ARP resolved and installed into hardware by calling SAI APIs.
• During the process of resolution of next-hop, any dependent resolved routes checked are programmed into hardware using NDI/southbound SAI interface.

Direct route thread

The main function of this thread is to walk through the route database (radix tree of the route) and resolves recursive and simple routes. During route resolution, resolved routes are programmed into hardware using NDI/southbound SAI interface.

Services provided via the CPS interface

• Configuration of NPU forwarding table modes, configuration of ECMP size, and ECMP hash
• Show/clear commands for FIB table entries, nexthop entries, and ECMP count
• Notification service to clients such as PBR, ERSPAN, and sFlow for monitoring reachability of routes and next-hops, and also a lookup service, notification of the next-hop state (reachable, down, up) with tracking ID using its own object tracking functionality.
• Get/Set of routes, next-hop/host entries into the FIB for the Linux kernel netlink events, or applications such as special RTM which needs to program routes into NAS.

NAS routing layers

• Upper-HAL routing (NAS)

  • Handles the generic routing forwarding information base (FIB)
  • Maintains direct routes, host routes, and recursive routes and their dependencies
  • Resolves the routes based on ARP resolution, host routes, and recursive dependencies
  • Adjusts the FIB actions for ACTION_DROP, TRAP_CPU, and ACTION_FORWARD based on route resolution
  • CPS APIs to interface with northbound Dell Networking OS and third-party applications

• Lower HAL routing

  • Handles NDI layer
  • Handles NPU programming for L3 routing (separate for each NPU vendor) and any NPU routing-specific databases (hardware egress object management, and so on)
  • HAL-lower-HAL APIs for routing has generic routing API definitions that each NPU vendor can use to adhere to our lower HAL

• Southbound NDI interfaces with the switch abstraction layer (SAL) which SAI implementation based on open source SAI APIs and vendor SDKs