Add details of sff_mgr regarding deterministic bringup for SFF compliant modules

doc/xrcvd/Interface-Link-bring-up-sequence-on-sff-modules.md

@@ -0,0 +1,181 @@
+# Feature Name
+Deterministic Approach for Interface Link bring-up sequence on SFF compliant modules
+
+# High Level Design Document
+#### Rev 0.1
+
+# Table of Contents
+ * [List of Tables](#list-of-tables)
+ * [Revision](#revision)
+ * [About This Manual](#about-this-manual)
+ * [Abbreviation](#abbreviation)
+ * [References](#references)
+ * [Problem Definition](#problem-definition)
+ * [Background](#background)
+ * [Objective](#objective)
+ * [Plan](#plan)
+ * [Breakout handling](#breakout-handling)
+ * [Feature enablement](#feature-enablement)
+ * [Pre-requisite](#pre-requisite)
+ * [Proposed Work-Flows](#proposed-work-flows)
+
+# List of Tables
+ * [Table 1: Definitions](#table-1-definitions)
+ * [Table 2: References](#table-2-references)
+
+# Revision
+| Rev | Date | Author | Change Description |
+|:---:|:-----------:|:----------------------------------:|-------------------------------------|
+| 0.1 | 07/12/2023 | Longyin Huang | Initial version |
+
+
+# About this Manual
+This is a high-level design document describing the need to have determinstic approach on SFF compliant modules for Interface link bring-up sequence and workflows for use-cases around it
+
+Parent HLD [Interface-Link-bring-up-sequence.md](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md) focuses on generic high level background/idea and details for CMIS modules, while this HLD focuses on SFF modules with details.
+
+# Abbreviation
+
+# Table 1: Definitions
+| **Term** | **Definition** |
+| -------------- | ------------------------------------------------ |
+| pmon | Platform Monitoring Service |
+| xcvr | Transceiver |
+| xcvrd | Transceiver Daemon |
+| gbsyncd | Gearbox (External PHY) docker container |
+
+# References
+
+# Table 2 References
+
+| **Document** | **Location** |
+|---------------------------------------------------------|---------------|
+| Deterministic Approach for Interface Link bring-up sequence for CMIS and SFF modules | [Interface-Link-bring-up-sequence.md](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md) |
+
+
+
+# Problem Definition
+According to parent [HLD](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md#plan), as already discussed with sonic-chassis workgroup and OCP community:
+
+1. Presently in SONiC, for SFF compliant modules (100G/40G), there is no synchronization between enabling Tx of optical module and enabling ASIC (NPU/PHY) Tx which may cause link instability during administrative interface enable “config interface startup Ethernet” configuration and bootup scenarios.
+
+2. During administrative interface disable “config interface shutdown Ethernet”, only the ASIC(NPU) Tx is disabled and not the optical module Tx/laser.
+ This will lead to power wastage, un-necessary fan power consumption to keep the module temperature in operating range, and potential lab hazard when the port is shut off but the laser is still on.
+
+# Background
+
+ Refer to parent [HLD](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md#background)
+
+# Objective
+
+According to parent [HLD](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md#objective), have a determistic approach for Interface link bring-up sequence for SFF compliant modules (100G/40G) i.e. below sequence to be followed:
+ 1. Initialize and enable NPU Tx and Rx path
+ 2. For system with 'External' PHY: Initialize and enable PHY Tx and Rx on both line and host sides; ensure host side link is up
+ 3. Then perform optics Tx enable
+
+# Plan
+
+Plan is to follow this high-level work-flow sequence to accomplish the Objective:
+- Add a new thread SFF task manager (called sff_mgr) inside xcvrd to subscribe to existing field “host_tx_ready” in port table state-DB
+- “host_tx_ready” is set to true only when admin_status is up and setting admin_status to syncd/gbsyncd is successful. (As part of setting admin_status to syncd/gbsyncd successfully, the NPU/PHY Tx is enabled/disabled)
+- sff_mgr processes the “host_tx_ready” value change event and do optics Tx enable/disable using tx_disable API
+
+# Breakout Handling
+
+Refer to parent [HLD](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md#breakout-handling)
+
+# Feature enablement
+
+ This feature (optics Interface Link bring-up sequence) would be enabled on per platform basis.
+ There could be cases where vendor(s)/platform(s) may take time to shift from existing codebase to the model (work-flows) described in this document.
+- By default, sff_mgr feature is disabled.
+- In order to enable sff_mgr feature, the platform would set ‘enable_xcvrd_sff_mgr’ to ‘true’ in their respective pmon_daemon_control.json. Xcvrd would parse ‘enable_xcvrd_sff_mgr’ and if found 'true', it would launch SFF task manager (sff_mgr).
+
+# Pre-requisite
+
+In addition to parent HLD's [pre-requisite](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md#pre-requisite),
+
+> **_Pre-requisite for enabling sff_mgr:_**
+Platform needs to leave the transceiver (if capable of disabling Tx) in Tx disabled state when an module inserted or during boot-up. This is to make sure the transceiver is not transmitting with Tx enabled before host_tx_ready is True.
+
+# Proposed Work-Flows
+
+ - ### Flow of pre-requisite for platform in insertion/bootup cases
+ ```mermaid
+ graph TD;
+ A[platfrom brings module out of RESET]
+ B[platform keeps module in Tx disabled state immediately after module out-of-RESET]
+ C[xcvrd detects module insertion via platform API get_transceiver_change_event, and update module status/info to DB]
+ D[Upon module insertion event, sff_mgr takes action accordingly if needed]
+
+ Start --> A
+ A --> B
+ B --> C
+ C --> D
+ D --> End
+ ```
+ - ### Feature enablment flow -- how xcvrd spawns sff_mgr thread based on enable_sff_mgr flag
+ ```mermaid
+ graph TD;
+ A[wait for PortConfigDone]
+ B[check if enable_sff_mgr flag exists and is set to true]
+ C[spawn sff_mgr]
+ D[proceed to other thread spawning and tasks]
+
+ Start --> A
+ A --> B
+ B -- true --> C
+ C --> D
+ B -- false --> D
+ D --> End
+ ```
+ - ### Flow of calculating target tx_disable value:
+ - When ```tx_disable value/status``` is ```True```, it means Tx is disabed
+ - when ```tx_disable value/status``` is ```False```, it means Tx is enabled
+ ```mermaid
+ graph TD;
+
+ A[check if both host_tx_ready is True AND admin_status is UP]
+ B[target tx_disable value is set to False, Tx should be turned ON]
+ C[target tx_disable value is set to True, Tx should be turned OFF]
+
+ Start --> A
+ A -- true --> B
+ A -- false --> C
+ B --> End
+ C --> End
+ ```
+ - ### Main flow of sff_mgr, covering below cases:
+ - system bootup
+ - transceiver insertion
+ - admin enable/disable configurations
+ ```mermaid
+ graph TD;
+ A[subscribe to events]
+ B[while task_stopping_event is not set]
+ C[check insertion event, host_tx_ready change event and admin_status change event for each intended port]
+ D[double check if module is present]
+ E[fetch DB and update host_tx_ready value in local cahce, if not available locally]
+ E2[fetch DB and update admin_status value in local cahce, if not available locally]
+ F[calculate target tx_disable value based on host_tx_ready and admin_status]
+ G[check if tx_disable status on module is already the target value]
+ H[go ahead to enable/disable Tx based on the target tx_disable value]
+
+ Start --> A
+ A --> B
+ B -- true --> C
+ C -- if either event happened --> E
+ C -- if neither event happened --> B
+ E --> E2
+ E2 --> D
+ D -- true --> F
+ D -- false --> B
+ F --> G
+ G -- true --> B
+ G -- false --> H
+ H --> B
+ B -- false --> End
+ ```
+
+# Out of Scope
+Refer to parent [HLD](https://github.com/sonic-net/SONiC/blob/master/doc/sfp-cmis/Interface-Link-bring-up-sequence.md)