Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chassis] packet forwarding issue on Clearwater2ms #92

Closed
arlakshm opened this issue Jun 24, 2023 · 4 comments
Closed

[chassis] packet forwarding issue on Clearwater2ms #92

arlakshm opened this issue Jun 24, 2023 · 4 comments

Comments

@arlakshm
Copy link

On some ports the packet forwarding on the clearwater2ms card doesn't work.
When we do ping from the peer device, on some port the ping fails. On these ports the packets are received or transmitted on the phy. The link is up on the ASIC and peer-device.

Some debugs we collected during debug session
lua(slice 0) > /serdes info
+-------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| SerDes Params | Index |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| Param | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_adapt | 2 | 2 | 5 | 5 | 2 | 2 | 4 | 4 | 2 | 2 | 2 | 2 | 1 | 1 | 1 | 1 | 3 | 3 | 3 | 3 | 1 | 1 | 1 | 1 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_agcgain | 16 | 23 | 9 | 18 | 20 | 25 | 24 | 28 | 34 | 30 | 27 | 46 | 49 | 61 | 62 | 36 | 29 | 29 | 36 | 38 | 57 | 40 | 40 | 27 |
| | 31 | 31 | 31 | 23 | 31 | 31 | 31 | 31 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_channel_est | 1.64 | 1.61 | 1.66 | 1.5 | 1.74 | 1.7 | 1.7 | 1.64 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_channel_hf | 28 | 28 | 29 | 30 | 27 | 27 | 27 | 28 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_channel_of | 46 | 45 | 48 | 45 | 47 | 46 | 46 | 46 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_ctle | 6 | 6 | 6 | 4 | 6 | 6 | 3 | 6 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 |
| | 3 | 3 | 3 | 6 | 3 | 3 | 5 | 3 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_dac | 15 | 15 | 8 | 9 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 14 | 13 | 14 | 14 | 15 | 15 | 15 | 15 |
| rx_delta | -4 | -2 | 3 | 3 | -6 | -4 | 0 | -4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_dfe | -0 | -0 | 0.766 | 0.728 | -0 | -0 | -0 | -0 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 5 | 4 | 4 | 5 | 4 | 4 | 4 | 4 |
| | -0 | -0 | 0.22 | 0.241 | -0 | -0 | -0 | -0 | -4 | -4 | -4 | -4 | -4 | -4 | -4 | -4 | -6 | -8 | -6 | -7 | -4 | -4 | -4 | -4 |
| | 0 | 0 | 0.288 | 0.331 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 3 | 1 | 3 | 0 | 0 | 0 | 0 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_edge | 39321 | 39321 | 39321 | 39321 | 43690 | 43690 | 43690 | 39321 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 | 43690 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_eye_height | 215 | 206 | 211 | 206 | 198 | 198 | 215 | 206 | 735 | 654 | 639 | 682 | 723 | 777 | 715 | 678 | 689 | 678 | 693 | 688 | 798 | 681 | 713 | 737 |
| | 258 | 241 | 234 | 241 | 232 | 232 | 232 | 241 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| | 223 | 206 | 211 | 208 | 206 | 198 | 206 | 215 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_f1over3 | 5 | 5 | 5 | 4 | 5 | 5 | 5 | 5 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 | 3 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_ffe_taps | 115 | 151 | 115 | 234 | 122 | 120 | 120 | 135 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 |
| | 94 | 63 | 55 | -6 | 19 | 77 | 75 | 122 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 |
| | 1 | 1 | 1 | 122 | 95 | 1 | 1 | 1 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 |
| | 3 | 1 | 5 | -165 | -4 | 2 | -26 | -24 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 |
| | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 |
| | 1 | 1 | 1 | 4 | 4 | 1 | 1 | 1 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 |
| | 136 | 136 | 136 | 139 | 136 | 136 | 136 | 136 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 | 204 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| rx_graycode | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_input_coupling | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| rx_linklost | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 |
| rx_msblsb | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_pll_cap | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 31 | 31 | 31 | 31 | 31 | 32 | 31 | 31 | 31 | 31 | 31 | 31 | 31 | 31 | 31 | 32 |
| rx_pol | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_ppm | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -35 | -35 | -35 | -35 | -35 | -35 | -35 | -35 | -40 | -40 | -35 | -40 | -35 | -35 | -35 | -35 |
| rx_precoder | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_readapt | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_ready | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 |
| rx_signal_detect | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 |
| rx_skef_addcap | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 |
| rx_skef_degen | 4 | 2 | 4 | 2 | 4 | 4 | 1 | 3 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 |
| rx_skef_en | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| rx_skef_gain | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 | 32531 |
| tx_graycode | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| tx_msblsb | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| tx_pll_cap | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 27 | 31 | 31 | 31 | 32 | 32 | 32 | 31 | 31 | 31 | 31 | 31 | 32 | 32 | 32 | 32 | 32 |
| tx_pol | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| tx_precoder | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| tx_taps | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| | -5 | -5 | -5 | -5 | -5 | -5 | -5 | -5 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 | -1 |
| | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 | 15 |
| | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 | -6 |
| | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
+-------------------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+

crlua(slice 0)
crlua(slice 0) > /lane status
+---------+------+---------+----------+---------------+------------+
| Lane | Mode | Speed | LB Mode | Tx State | Rx State |
| | | | | | SD RDY ADP |
+---------+------+---------+----------+---------------+------------+
| A0(0) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A1(1) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A2(2) | PAM4 | 53.125G | DISABLED | TRAFFIC | 1 1 1 |
| A3(3) | PAM4 | 53.125G | DISABLED | TRAFFIC | 1 1 1 |
| A4(4) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A5(5) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A6(6) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A7(7) | PAM4 | 53.125G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A8(8) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A9(9) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A10(10) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| A11(11) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B0(12) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B1(13) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B2(14) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B3(15) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B4(16) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 |
| B5(17) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 |
| B6(18) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 |
| B7(19) | NRZ | 25.781G | DISABLED | TRAFFIC | 1 1 1 |
| B8(20) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B9(21) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B10(22) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
| B11(23) | NRZ | 25.781G | DISABLED | FORCE_DISABLE | 0 0 0 |
+---------+------+---------+----------+---------------+------------+

crlua(slice 0) >

@arlakshm
Copy link
Author

@kenneth-arista created this issue to track the clearwater2ms issue

lguohan pushed a commit to sonic-net/sonic-buildimage that referenced this issue Jul 5, 2023
Reverts #14149

This SAI libsaicredo_0.9.0_amd64.deb causing packet forwarding issues on Linecards aristanetworks/sonic#92

This reverts commit c4c621c.
mssonicbld pushed a commit to mssonicbld/sonic-buildimage that referenced this issue Jul 7, 2023
…15708)

Reverts sonic-net#14149

This SAI libsaicredo_0.9.0_amd64.deb causing packet forwarding issues on Linecards aristanetworks/sonic#92

This reverts commit c4c621c.
mssonicbld pushed a commit to mssonicbld/sonic-buildimage that referenced this issue Jul 7, 2023
…15708)

Reverts sonic-net#14149

This SAI libsaicredo_0.9.0_amd64.deb causing packet forwarding issues on Linecards aristanetworks/sonic#92

This reverts commit c4c621c.
mssonicbld pushed a commit to mssonicbld/sonic-buildimage that referenced this issue Jul 17, 2023
…15708)

Reverts sonic-net#14149

This SAI libsaicredo_0.9.0_amd64.deb causing packet forwarding issues on Linecards aristanetworks/sonic#92

This reverts commit c4c621c.
@kenneth-arista
Copy link

kenneth-arista commented Jul 26, 2023

Quick update. A new Credo SAI release is forthcoming to address this issue.

@byu343
Copy link
Contributor

byu343 commented Jul 28, 2023

A new version v0.9.3 has been released. It can fix this issue from my testing.

@kenneth-arista
Copy link

Closing this issue as v0.9.3 Credo SAI should address this issue.

sonic-otn pushed a commit to sonic-otn/sonic-buildimage that referenced this issue Sep 20, 2023
…15708)

Reverts sonic-net#14149

This SAI libsaicredo_0.9.0_amd64.deb causing packet forwarding issues on Linecards aristanetworks/sonic#92

This reverts commit c4c621c.
lguohan pushed a commit to sonic-net/sonic-buildimage that referenced this issue Oct 25, 2023
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
mssonicbld pushed a commit to mssonicbld/sonic-buildimage that referenced this issue Nov 9, 2023
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
mssonicbld pushed a commit to mssonicbld/sonic-buildimage that referenced this issue Nov 20, 2023
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
mssonicbld pushed a commit to mssonicbld/sonic-buildimage that referenced this issue Nov 20, 2023
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
mssonicbld pushed a commit to sonic-net/sonic-buildimage that referenced this issue Nov 21, 2023
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
mssonicbld pushed a commit to sonic-net/sonic-buildimage that referenced this issue Nov 21, 2023
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
fengkm pushed a commit to ufispace-dev/sonic-buildimage that referenced this issue Apr 2, 2024
…15708)

Reverts sonic-net#14149

This SAI libsaicredo_0.9.0_amd64.deb causing packet forwarding issues on Linecards aristanetworks/sonic#92

This reverts commit c4c621c.
fengkm pushed a commit to ufispace-dev/sonic-buildimage that referenced this issue Apr 2, 2024
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
lukasstockner pushed a commit to genesiscloud/sonic-buildimage that referenced this issue Jul 2, 2024
Update credo sai package to the latest v0.9.3, which fixes the issue aristanetworks/sonic#92.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants