Supports FRR-VRRP configuration #18617

philo-micas · 2024-04-10T07:48:17Z

Why I did it

Work item tracking

Microsoft ADO (number only):

How I did it

hld:sonic-net/SONiC#1446

How to verify it

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: philo <philo@micasnetworks.com>

…28-support-vrrp6-commands-and-tracking-interface.patch

…30-support-vrrp6-commands-and-tracking-interface.patch

Signed-off-by: philo <philo@micasnetworks.com>

venkatmahalingam · 2024-09-04T23:40:19Z

src/sonic-vrrpcfgd/vrrpcfgd/vrrpcfgd.py

+    PROXY_SERVER_ADDR = '/etc/frr/vrrpd_client_sock'
+    ALL_DAEMONS = ['vrrpd']
+    TABLE_DAEMON = {
+            'VRRP': ['vrrpd'],


I believe, we discussed to have VRRP config handler for FRR as part of frrcfgd itself, what are we doing here?

Yes, we have also implemented this functionality in frrcfgd. This provides an alternative for usage when frrcfgd is not enabled. Some users may need to use it this way.

vrrcfgd is totally a duplicate code, I would like to better understand in which case vrrpcfgd will be enabled but not frrcfgd. Keeping both the code will be hard to manage, I would recommend using frrcfgd only for VRRP feature.

vrrcfgd is totally a duplicate code, I would like to better understand in which case vrrpcfgd will be enabled but not frrcfgd. Keeping both the code will be hard to manage, I would recommend using frrcfgd only for VRRP feature.

OK, vrrpcfgd has been removed, thanks.

…54-support-vrrp6-commands-and-tracking-interface.patch

Signed-off-by: philo <philo@micasnetworks.com>

nmoray · 2024-09-27T10:28:00Z

@philo-micas As per the HLD, I am unable to find the changes related to vrrpsyncd and vrrporch in any of your PRs. Can you please point me to the respective PR?
cc: @venkatmahalingam

Signed-off-by: philo <philo@micasnetworks.com>

philo-micas · 2024-10-11T01:51:25Z

@vvbrcm @madhupalu please help review

src/sonic-frr-mgmt-framework/frrcfgd/frrcfgd.py

nmoray · 2024-10-23T07:40:38Z

Please check the CPU utilisation of frrcfgd when VRRP is enabled. I found it to be constant to 100%.

src/sonic-frr/patch/0054-support-vrrp6-commands-and-tracking-interface.patch

Signed-off-by: philo <philo@micasnetworks.com>

philo-micas · 2024-10-31T12:37:14Z

Please check the CPU utilisation of frrcfgd when VRRP is enabled. I found it to be constant to 100%.

Do you know why it is constant 100% CPU utilization? adding some context would help e.g in which scenario you got into the issue.

No @venkatmahalingam yet to root cause the issue. But found that even if the VRRP is not configured, frrcfgd continuously consuming high CPU. Same was the case with vrrpcfgd which is now removed.
Logs:

top - 18:54:41 up  2:30,  1 user,  load average: 7.70, 7.95, 8.19
Tasks: 277 total,   4 running, 269 sleeping,   0 stopped,   4 zombie
%Cpu(s): 26.6 us,  5.8 sy,  0.0 ni, 42.5 id,  0.0 wa,  0.0 hi, 25.1 si,  0.0 st
MiB Mem :  15964.3 total,  12048.7 free,   2273.7 used,   1641.9 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  13338.7 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
     11 root      20   0       0      0      0 R 100.0   0.0 126:41.92 ksoftir+
   3579 root      20   0  186544  23128  13692 S 100.3   0.1 148:38.92 vrrpcfgd

Docker Stats:
CONTAINER ID   NAME      CPU %     MEM USAGE / LIMIT   MEM %     NET I/O   BLOCK I/O         PIDS
8655565a9f12   bgp       101.00%   191MiB / 15.59GiB   1.20%     0B / 0B   24.4MB / 14.4MB   37   


root@sonic:/home/admin# show vrrp summary
 Interface   VRID   Priority   IPv4   State (v4)
 --------------------------------------------------
 Vlan100     1      100        1      Master

I tried to reproduce it through the following configuration, but the result was normal:

root@sonic:/home/admin# config interface vrrp ip add Ethernet53 5 10.5.5.1/24
root@sonic:/home/admin# config interface vrrp add Ethernet57 6
root@sonic:/home/admin# show vrrp summary 

 Interface    VRID   Priority   IPv4   State (v4)   
 ---------------------------------------------------
 Ethernet53   5      100        1      Master       
 Ethernet57   6      100        0      Initialize

top - 09:00:42 up 36 min,  0 user,  load average: 0.45, 0.69, 0.78
Tasks:  15 total,   1 running,  14 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.3 us,  3.9 sy,  0.0 ni, 93.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st 
MiB Mem :   7827.1 total,   2916.1 free,   3031.6 used,   2289.0 buff/cache     
MiB Swap:      0.0 total,      0.0 free,      0.0 used.   4795.5 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                         
     39 frr       20   0   43764   7732   5556 S   0.0   0.1   0:00.15 vrrpd                                                                                         
     70 root      20   0  190244  27192  14016 S   0.0   0.3   0:00.13 frrcfgd

Perhaps I missed some details, could you please provide your testing steps? Thanks.

@philo-micas Following are configs that I have used.

Sw1:
config vlan add 100
config vlan member add -u 100 Ethernet120
config interface ip add Vlan100 100.0.0.2/24
config interface vrrp add Vlan100 1
config interface vrrp ip add  Vlan100 1 100.0.0.1/24
config interface vrrp priority Vlan100 1 100
#config interface vrrp track_interface add Vlan100 1 Ethernet48
config interface vrrp ip remove  Vlan100 1 100.0.0.1/24
Sw2:
config vlan add 100
config vlan member add -u 100 Ethernet120
config interface ip add Vlan100 100.0.0.3/24
config interface vrrp add Vlan100 1
config interface vrrp ip add  Vlan100 1 100.0.0.1/24
config interface vrrp priority Vlan100 1 90
#config interface vrrp track_interface add Vlan100 1 Ethernet48


root@sonic1:~# show vrrp summary
 Interface   VRID   Priority   IPv4   State (v4)
 --------------------------------------------------
 Vlan100     1      100        1      Master

root@sonic2:/host# show vrrp summary
 Interface   VRID   Priority   IPv4   State (v4)
 --------------------------------------------------
 Vlan100     1      90         1      Backup


top - 18:38:51 up  7:25,  1 user,  load average: 4.27, 2.50, 2.15
Threads: 663 total,   4 running, 655 sleeping,   0 stopped,   4 zombie
%Cpu(s): 23.6 us, 25.6 sy,  0.0 ni, 50.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  15964.2 total,  12103.7 free,   2044.1 used,   1816.4 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  13543.6 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 633361 root      20   0  194488  31360  14032 R  99.9   0.2   0:39.58 frrcfgd
 634902 root      20   0   53084  35336  15400 R  72.7   0.2   0:03.21 python3
 633555 root      20   0   91216   8716   7784 S   4.9   0.1   0:00.75 vrrpsyn+
 633325 root      20   0 2815108 682996 238836 S   4.3   4.2   0:02.26 bcmCNTR+
 632800 root      20   0  558444  13504   9304 S   3.0   0.1   0:00.29 eventd
 633378 root      20   0 2815108 682996 238836 S   2.0   4.2   0:00.79 syncd
 634911 root      20   0  129052  35304  16084 S   1.6   0.2   0:01.06 python3
     12 root      20   0       0      0      0 I   1.0   0.0   0:21.06 rcu_sch+
 634863 root      20   0   11424   4632   3372 R   1.0   0.0   0:00.29 top
 633369 root      20   0 2815108 682996 238836 S   0.7   4.2   0:00.32 bcmLINK+
     43 root      39  19       0      0      0 S   0.3   0.0   0:01.35 khugepa+
   2702 message+  20   0  138988  68444   7784 R   0.3   0.4  14:43.18 redis-s+
 632382 root      20   0   29596  25340   9488 S   0.3   0.2   0:00.49 supervi+
 633252 root      20   0 2815108 682996 238836 S   0.3   4.2   0:00.14 socdmad+
 634517 root      20   0   29740  25492   9488 S   0.3   0.2   0:00.72 supervi+
 634743 root      20   0 1416832  51600  27100 S   0.3   0.3   0:00.05 telemet+
 634752 root      20   0 1337996  46192  23732 S   0.3   0.3   0:00.06 dialout+

root@sonic2:/host# docker stats
8a79ed27273c   swss             0.70%     65.12MiB / 15.59GiB   0.41%     0B / 0B   1.6MB / 328kB     50
8d323830e8d1   eventd           0.11%     34.2MiB / 15.59GiB    0.21%     0B / 0B   0B / 77.8kB       19
708375a47cd9   database         3.68%     142.2MiB / 15.59GiB   0.89%     0B / 0B   48.8MB / 81.9kB   13
CONTAINER ID   NAME             CPU %     MEM USAGE / LIMIT     MEM %     NET I/O   BLOCK I/O         PIDS
bebb7927a6e5   snmp             0.27%     55.35MiB / 15.59GiB   0.35%     0B / 0B   0B / 139kB        11
f3d7927eaca5   pmon             0.24%     136.9MiB / 15.59GiB   0.86%     0B / 0B   0B / 139kB        18
1d99c6a6e3f8   mgmt-framework   0.02%     29.55MiB / 15.59GiB   0.19%     0B / 0B   0B / 69.6kB       15
dc3c3ddfe6b2   lldp             0.02%     53.65MiB / 15.59GiB   0.34%     0B / 0B   0B / 115kB        13
08cf02d2efc2   gnmi             0.48%     59.77MiB / 15.59GiB   0.37%     0B / 0B   81.9kB / 90.1kB   28
85e3888f18b2   bgp              99.99%    85.96MiB / 15.59GiB   0.54%     0B / 0B   3.57MB / 221kB    36
44d14a5ea772   radv             0.02%     31.05MiB / 15.59GiB   0.19%     0B / 0B   0B / 81.9kB       8
4532893858f1   syncd            11.82%    536.5MiB / 15.59GiB   3.36%     0B / 0B   0B / 365kB        51
c9ba459d907d   teamd            0.04%     33.18MiB / 15.59GiB   0.21%     0B / 0B   0B / 94.2kB       14
8a79ed27273c   swss             0.70%     65.12MiB / 15.59GiB   0.41%     0B / 0B   1.6MB / 328kB     50
8d323830e8d1   eventd           0.11%     34.2MiB / 15.59GiB    0.21%     0B / 0B   0B / 77.8kB       19
708375a47cd9   database         3.68%     142.2MiB / 15.59GiB   0.89%     0B / 0B   48.8MB / 81.9kB   13

And If neither vrrpcfgd nor ffrcfgd is running, I saw that "bufferM+" process is consuming all of the CPU.

top - 18:06:02 up  6:52,  1 user,  load average: 1.70, 1.80, 2.04
Tasks: 273 total,   2 running, 267 sleeping,   0 stopped,   4 zombie
%Cpu(s): 32.4 us,  2.8 sy,  0.0 ni, 64.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  15964.2 total,  11828.2 free,   2341.0 used,   1795.0 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  13249.8 avail Mem
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 521297 root      20   0   91560   9944   8820 R 106.2   0.1  42:24.74 bufferm+
 520932 root      20   0 2818008 752312 239464 S  12.5   4.6   4:33.70 syncd
   2702 message+  20   0  138988  72004   7784 S   6.2   0.4  13:41.09 redis-s+
 587963 root      20   0   10928   3948   3268 R   6.2   0.0   0:00.02 top
      1 root      20   0  166092  12732   8972 S   0.0   0.1   1:15.38 systemd

We tried to verify again, but the result is still normal. Perhaps there are other factors affecting your environment. After this PR is merged, you can try using the community version for verification. Thanks.

Signed-off-by: philo <philo@micasnetworks.com>

mssonicbld · 2024-12-26T03:37:33Z