Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move teamd warm reboot code to service script #5163

Merged
merged 19 commits into from
Nov 13, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
d16d163
[sonic-buildimage] Move BGP warm reboot scripts into BGP service /usr…
Jan 8, 2020
93add80
Revert "[sonic-buildimage] Move BGP warm reboot scripts into BGP serv…
Jan 8, 2020
bf4cee2
Merge remote-tracking branch 'upstream/master'
Feb 5, 2020
1b9e1f9
Merge remote-tracking branch 'upstream/master'
May 18, 2020
790b2d0
Merge remote-tracking branch 'upstream/master'
Jun 5, 2020
4151184
Merge branch 'master' of https://github.com/Azure/sonic-buildimage
heidinet2007 Jul 29, 2020
5e4b025
Move teamd warm reboot functions to service scripts
heidinet2007 Aug 12, 2020
ab241d5
Merge branch 'master' of https://github.com/Azure/sonic-buildimage
heidinet2007 Aug 23, 2020
a912623
Merge branch 'master' into teamd
heidinet2007 Aug 24, 2020
76cb10a
Move teamd warm reboot code to teamd service at /usr/local/bin
heidinet2007 Aug 26, 2020
628d41f
Move teamd warm reboot code to service script
heidinet2007 Aug 26, 2020
e06df98
Address review comments: pass different sigusr to warm reboot and fas…
heidinet2007 Oct 20, 2020
95ac98f
Address code review comment: pass different sigusr to warm reboot and…
heidinet2007 Oct 20, 2020
02f3abf
Address code review comments: 1) remove space in variable assignement…
heidinet2007 Oct 27, 2020
c927519
Address code review comments:
heidinet2007 Oct 27, 2020
c7e32db
Address code review comment:
heidinet2007 Oct 27, 2020
0570547
Update comment context to reflect current code
heidinet2007 Oct 27, 2020
e1538e9
Address code revew comments: update comments and remove unused variab…
heidinet2007 Oct 31, 2020
8a4e634
chmod +x teamd.sh
yxieca Nov 13, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions files/build_templates/per_namespace/teamd.service.j2
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ StartLimitBurst=3

[Service]
User={{ sonicadmin_user }}
ExecStartPre=/usr/bin/{{docker_container_name}}.sh start{% if multi_instance == 'true' %} %i{% endif %}
ExecStart=/usr/bin/{{docker_container_name}}.sh wait{% if multi_instance == 'true' %} %i{% endif %}
ExecStop=/usr/bin/{{docker_container_name}}.sh stop{% if multi_instance == 'true' %} %i{% endif %}
ExecStartPre=/usr/local/bin/{{docker_container_name}}.sh start{% if multi_instance == 'true' %} %i{% endif %}
ExecStart=/usr/local/bin/{{docker_container_name}}.sh wait{% if multi_instance == 'true' %} %i{% endif %}
ExecStop=/usr/local/bin/{{docker_container_name}}.sh stop{% if multi_instance == 'true' %} %i{% endif %}
Restart=always
RestartSec=30

Expand Down
3 changes: 2 additions & 1 deletion files/build_templates/sonic_debian_extension.j2
Original file line number Diff line number Diff line change
Expand Up @@ -529,10 +529,11 @@ sudo LANG=C chroot $FILESYSTEM_ROOT fuser -km /sys || true
sudo LANG=C chroot $FILESYSTEM_ROOT umount -lf /sys
{% endif %}

# Copy service scripts (swss, syncd, bgp, radv)
# Copy service scripts (swss, syncd, bgp, teamd, radv)
sudo LANG=C cp $SCRIPTS_DIR/swss.sh $FILESYSTEM_ROOT/usr/local/bin/swss.sh
sudo LANG=C cp $SCRIPTS_DIR/syncd.sh $FILESYSTEM_ROOT/usr/local/bin/syncd.sh
sudo LANG=C cp $SCRIPTS_DIR/bgp.sh $FILESYSTEM_ROOT/usr/local/bin/bgp.sh
sudo LANG=C cp $SCRIPTS_DIR/teamd.sh $FILESYSTEM_ROOT/usr/local/bin/teamd.sh
vaibhavhd marked this conversation as resolved.
Show resolved Hide resolved
sudo LANG=C cp $SCRIPTS_DIR/radv.sh $FILESYSTEM_ROOT/usr/local/bin/radv.sh

# Copy sonic-netns-exec script
Expand Down
108 changes: 108 additions & 0 deletions files/scripts/teamd.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
#!/bin/bash

function debug()
{
/usr/bin/logger $1
/bin/echo `date` "- $1" >> ${DEBUGLOG}
}

function check_warm_boot()
{
SYSTEM_WARM_START=`$SONIC_DB_CLI STATE_DB hget "WARM_RESTART_ENABLE_TABLE|system" enable`
SERVICE_WARM_START=`$SONIC_DB_CLI STATE_DB hget "WARM_RESTART_ENABLE_TABLE|${SERVICE}" enable`
if [[ x"$SYSTEM_WARM_START" == x"true" ]] || [[ x"$SERVICE_WARM_START" == x"true" ]]; then
WARM_BOOT="true"
else
WARM_BOOT="false"
fi
}

function validate_restore_count()
{
if [[ x"$WARM_BOOT" == x"true" ]]; then
RESTORE_COUNT=`$SONIC_DB_CLI STATE_DB hget "WARM_RESTART_TABLE|${SERVICE}" restore_count`
# We have to make sure db data has not been flushed.
if [[ -z "$RESTORE_COUNT" ]]; then
WARM_BOOT="false"
fi
fi
}

function check_fast_boot ()
{
if [[ $($SONIC_DB_CLI STATE_DB GET "FAST_REBOOT|system") == "1" ]]; then
FAST_BOOT="true"
else
FAST_BOOT="false"
fi
}

start() {
debug "Starting ${SERVICE}$DEV service..."

check_warm_boot
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also wait for database initialization before accessing it? I see that some of the service scripts use it, and some are missing it already.

https://github.com/Azure/sonic-buildimage/blob/master/files/scripts/swss.sh#L60
https://github.com/Azure/sonic-buildimage/blob/master/files/scripts/swss.sh#L137

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw this wait_for_database_service() is called from swss and syncd. Teamd already has dependancy on swss.

validate_restore_count

check_fast_boot

debug "Warm boot flag: ${SERVICE}$DEV ${WARM_BOOT}."
debug "Fast boot flag: ${SERVICE}$DEV ${Fast_BOOT}."

# start service docker
/usr/bin/${SERVICE}.sh start $DEV
debug "Started ${SERVICE}$DEV service..."
}

wait() {
/usr/bin/${SERVICE}.sh wait $DEV
}

stop() {
debug "Stopping ${SERVICE}$DEV service..."

check_warm_boot
check_fast_boot
debug "Warm boot flag: ${SERVICE}$DEV ${WARM_BOOT}."
debug "Fast boot flag: ${SERVICE}$DEV ${FAST_BOOT}."

if [[ x"$WARM_BOOT" == x"true" ]]; then
# Send USR1 signal to all teamd instances to stop them
# It will prepare teamd for warm-reboot
# Note: We must send USR1 signal before syncd, because it will send the last packet through CPU port
docker exec -i ${SERVICE}$DEV pkill -USR1 ${SERVICE} > /dev/null || [ $? == 1 ]
elif [[ x"$FAST_BOOT" == x"true" ]]; then
# Kill teamd processes inside of teamd container with SIGUSR2 to allow them to send last LACP frames
# We call `docker kill teamd` to ensure the container stops as quickly as possible,
# Note: teamd must be killed before syncd, because it will send the last packet through CPU port
docker exec -i ${SERVICE}$DEV pkill -USR2 ${SERVICE} || [ $? == 1 ]
while docker exec -i ${SERVICE}$DEV pgrep ${SERVICE} > /dev/null; do
sleep 0.05
done
docker kill ${SERVICE}$DEV &> /dev/null || debug "Docker ${SERVICE}$DEV is not running ($?) ..."
fi

/usr/bin/${SERVICE}.sh stop $DEV
debug "Stopped ${SERVICE}$DEV service..."
}

DEV=$2

SERVICE="teamd"
DEBUGLOG="/tmp/teamd-debug$DEV.log"
NAMESPACE_PREFIX="asic"
if [ "$DEV" ]; then
NET_NS="$NAMESPACE_PREFIX$DEV" #name of the network namespace
SONIC_DB_CLI="sonic-db-cli -n $NET_NS"
else
SONIC_DB_CLI="sonic-db-cli"
fi

case "$1" in
start|wait|stop)
$1
;;
*)
echo "Usage: $0 {start|wait|stop}"
exit 1
;;
esac