Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fwutil Auto-update support #1165

Closed
wants to merge 64 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
8bf63af
initial commit
sujinmkang Oct 9, 2020
8a3a278
second commit
sujinmkang Oct 12, 2020
a71c72b
[show] Add 'show' CLI for system-health feature (#971)
shlomibitton Oct 12, 2020
aa119ff
auto_update_fix
sujinmkang Oct 13, 2020
b6af9f4
Fix expected neighbor when multiple ports connect to same neighbor (#…
lguohan Oct 13, 2020
3a7457c
[celestica] consutil to support customize tty device name (#1155)
sandycelestica Oct 13, 2020
921e89c
reboot_pluging for fwutil auto-update task
sujinmkang Oct 14, 2020
059cdca
Fix exception for ipaddress in python2 (#1164)
bingwang-ms Oct 14, 2020
a733df5
[watermarkstat] Add unit tests for watermarkstat show commands (#1157)
neethajohn Oct 14, 2020
e51d44f
[consutil] Fix issue where the show line command crash if no ttyUSB e…
Blueve Oct 15, 2020
c382d89
[config/console] Support update console configuration related command…
Blueve Oct 15, 2020
c395e14
Modify fast-reboot script to use BGP service script to stop bgp servi…
vaibhavhd Oct 19, 2020
510d0ad
[consutil] Add brief option to show line command (#1176)
Blueve Oct 20, 2020
59a511d
[config/show] Add CLI support for proxy arp (#1168)
theasianpianist Oct 20, 2020
13bd06b
Fixes the issue with show interface counters and for pfc and queue co…
abdosi Oct 20, 2020
ca92e36
[counterpoll] Disable Counter Poll When Entering Fast Reboot (#1174)
tahmed-dev Oct 21, 2020
0c3a97f
Remove stdeb.cfg; no longer used now that we build as wheel (#1182)
jleveque Oct 23, 2020
00a2570
fdbshow and nbrshow use SonicV2Connector with decode_responses=True, …
qiluo-msft Oct 24, 2020
f14bbe5
[CLI][show][platform] Added ASIC count in the output. (#1185)
smaheshm Oct 26, 2020
e889212
[fwutil]: Set min log priority to INFO. (#1191)
nazariig Oct 26, 2020
63c6733
Add license file, remove third-party licenses file (#1192)
jleveque Oct 27, 2020
1753f22
[show] remove 'device2interface_dict' in show interface neighbor expe…
lolyu Oct 27, 2020
8af9aee
Show FG_NHG CLI Commands Added (#1056)
kktheballer Oct 29, 2020
ef5f212
[sfputil] Display 'N/A' for non-SFP ports (#1078)
PJHsieh Nov 2, 2020
e3d3d92
[consutil] Store console port state in STATE_DB (#1208)
Blueve Nov 4, 2020
acfa824
Add a switch to route_check to control whether write log to syslog (#…
bingwang-ms Nov 4, 2020
061f428
[show/fgnhg.py] Fix whitespace issues (#1211)
jleveque Nov 4, 2020
42efc03
SONiC installer - fix string formatting during image type check (#1197)
vaibhavhd Nov 4, 2020
99de167
[Mellanox] SKU creator Tool (#1163)
madhanmellanox Nov 5, 2020
97dec12
[consutil][show] Remove root need from show line command (#1218)
Blueve Nov 5, 2020
927bf9b
[connect][clear] Support --devicename option for connect/clear line c…
Blueve Nov 5, 2020
e2ec825
support fwpackage option and fix modular chassis auto-update
sujinmkang Nov 5, 2020
ee6bc49
Copy missing values from INIT_CFG to config_db as part of db_migratio…
vaibhavhd Nov 6, 2020
0d49a85
Replace swsssdk.SonicV2Connector with swsscommon.SonicV2Connector (SW…
qiluo-msft Nov 6, 2020
11526df
add api to check missing api
sujinmkang Nov 6, 2020
673f98d
[sonic_sku_create] Move tests from sonic-utilities-tests/ folder to t…
madhanmellanox Nov 7, 2020
a9cdd27
[show] Fix `show ip bgp sum` (#1194)
lolyu Nov 9, 2020
12e1a0f
[show][techsupport][multi-ASIC] Add support to collect tech support o…
smaheshm Nov 9, 2020
b693cf6
Remove SKU create pytest output directory before execution of the scr…
madhanmellanox Nov 9, 2020
6cbec6b
[multi-ASIC] CRM show/config commands changes for multi-asic (#1127)
judyjoseph Nov 10, 2020
d5eb2f8
[consutil][connect] Remove root need from connect line command (#1225)
Blueve Nov 11, 2020
40377d3
[chassis]: Configure and show for platform chassis_modules (#1145)
mprabhu-nokia Nov 11, 2020
5f42bbe
add and address review comments
sujinmkang Nov 12, 2020
b226159
[CLI][PFCWD] Fix issue with specifying ports in pfcwd start on masic …
smaheshm Nov 12, 2020
dadfc18
Copp Changes (#1004)
Nov 12, 2020
3f15835
[sflow] Add mVRF support (#1230)
GarrickHe Nov 13, 2020
8f156df
Added 'import sys' in feature.py file (#1232)
AkhileshSamineni Nov 13, 2020
599b43a
[doc]: Update Command-Reference.md (#1231)
rdarbha Nov 13, 2020
939e331
[cli]: NAT show commands newline issue after migrated to Python3 (#1204)
AkhileshSamineni Nov 13, 2020
929ff7c
Updates to bgp config and show commands with BGP_INTERNAL_NEIGHBOR ta…
judyjoseph Nov 13, 2020
71aa221
[tests] Do not trim from PATH if we did not append to it; Clean up/fi…
jleveque Nov 13, 2020
f412e52
lgtm
sujinmkang Nov 13, 2020
8079558
Fix mock SonicV2Connector in python3: use decode_responses mode so ca…
qiluo-msft Nov 14, 2020
bd8d6c7
Convert to Python 3 (#1128)
jleveque Nov 15, 2020
f46c27e
Add 'requests' as install dependency in setup.py (#1240)
jleveque Nov 16, 2020
412cfa1
initial commit
sujinmkang Oct 9, 2020
bd32dbe
second commit
sujinmkang Oct 12, 2020
d0fc10c
auto_update_fix
sujinmkang Oct 13, 2020
05dbbb8
reboot_pluging for fwutil auto-update task
sujinmkang Oct 14, 2020
3f8db37
support fwpackage option and fix modular chassis auto-update
sujinmkang Nov 5, 2020
b657538
add api to check missing api
sujinmkang Nov 6, 2020
dfcf286
add and address review comments
sujinmkang Nov 12, 2020
cf81eb6
lgtm
sujinmkang Nov 13, 2020
e54cc58
Merge branch 'fwutil' of https://github.com/sujinmkang/sonic-utilitie…
sujinmkang Nov 16, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 75 additions & 1 deletion fwutil/lib.py
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,7 @@ class PlatformComponentsParser(object):
MODULE_KEY = "module"
COMPONENT_KEY = "component"
FIRMWARE_KEY = "firmware"
UTILITY_KEY = "utility"
VERSION_KEY = "version"

UTF8_ENCODING = "utf-8"
Expand Down Expand Up @@ -538,8 +539,9 @@ def __validate_platform_schema(self, pcp):
pcp.module_component_map
)

def get_status(self):
def get_updates_status(self):
status_table = [ ]
update_status_table = [ ]

append_chassis_name = self.is_chassis_has_components()
append_module_na = not self.is_modular_chassis()
Expand Down Expand Up @@ -583,6 +585,17 @@ def get_status(self):
]
)

update_status_table.append(
[
chassis_name,
module_name,
chassis_component_name,
firmware_path,
firmware_version,
status
]
)

if append_chassis_name:
append_chassis_name = False

Expand Down Expand Up @@ -633,17 +646,41 @@ def get_status(self):
]
)

update_status_table.append(
[
chassis_name,
module_name,
chassis_component_name,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has to be module_component_name instead chassis_component_name.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, fixed in the latest pr.

firmware_path,
firmware_version,
status
]
)

if append_chassis_name:
append_chassis_name = False

if append_module_name:
append_module_name = False

return status_table, update_status_table

def get_status(self):
status_table, update_status_table = self.get_updates_status()
if not status_table:
return None

return tabulate(status_table, self.STATUS_HEADER, tablefmt=self.FORMAT)

def get_update_available_components(self):
update_available_components = []
status_table, update_status_table = self.get_updates_status()
for component_status in update_status_table:
if component_status[-1] is self.FW_STATUS_UPDATE_REQUIRED:
update_available_components.append(component_status)

return update_available_components

def get_notification(self, chassis_name, module_name, component_name):
if self.is_modular_chassis():
component = self.module_component_map[module_name][component_name]
Expand Down Expand Up @@ -695,6 +732,43 @@ def update_firmware(self, chassis_name, module_name, component_name):
log_helper.log_fw_update_end(component_path, firmware_path, False, e)
raise

def auto_update_firmware(self, component, boot):
chassis_name = component[-6]
module_name = component[-5]
component_name = component[-4]

if self.is_modular_chassis():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the module_name is N/A, for modular chassis, it fails since module_component_map is being used

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch! fixed in the latest.

component = self.module_component_map[module_name][component_name]
parser = self.__pcp.module_component_map[module_name][component_name]

component_path = "{}/{}/{}".format(chassis_name, module_name, component_name)
else:
component = self.chassis_component_map[chassis_name][component_name]
parser = self.__pcp.chassis_component_map[chassis_name][component_name]

component_path = "{}/{}".format(chassis_name, component_name)

if not parser:
return

firmware_path = parser[self.__pcp.FIRMWARE_KEY]

if self.__root_path is not None:
firmware_path = self.__root_path + firmware_path

try:
click.echo("Autoupdating firmware for {} with boot action {}".format(component_name, boot))
click.echo(TAB + firmware_path)
log_helper.log_fw_auto_update_start(component_path, firmware_path, boot)
component.auto_update_firmware(firmware_path, boot)
log_helper.log_fw_auto_update_end(component_path, firmware_path, True, boot)
except KeyboardInterrupt:
log_helper.log_fw_auto_update_end(component_path, firmware_path, False, "Keyboard interrupt", boot)
raise
except Exception as e:
log_helper.log_fw_auto_update_end(component_path, firmware_path, False, e, boot)
raise

def is_firmware_update_available(self, chassis_name, module_name, component_name):
if self.is_modular_chassis():
component = self.__pcp.module_component_map[module_name][component_name]
Expand Down
79 changes: 61 additions & 18 deletions fwutil/log.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,12 @@ class LogHelper(object):
FW_ACTION_DOWNLOAD = "download"
FW_ACTION_INSTALL = "install"
FW_ACTION_UPDATE = "update"
FW_ACTION_AUTO_UPDATE = "auto-update"

STATUS_SUCCESS = "success"
STATUS_FAILURE = "failure"

def __log_fw_action_start(self, action, component, firmware):
def __log_fw_action_start(self, action, component, firmware, boot=None):
caption = "Firmware {} started".format(action)
template = "{}: component={}, firmware={}"

Expand All @@ -44,41 +45,77 @@ def __log_fw_action_start(self, action, component, firmware):
)
)

def __log_fw_action_end(self, action, component, firmware, status, exception=None):
def __log_fw_action_end(self, action, component, firmware, status, exception=None, boot=None):
caption = "Firmware {} ended".format(action)

status_template = "{}: component={}, firmware={}, status={}"
status_boot_template = "{}: component={}, firmware={}, boot={}, status={}"
exception_template = "{}: component={}, firmware={}, status={}, exception={}"
exception_boot_template = "{}: component={}, firmware={}, boot={}, status={}, exception={}"

if status:
log.log_info(
status_template.format(
caption,
component,
firmware,
self.STATUS_SUCCESS
)
)
else:
if exception is None:
log.log_error(
if boot is None:
log.log_info(
status_template.format(
caption,
component,
firmware,
self.STATUS_FAILURE
self.STATUS_SUCCESS
)
)
else:
log.log_error(
exception_template.format(
log.log_info(
status_boot_template.format(
caption,
component,
firmware,
self.STATUS_FAILURE,
str(exception)
boot,
self.STATUS_SUCCESS
)
)
else:
if exception:
if boot is None:
log.log_error(
status_template.format(
caption,
component,
firmware,
self.STATUS_FAILURE
)
)
else:
log.log_info(
status_boot_template.format(
caption,
component,
firmware,
boot,
self.STATUS_FAILURE
)
)
else:
if boot is None:
log.log_error(
status_template.format(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has to be exception_template?

caption,
component,
firmware,
self.STATUS_FAILURE,
str(exception)
)
)
else:
log.log_error(
exception_boot_template.format(
caption,
component,
firmware,
boot,
self.STATUS_FAILURE,
str(exception)
)
)

def log_fw_download_start(self, component, firmware):
self.__log_fw_action_start(self.FW_ACTION_DOWNLOAD, component, firmware)
Expand All @@ -98,6 +135,12 @@ def log_fw_update_start(self, component, firmware):
def log_fw_update_end(self, component, firmware, status, exception=None):
self.__log_fw_action_end(self.FW_ACTION_UPDATE, component, firmware, status, exception)

def log_fw_auto_update_start(self, component, firmware, boot):
self.__log_fw_action_start(self.FW_ACTION_AUTO_UPDATE, component, firmware, boot)

def log_fw_auto_update_end(self, component, firmware, status, exception=None, boot=None):
self.__log_fw_action_end(self.FW_ACTION_AUTO_UPDATE, component, firmware, status, exception, boot)

def print_error(self, msg):
click.echo("Error: {}.".format(msg))

Expand Down
44 changes: 44 additions & 0 deletions fwutil/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import click

from lib import PlatformDataProvider, ComponentStatusProvider, ComponentUpdateProvider
#from lib import URL, SquashFs, FwPackage
from lib import URL, SquashFs
from log import LogHelper
except ImportError as e:
Expand Down Expand Up @@ -89,6 +90,14 @@ def update(ctx):
ctx.obj[COMPONENT_PATH_CTX_KEY] = [ ]


# 'auto_update' group
@cli.group()
@click.pass_context
def auto_update(ctx):
"""Auto-update platform firmware"""
pass


def chassis_handler(ctx):
ctx.obj[CHASSIS_NAME_CTX_KEY] = pdp.chassis.get_name()
ctx.obj[COMPONENT_PATH_CTX_KEY].append(pdp.chassis.get_name())
Expand Down Expand Up @@ -334,6 +343,41 @@ def fw_update(ctx, yes, force, image):
cli_abort(ctx, str(e))


# 'fw' subcommand
@auto_update.command(name='fw')
@click.option('-i', '--image', 'image', type=click.Choice(["current", "next"]), default="current", show_default=True, help="Update firmware using current/next SONiC image")
@click.option('-f', '--fw_image', 'fw_image', help="Custom FW package path")
@click.option('-b', '--boot', 'boot', type=click.Choice(["any", "cold", "fast", "warm", "none"]), default="none", show_default=True, help="Necessary boot option after the firmware update")
@click.pass_context
def fw_auto_update(ctx, boot, image=None, fw_image=None):
"""Update firmware from SONiC image"""
component_list = {}
squashfs = None

try:
if image == IMAGE_NEXT:
squashfs = SquashFs()

if squashfs.is_next_boot_set():
fs_path = squashfs.mount_next_image_fs()
cup = ComponentUpdateProvider(fs_path)
else:
log_helper.print_warning("Next boot is set to current: fallback to defaults")
cup = ComponentUpdateProvider()
else:
cup = ComponentUpdateProvider()

component_list = cup.get_update_available_components()
if component_list:
for component in component_list:
cup.auto_update_firmware(component, boot)
else:
log_helper.print_warning("All components: {}".format(cup.FW_STATUS_UP_TO_DATE))
finally:
if squashfs is not None:
squashfs.umount_next_image_fs()


# 'show' subgroup
@cli.group()
def show():
Expand Down