Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add firmware_update phase support for HPE Gen10/Gen11/RL machines #255

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Add firmware_update phase support for HPE Gen10/Gen11/RL machines #255

wants to merge 10 commits into from

Conversation

nancyc12
Copy link
Contributor

@nancyc12 nancyc12 commented Apr 16, 2024 •
edited
Loading

Description

Based on CR033 - Firmware updates for regression testing, add firmware_update phase support for HPE server machines utilising ilorest (apt install from HPE repository) and the firmware files downloaded from HPE firmware repository.

HPE firmware repository index files (fwrepo.json) on HPE firmware repository have been updated to support multiple targets (hardware components) per firmware file, after communicating with HPE.

This PR also includes these changes:

  • move check_connectable implementation from LVFSDevice to AbstractDevice
  • introduce FirmwareUpdateError
  • remove password from AbstractDevice.__init__ as device connector use SSH key for DUT connection
  • move OEMDevice to base.py

Resolved issues

N/A

Documentation

test-phases.rst updated accordingly.

Tests

  • add unit test for HPE.py
  • validate the code change didn't impact the functionality for PC device
  • run the agent in Server Lab environment
************************************************************

* Starting testflinger firmware_update phase on doubletusk *

************************************************************
2024-04-16 08:41:14,964 doubletusk INFO: DEVICE CONNECTOR: BEGIN firmware_update
2024-04-16 08:41:14,965 doubletusk INFO: DEVICE CONNECTOR: check and wait for 60s until 10.245.130.29 is SSHable
2024-04-16 08:41:20,970 doubletusk INFO: DEVICE CONNECTOR: 10.245.130.29 is SSHable after 6s
2024-04-16 08:41:21,528 doubletusk INFO: DEVICE CONNECTOR: 10.245.130.29 is a HPE HPEDevice
2024-04-16 08:41:30,833 doubletusk INFO: DEVICE CONNECTOR: successfully installed RESTful Interface Tool 4.2.0.0
2024-04-16 08:42:10,794 doubletusk INFO: DEVICE CONNECTOR: HPE firmware repository: https://downloads.linux.hpe.com/SDR/repo/fwpp-gen10/
2024-04-16 08:42:17,356 doubletusk INFO: DEVICE CONNECTOR: Available firmware options: ['2023.09.00.04', '2023.09.00.03', '2023.09.00.02', '2023.09.00.01', '2023.09.00.00', '2023.03.00.00', '2022.09.1', '2022.03.1', '2022.03.0_supspp_rhel9.0_x86_64', '2022.03.0', '2021.10.0', '2021.04.0', '2020.09.0_hotfix_1', '2020.09.0', '2020.03.2', '2020.03.0_hotfix_1', '2020.03.0', '2019.12.0_hotfix_1', '2019.12.0']
2024-04-16 08:42:22,543 doubletusk INFO: DEVICE CONNECTOR: start flashing all firmware with files in SPP 2023.09.00.04
2024-04-16 08:42:26,814 doubletusk INFO: DEVICE CONNECTOR: firmware updatable on HPE machine:
[{'Firmware Name': 'System ROM', 'Firmware Version': 'U30 v3.00 (10/19/2023)', 'Location': 'System Board', 'Fwpkg Available': {'2023.09.00.04': {'file': 'U30_3.00_10_19_2023.fwpkg', 'date': '20231027', 'description': 'ROM Flash Firmware Package - HPE ProLiant DL380 Gen10 (U30) Servers', 'deviceclass': 'aa148d2e-6e09-453e-bc6f-63baa5f5ccc4', 'minimum_active_version': 'null', 'reboot_required': 'yes', 'target': ['00000000-0000-0000-0000-000000000205'], 'version': '3.00_10-19-2023'}}, 'Update Order': 0, 'Device Class': 'aa148d2e-6e09-453e-bc6f-63baa5f5ccc4', 'Target ID': ['00000000-0000-0000-0000-000000000205', '00000000-0000-0000-0000-000001553330']}, {'Firmware Name': 'Server Platform Services (SPS) Firmware', 'Firmware Version': '4.1.5.2', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 1, 'Device Class': 'b34e5677-21dc-45d3-872b-42f76fee9053', 'Target ID': ['00000000-0000-0000-0000-000000000205', 'a6b1a447-382a-5a4f-3c10-86800a000101']}, {'Firmware Name': 'Server Platform Services (SPS) Descriptor', 'Firmware Version': '1.2 0', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 1, 'Device Class': '3f30a329-d03c-46b8-8f0e-9567fad4ea9f', 'Target ID': ['00000000-0000-0000-0000-000000000205']}, {'Firmware Name': 'Innovation Engine (IE) Firmware', 'Firmware Version': '0.2.3.0', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 2, 'Device Class': 'c734e171-8721-48c9-9ed6-d5bc7da5ef8d', 'Target ID': ['00000000-0000-0000-0000-000000000205', 'a6b1a447-382a-5a4f-3c10-3c100a000303']}, {'Firmware Name': 'iLO 5', 'Firmware Version': '3.00 Dec 14 2023', 'Location': 'System Board', 'Fwpkg Available': {'2023.09.00.04': {'file': 'ilo5_301.fwpkg', 'date': '20240123', 'description': 'Online ROM Flash Firmware Package - HPE Integrated Lights-Out 5', 'deviceclass': '2f317b9d-c9e3-4d76-bff6-b9d0d085a952', 'minimum_active_version': 'null', 'reboot_required': 'no', 'target': ['9c057a7a-0d48-4260-b033-558f22ef7c6a', 'c5b14abd-6d2f-41d7-b665-0447dfdac3a7', 'c0bcf2b9-1141-49af-aab8-c73791f0349c'], 'version': '3.01'}}, 'Update Order': 3, 'Device Class': '2f317b9d-c9e3-4d76-bff6-b9d0d085a952', 'Target ID': ['4764a662-b342-4fc7-9ce9-258c5d99e815', 'c0bcf2b9-1141-49af-aab8-c73791f0349c']}, {'Firmware Name': 'Redundant System ROM', 'Firmware Version': 'U30 v2.90 (07/20/2023)', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 4, 'Device Class': '', 'Target ID': ''}, {'Firmware Name': 'Intelligent Provisioning', 'Firmware Version': '3.50.100', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 4, 'Device Class': '', 'Target ID': ''}, {'Firmware Name': 'Power Management Controller FW Bootloader', 'Firmware Version': '1.1', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 4, 'Device Class': '', 'Target ID': ''}, {'Firmware Name': 'Intelligent Platform Abstraction Data', 'Firmware Version': '16.5.0 Build 53', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': 'b8f46d06-85db-465c-94fb-d106e61378ed', 'Target ID': ['00000000-0000-0000-0000-000000000205', '00000000-0000-0000-0000-000001553330']}, {'Firmware Name': 'System Programmable Logic Device', 'Firmware Version': '0x31', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': 'b1ad439a-9dd1-41c1-a496-2da9313f1f07', 'Target ID': ['00000000-0000-0000-0000-000000000205']}, {'Firmware Name': 'Power Management Controller Firmware', 'Firmware Version': '1.1.0', 'Location': 'System Board', 'Fwpkg Available': {'2023.09.00.04': {'file': 'PICGen10-1.1.0-1.fwpkg', 'date': '20231022', 'description': 'ROM Flash Firmware Package - Advanced Power Capping Microcontroller Firmware for HPE Gen10 and Gen10 Plus Servers', 'deviceclass': '9e48a28a-586c-4519-8405-a04f84e27f0f', 'minimum_active_version': 'null', 'reboot_required': 'yes', 'target': ['00000000-0000-0000-0000-000000504d05'], 'version': '1.1.0'}}, 'Update Order': 5, 'Device Class': '9e48a28a-586c-4519-8405-a04f84e27f0f', 'Target ID': ['00000000-0000-0000-0000-000000000205', '00000000-0000-0000-0000-000000504d05']}, {'Firmware Name': 'NVMe Backplane Firmware', 'Firmware Version': '1.24', 'Location': 'System Board', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': '3653aa90-7089-453a-899c-792827a50d67', 'Target ID': ['00000000-0000-0000-0000-000000000205', '00000000-0000-0000-0000-000049535320']}, {'Firmware Name': 'Power Supply Firmware', 'Firmware Version': '2.00', 'Location': 'Bay 1', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': '6bb86077-0275-4614-aae1-86618e8b1c27', 'Target ID': ['0302b505-0002-0000-0000-0cf38db966ea']}, {'Firmware Name': 'Power Supply Firmware', 'Firmware Version': '2.00', 'Location': 'Bay 2', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': '6bb86077-0275-4614-aae1-86618e8b1c27', 'Target ID': ['0302b505-0002-0000-0000-0cf38db966ea']}, {'Firmware Name': 'HPE Eth 10/25Gb 2p 640FLR-SFP28 Adptr', 'Firmware Version': '14.32.1010', 'Location': 'Embedded ALOM', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': None, 'Target ID': ['a6b1a447-382a-5a4f-15b3-1015159000d3']}, {'Firmware Name': 'HPE Smart Array E208i-a SR Gen10', 'Firmware Version': '6.22', 'Location': 'Embedded RAID', 'Fwpkg Available': {'2023.09.00.04': {'file': 'HPE_SR_Gen10_6.52_A.fwpkg', 'date': '20231031', 'description': 'Firmware Package  - HPE Smart Array P408i-p, P408e-p, P408i-a, P408i-c, E208i-p, E208e-p, E208i-c, E208i-a, P408e-m, P204i-c, P416ie-m and P816i-a SR Gen10 and Gen11 controllers', 'deviceclass': '79f0c163-0c13-4662-9dea-09235fef90cb', 'minimum_active_version': '5.61', 'reboot_required': 'yes', 'target': ['a6b1a447-382a-5a4f-9005-028f103c1100', 'a6b1a447-382a-5a4f-9005-028f103c1101', 'a6b1a447-382a-5a4f-9005-028f103c0601', 'a6b1a447-382a-5a4f-9005-028f103c0602', 'a6b1a447-382a-5a4f-9005-028f103c0603', 'a6b1a447-382a-5a4f-9005-028f103c0600', 'a6b1a447-382a-5a4f-9005-028f103c0700', 'a6b1a447-382a-5a4f-9005-028f103c0651', 'a6b1a447-382a-5a4f-9005-028f103c0654', 'a6b1a447-382a-5a4f-9005-028f103c0652', 'a6b1a447-382a-5a4f-9005-028f103c0650', 'a6b1a447-382a-5a4f-9005-028f15900382', 'a6b1a447-382a-5a4f-9005-028f15900383'], 'version': '6.52'}}, 'Update Order': 5, 'Device Class': None, 'Target ID': ['a6b1a447-382a-5a4f-9005-028f103c0654']}, {'Firmware Name': 'NVIDIA Tesla T4', 'Firmware Version': '90.04.96.00.01', 'Location': 'PCI-E Slot 2', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': None, 'Target ID': ['a6b1a447-382a-5a4f-10de-1eb810de12a2']}, {'Firmware Name': 'NVIDIA Tesla T4', 'Firmware Version': '90.04.96.00.01', 'Location': 'PCI-E Slot 3', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': None, 'Target ID': ['a6b1a447-382a-5a4f-10de-1eb810de12a2']}, {'Firmware Name': 'NVIDIA Tesla T4', 'Firmware Version': '90.04.96.00.01', 'Location': 'PCI-E Slot 5', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': None, 'Target ID': ['a6b1a447-382a-5a4f-10de-1eb810de12a2']}, {'Firmware Name': 'NVIDIA Tesla T4', 'Firmware Version': '90.04.96.00.01', 'Location': 'PCI-E Slot 6', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': None, 'Target ID': ['a6b1a447-382a-5a4f-10de-1eb810de12a2']}, {'Firmware Name': 'Embedded Video Controller', 'Firmware Version': '2.5', 'Location': 'Embedded Device', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': None, 'Target ID': ['a6b1a447-382a-5a4f-102b-0538159000e4']}, {'Firmware Name': 'NVMe Drive', 'Firmware Version': 'EDA5302Q', 'Location': 'NVMe Drive Port 5B Box 1 Bay 1', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': 'f6de0320-2e0f-489a-b238-6dd8ae7c3811', 'Target ID': ['532340a5-6d61-7573-6e67-20532a9e3acf']}, {'Firmware Name': 'NVMe Drive', 'Firmware Version': 'EDA5302Q', 'Location': 'NVMe Drive Port 5B Box 1 Bay 2', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': 'f6de0320-2e0f-489a-b238-6dd8ae7c3811', 'Target ID': ['532340a5-6d61-7573-6e67-20532a9e3acf']}, {'Firmware Name': 'HPE 2SFF NVMe/SAS/uFF Backplane', 'Firmware Version': '1.24', 'Location': 'Box=1', 'Fwpkg Available': {}, 'Update Order': 5, 'Device Class': '4149ebb7-d64b-4654-b679-4280f349ea50', 'Target ID': ['9675b54e-1c96-44c3-a7a1-901500000000']}]
2024-04-16 08:42:29,623 doubletusk INFO: DEVICE CONNECTOR: [System ROM] no update is needed, already U30 v3.00 (10/19/2023)
2024-04-16 08:42:29,623 doubletusk INFO: DEVICE CONNECTOR: [iLO 5] update current firmware 3.00 Dec 14 2023 to 3.01
2024-04-16 08:42:29,623 doubletusk INFO: DEVICE CONNECTOR: ilo5_301.fwpkg is already downloaded
2024-04-16 08:42:29,623 doubletusk INFO: DEVICE CONNECTOR: [Power Management Controller Firmware] no update is needed, already 1.1.0
2024-04-16 08:42:29,623 doubletusk INFO: DEVICE CONNECTOR: [HPE Smart Array E208i-a SR Gen10] update current firmware 6.22 to 6.52
2024-04-16 08:42:29,623 doubletusk INFO: DEVICE CONNECTOR: HPE_SR_Gen10_6.52_A.fwpkg is already downloaded
2024-04-16 08:42:29,623 doubletusk INFO: DEVICE CONNECTOR: check and clear iLO taskqueue
2024-04-16 08:42:50,030 doubletusk INFO: DEVICE CONNECTOR: start flashing ilo5_301.fwpkg
2024-04-16 08:46:44,093 doubletusk INFO: DEVICE CONNECTOR: Uploading firmware: ilo5_301.fwpkg
Successfully checked 'ilo5_301.fwpkg'.
Uploading component ilo5_301.fwpkg.
[200] The operation completed successfully.
Component ilo5_301.fwpkg uploaded successfully.
Waiting for iLO UpdateService to finish processing the component
00 hour(s) 04 minute(s) 38 second(s) 
Firmware has successfully been flashed.
iLO will reboot to complete flashing. Session will be terminated.
2024-04-16 08:46:44,094 doubletusk INFO: DEVICE CONNECTOR: wait until iLO complete reboot
2024-04-16 08:47:43,454 doubletusk INFO: DEVICE CONNECTOR: start flashing HPE_SR_Gen10_6.52_A.fwpkg
2024-04-16 08:49:47,111 doubletusk INFO: DEVICE CONNECTOR: Uploading firmware: HPE_SR_Gen10_6.52_A.fwpkg
Successfully checked 'HPE_SR_Gen10_6.52_A.fwpkg'.
Uploading component HPE_SR_Gen10_6.52_A.fwpkg.
[200] The operation completed successfully.
Component HPE_SR_Gen10_6.52_A.fwpkg uploaded successfully.
Waiting for iLO UpdateService to finish processing the component
00 hour(s) 02 minute(s) 44 second(s) 
Firmware has successfully been flashed and a reboot is required for this firmware to take effect.
2024-04-16 08:49:47,113 doubletusk INFO: DEVICE CONNECTOR: check iLO access
2024-04-16 08:50:09,507 doubletusk INFO: DEVICE CONNECTOR: iLO connection checked
2024-04-16 08:50:09,507 doubletusk INFO: DEVICE CONNECTOR: reboot DUT
2024-04-16 08:59:02,384 doubletusk INFO: DEVICE CONNECTOR: HPE machine reaches FinishedPost after 512s
2024-04-16 08:59:12,393 doubletusk INFO: DEVICE CONNECTOR: check and wait for 1800s until 10.245.130.29 is SSHable
2024-04-16 08:59:18,463 doubletusk INFO: DEVICE CONNECTOR: 10.245.130.29 is SSHable after 6s
2024-04-16 08:59:35,577 doubletusk INFO: DEVICE CONNECTOR: [iLO 5] firmware flashed 3.00 Dec 14 2023 → 3.01 Jan 23 2024
2024-04-16 08:59:35,578 doubletusk INFO: DEVICE CONNECTOR: [HPE Smart Array E208i-a SR Gen10] firmware flashed 6.22 → 6.52
2024-04-16 08:59:41,533 doubletusk INFO: DEVICE CONNECTOR: END firmware_update

@nancyc12 nancyc12 mentioned this pull request Apr 16, 2024
Closed
3 tasks
@nancyc12 nancyc12 marked this pull request as ready for review April 19, 2024 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@nancyc12