Rubik Pi Image Builder #70
Open
+313
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added a pre-processing step to download and extract the rubikpi3 image if needed. The workflow now checks for the presence of the .img.xz file and sets the RUBIKPI3_IMAGE_PATH environment variable accordingly. Updated the base_image parameter to use this path when applicable.
…eps for limelight and opi5 variants. Updated rubikpi3 base image URL and streamlined the base_image parameter handling.
…oved performance and remove unnecessary whitespace
This reverts commit 02cac54.
…age preparation step and adding debugging commands for partition layout. Updated the handling of the base_image parameter and ensured the script execution order is maintained.
… build process with specific mounting and chroot environment setup. Removed obsolete debugging commands and streamlined image handling for different configurations.
…ting process. Removed the explicit loop device setup and allowed the mount command to create the loop device automatically, enhancing clarity and efficiency.
…ing the repository into the chroot environment, improving efficiency and excluding unnecessary files and directories.
- Replace direct image download with Canonical YAML manifest processing for rubikpi3 - Implement artifact assembly via yq parsing, SHA verification, and .gz→.xz conversion - Remove obsolete opi5 target and associated arm-runner dependency - Streamline build process using official Canonical image components (20250912)
- Replace multi‑step `yq` parsing with a single `yq eval -r` command that outputs URL and SHA256 directly. - Derive the downloaded filename using `basename` instead of extracting a `content` field. - Remove the previous `.gz` → `.xz` rename logic. - Add conditional handling for `.xz` files and `.tar.gz` archives, using `unxz` and `tar -xzf` only when such files exist. - Make file moves into the `QLI` directory tolerant (`|| true`) to avoid failures when files are absent. - Overall, the workflow script is now shorter, clearer, and more resilient to missing or differently‑named artifacts.
…d checksum Update the `yq` expression in the GitHub Actions workflow to use double‑quoted strings and explicit concatenation. This ensures proper handling of URLs and SHA256 sums, avoiding quoting issues that broke component processing.
Updated the `yq` invocation in `.github/workflows/main.yml` to use single‑quoted syntax (`'.urls[] | .url + " " + .sha256sum'`) instead of double‑quoted with escaped characters. This prevents quoting errors and ensures the command works reliably across different shell environments.
Update the awk commands in the CI workflow to match `url:` and `sha256sum:` lines with optional leading whitespace. The new regex (`^[[:space:]]*url:` and `^[[:space:]]*sha256sum:`) and corresponding `gsub` calls strip any indentation before extracting the values, ensuring the manifest is processed correctly even when fields are indented. This prevents failures when parsing manifests with formatted spacing.
Update the awk script in the GitHub Actions workflow to more reliably extract `url` and `sha256sum` fields from the manifest. The new pattern handles list prefixes and simplifies the regex, ensuring correct parsing of component entries regardless of leading spaces or formatting variations. This makes the CI download and checksum verification step more robust.
Remove the previous step that created and compressed a full image, as only the QLI directory is required. Add a new step to tar and gzip the entire QLI directory and update the upload‑artifact configuration accordingly. This simplifies the workflow and reduces unnecessary processing.
- Replace simple image extraction with proper partition detection and mounting - Add dynamic partition offset calculation using sfdisk/fdisk - Implement chroot environment with qemu-user-static for ARM emulation - Expand rootfs image by 3800MB to accommodate PhotonVision and dependencies - Add filesystem resizing after image expansion - Mount proc, sys, dev, and run for proper chroot operation - Copy build files into chroot using rsync with appropriate exclusions - Update install script to handle in-chroot execution context This enables proper package installation and configuration within the target ARM rootfs environment during the build process.
Enhance the partition detection logic to properly handle GPT (GUID Partition Table) disk images by: - Skip protective MBR partitions (type=ee) when detecting rootfs - Add fallback to fdisk-based detection if sfdisk fails - Sort partitions by size to find the largest non-protective partition - Add debug output to trace partition detection process - Improve error handling for edge cases This fixes issues with mounting rootfs from GPT-formatted images where the protective MBR was incorrectly selected as the target partition.
- Add gdisk and parted packages to workflow dependencies - Use sgdisk to fix GPT partition table after image expansion - Simplify partition detection logic using fdisk output directly - Remove complex sfdisk parsing in favor of more reliable fdisk method - Improve error handling and debug output for partition detection - Handle boot flag (*) in fdisk output when extracting start sector
The workflow script now uses a more robust `awk`‑based method to locate partition lines, handling both MBR and GPT layouts. It removes the previous EFI‑specific filter, adds detailed error output when no partition is found, and implements safer start‑sector extraction with a fallback parsing step. This enhances reliability of partition detection across different image formats.
Replace the fragile `fdisk` text‑parsing logic with `sfdisk`’s machine‑readable output, adding a `parted` fallback. This makes partition start‑sector detection more reliable, especially for GPT images.
|
~~We gotta figure out some sort of way to mount the image for copying the jar in the monorepo. This might be a little tricky, as we currently use an armrunner for doing this. We'd also have to extract everything, and then repackage it, since we're shipping a tar archive instead of just an image. It's doable, but it'll likely take some customization and I don't love having to do something outside of the matrix like this. Edit: I've done some more thunking, and I'm working on setting up Seems to be done, bit more testing happening now |
samfreund
force-pushed
the
rubik-pi-image
branch
from
669faee to
436465b
Compare
samfreund
commented
Oct 8, 2025
This comment was marked as resolved.
This comment was marked as resolved.
7 tasks
|
Bro left the MR description totally blank (skull) |
Idk what you're talking about.... |
mcm001
reviewed
Oct 9, 2025
crschardt
reviewed
Oct 9, 2025
Co-authored-by: Craig Schardt <crschardt@fastem.com>
crschardt
reviewed
Oct 9, 2025
crschardt
reviewed
Oct 9, 2025
crschardt
reviewed
Oct 9, 2025
Co-authored-by: Craig Schardt <crschardt@fastem.com>
samfreund
force-pushed
the
rubik-pi-image
branch
from
8a44a87 to
766166c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Build an image for the Rubik Pi 3
This has some unique quirks as the image is actually a tarball, consisting of various firmware files that get flashed as well. This is one of the reasons it can't get built in the matrix alongside our other images. Another reason is that the image requires an offset when mounting it, which isn't supported by the armrunner we use for our other images (trust me, I checked (might add support for that later and we can look at adding it to the matrix)).
Also note, the
mount_rubikpi3.shscript is designed to run an arbitrary script. This is by design, so we can reuse it in the monorepo.