Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Create data disk for external image #123

Merged
merged 9 commits into from
Jun 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,12 @@
ARG DEBIAN_FRONTEND="noninteractive"
ARG DEBCONF_NONINTERACTIVE_SEEN="true"

RUN set -eu && \

Check failure on line 10 in Dockerfile

View workflow job for this annotation

GitHub Actions / Test / shellcheck

SC2035 info: Use ./*glob* or -- *glob* so names with dashes won't become options.
apt-get update && \
apt-get --no-install-recommends -y install \
tini \
wget \
7zip \
fdisk \
nginx \
procps \
seabios \
Expand Down
18 changes: 15 additions & 3 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,9 +270,21 @@ kubectl apply -f kubernetes.yml

### What image formats are supported?

You can set the `BOOT` URL to any `.iso`, `.img`, `.raw`, `.qcow2`, `.vhd`, `.vhdx`, `.vdi` or `.vmdk` file.

It will even automaticly extract compressed images, like `.img.gz`, `.qcow2.xz`, `.iso.zip` and many more!
The `BOOT` URL accepts files in any of the following formats:

| **Extension** | **Format** |
|---|---|
| `.img` | Raw |
| `.raw` | Raw |
| `.iso` | Optical |
| `.qcow2` | QEMU |
| `.vmdk` | VMware |
| `.vhd` | VirtualPC |
| `.vhdx` | Hyper-V |
| `.vdi` | VirtualBox |

> [!TIP]
> It will also accept `.img.gz`, `.qcow2.xz`, `.iso.zip` and many more, as it automaticly extracts compressed files.

## Stars 🌟
[![Stars](https://starchart.cc/qemus/qemu-arm.svg?variant=adaptive)](https://starchart.cc/qemus/qemu-arm)
Expand Down
9 changes: 5 additions & 4 deletions src/boot.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,24 @@ set -Eeuo pipefail

# Docker environment variables
: "${BIOS:=""}" # BIOS file
: "${BOOT_MODE:="legacy"}" # Boot mode

SECURE="off"
BOOT_OPTS=""
BOOT_DESC=""

if [ -n "$BIOS" ]; then
BOOT_MODE="custom"
BOOT_OPTS="-bios $BIOS"
BOOT_DESC=" with custom BIOS file"
return 0
fi

case "${BOOT_MODE,,}" in
"legacy" )
BOOT_OPTS=""
BOOT_DESC=" with SeaBIOS"
;;
"uefi" )
BOOT_DESC=" with OVMF"
"uefi" | "" )
BOOT_MODE="uefi"
ROM="AAVMF_CODE.no-secboot.fd"
VARS="AAVMF_VARS.fd"
;;
Expand Down
39 changes: 20 additions & 19 deletions src/disk.sh
Original file line number Diff line number Diff line change
Expand Up @@ -397,12 +397,11 @@ addMedia () {

local DISK_FILE=$1
local DISK_TYPE=$2
local DISK_BUS=$3
local DISK_INDEX=$4
local DISK_ADDRESS=$5
local DISK_INDEX=$3
local DISK_ADDRESS=$4

local index=""
local DISK_ID="cdrom$DISK_BUS"
local DISK_ID="cdrom$DISK_INDEX"
[ -n "$DISK_INDEX" ] && index=",bootindex=$DISK_INDEX"
local result=" -drive file=$DISK_FILE,id=$DISK_ID,format=raw,cache=unsafe,readonly=on,media=cdrom"

Expand All @@ -417,8 +416,8 @@ addMedia () {
;;
"ide" )
result+=",if=none \
-device ich9-ahci,id=ahci${DISK_BUS},addr=$DISK_ADDRESS \
-device ide-cd,drive=${DISK_ID},bus=ahci${DISK_BUS}.0${index}"
-device ich9-ahci,id=ahci${DISK_INDEX},addr=$DISK_ADDRESS \
-device ide-cd,drive=${DISK_ID},bus=ahci${DISK_INDEX}.0${index}"
echo "$result"
;;
"blk" | "virtio-blk" )
Expand Down Expand Up @@ -529,12 +528,12 @@ addDevice () {
html "Initializing disks..."

[ -z "${DISK_OPTS:-}" ] && DISK_OPTS=""
[ -z "${DISK_NAME:-}" ] && DISK_NAME="data"
[ -z "${DISK_TYPE:-}" ] && DISK_TYPE="scsi"
[ -z "${DISK_NAME:-}" ] && DISK_NAME="data"

case "${DISK_TYPE,,}" in
"ide" | "usb" | "scsi" | "blk" | "auto" ) ;;
* ) error "Invalid DISK_TYPE specified, value \"$DISK_TYPE\" is unrecognized!" && exit 80 ;;
* ) error "Invalid DISK_TYPE specified, value \"$DISK_TYPE\" is not recognized!" && exit 80 ;;
esac

case "${MACHINE,,}" in
Expand All @@ -556,27 +555,30 @@ fi

case "${MEDIA_TYPE,,}" in
"ide" | "usb" | "scsi" | "blk" | "auto" ) ;;
* ) error "Invalid MEDIA_TYPE specified, value \"$MEDIA_TYPE\" is unrecognized!" && exit 80 ;;
* ) error "Invalid MEDIA_TYPE specified, value \"$MEDIA_TYPE\" is not recognized!" && exit 80 ;;
esac

if [ -f "$BOOT" ] && [ -s "$BOOT" ]; then
DISK_OPTS+=$(addMedia "$BOOT" "$MEDIA_TYPE" "0" "$BOOT_INDEX" "0x5")
case "${BOOT,,}" in
*".iso" )
DISK_OPTS+=$(addMedia "$BOOT" "$MEDIA_TYPE" "$BOOT_INDEX" "0x5") ;;
*".img" | *".raw" )
DISK_OPTS+=$(createDevice "$BOOT" "$DISK_TYPE" "$BOOT_INDEX" "0x5" "raw" "$DISK_IO" "$DISK_CACHE") ;;
*".qcow2" )
DISK_OPTS+=$(createDevice "$BOOT" "$DISK_TYPE" "$BOOT_INDEX" "0x5" "qcow2" "$DISK_IO" "$DISK_CACHE") ;;
* )
error "Invalid BOOT image specified, extension \".${BOOT/*./}\" is not recognized!" && exit 80 ;;
esac
fi

DRIVERS="/drivers.iso"
[ ! -f "$DRIVERS" ] || [ ! -s "$DRIVERS" ] && DRIVERS="$STORAGE/drivers.iso"

if [ -f "$DRIVERS" ] && [ -s "$DRIVERS" ]; then
DISK_OPTS+=$(addMedia "$DRIVERS" "$FALLBACK" "1" "" "0x6")
fi

DISK1_FILE="/boot"
if [ ! -f "$DISK1_FILE.img" ] || [ ! -s "$DISK1_FILE.img" ]; then
if [ ! -f "$DISK1_FILE.qcow2" ] || [ ! -s "$DISK1_FILE.qcow2" ]; then
DISK1_FILE="$STORAGE/${DISK_NAME}"
fi
DISK_OPTS+=$(addMedia "$DRIVERS" "$FALLBACK" "" "0x6")
fi

DISK1_FILE="$STORAGE/${DISK_NAME}"
DISK2_FILE="/storage2/${DISK_NAME}2"
DISK3_FILE="/storage3/${DISK_NAME}3"
DISK4_FILE="/storage4/${DISK_NAME}4"
Expand Down Expand Up @@ -648,5 +650,4 @@ fi
DISK_OPTS+=" -object iothread,id=io2"

html "Initialized disks successfully..."

return 0
79 changes: 28 additions & 51 deletions src/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,46 +10,24 @@ detectType() {
[ ! -s "$file" ] && return 1

case "${file,,}" in
*".iso" )

BOOT="$file"
[ -n "${BOOT_MODE:-}" ] && return 0

# Automaticly detect UEFI-compatible ISO's
dir=$(isoinfo -f -i "$file")
[ -z "$dir" ] && error "Failed to read ISO file, invalid format!" && BOOT="" && return 1

dir=$(echo "${dir^^}" | grep "^/EFI")
[ -n "$dir" ] && BOOT_MODE="uefi"
;;

*".img" )

DISK_NAME=$(basename "$file")
DISK_NAME="${DISK_NAME%.*}"
[ -n "${BOOT_MODE:-}" ] && return 0

# Automaticly detect UEFI-compatible images
dir=$(sfdisk -l "$file")
[ -z "$dir" ] && error "Failed to read IMG file, invalid format!" && DISK_NAME="" && return 1

dir=$(echo "${dir^^}" | grep "EFI SYSTEM")
[ -n "$dir" ] && BOOT_MODE="uefi"
;;
*".iso" | *".img" | *".raw" | *".qcow2" )
BOOT="$file" ;;
* ) return 1 ;;
esac

*".qcow2" )
[ -n "$BOOT_MODE" ] && return 0
[[ "${file,,}" != *".iso" ]] && return 0

DISK_NAME=$(basename "$file")
DISK_NAME="${DISK_NAME%.*}"
[ -n "${BOOT_MODE:-}" ] && return 0
# Automaticly detect UEFI-compatible ISO's
dir=$(isoinfo -f -i "$file")

# TODO: Detect boot mode from partition table in image
BOOT_MODE="uefi"
;;
if [ -z "$dir" ]; then
BOOT=""
error "Failed to read ISO file, invalid format!" && return 1
fi

* )
return 1 ;;
esac
dir=$(echo "${dir^^}" | grep "^/EFI")
[ -z "$dir" ] && BOOT_MODE="legacy"

return 0
}
Expand Down Expand Up @@ -105,12 +83,13 @@ convertImage() {
local source_fmt=$2
local dst_file=$3
local dst_fmt=$4
local dir base fs fa cur_size src_size space disk_param
local dir base fs fa space
local cur_size src_size disk_param

[ -f "$dst_file" ] && error "Conversion failed, destination file $dst_file already exists?" && return 1
[ ! -f "$source_file" ] && error "Conversion failed, source file $source_file does not exists?" && return 1

if [[ "$source_fmt" == "raw" ]] && [[ "$dst_fmt" == "raw" ]]; then
if [[ "${source_fmt,,}" == "${dst_fmt,,}" ]]; then
mv -f "$source_file" "$dst_file"
return 0
fi
Expand Down Expand Up @@ -181,7 +160,6 @@ convertImage() {
fi

html "Conversion completed..."

return 0
}

Expand All @@ -199,6 +177,7 @@ findFile() {

findFile "iso" && return 0
findFile "img" && return 0
findFile "raw" && return 0
findFile "qcow2" && return 0

if [ -z "$BOOT" ] || [[ "$BOOT" == *"example.com/image.iso" ]]; then
Expand All @@ -212,23 +191,24 @@ base=$(echo "$base" | sed -e 's/[^A-Za-z0-9._-]/_/g')

case "${base,,}" in

*".iso" | *".img" | *".qcow2" )
*".iso" | *".img" | *".raw" | *".qcow2" )

detectType "$STORAGE/$base" && return 0 ;;

*".raw" | *".vdi" | *".vmdk" | *".vhd" | *".vhdx" )
*".vdi" | *".vmdk" | *".vhd" | *".vhdx" )

detectType "$STORAGE/${base%.*}.img" && return 0
detectType "$STORAGE/${base%.*}.qcow2" && return 0 ;;

*".gz" | *".gzip" | *".xz" | *".7z" | *".zip" | *".rar" | *".lzma" | *".bz" | *".bz2" )

case "${base%.*}" in
*".iso" | *".img" | *".qcow2" )

*".iso" | *".img" | *".raw" | *".qcow2" )

detectType "$STORAGE/${base%.*}" && return 0 ;;

*".raw" | *".vdi" | *".vmdk" | *".vhd" | *".vhdx" )
*".vdi" | *".vmdk" | *".vhd" | *".vhdx" )

find="${base%.*}"

Expand All @@ -237,8 +217,7 @@ case "${base,,}" in

esac ;;

* )
error "Unknown file format, extension \".${base/*./}\" is not recognized!" && exit 33 ;;
* ) error "Unknown file extension, type \".${base/*./}\" is not recognized!" && exit 33 ;;
esac

if ! downloadFile "$BOOT" "$base"; then
Expand Down Expand Up @@ -288,7 +267,7 @@ case "${base,,}" in
esac

case "${base,,}" in
*".iso" | *".img" | *".qcow2" )
*".iso" | *".img" | *".raw" | *".qcow2" )
detectType "$STORAGE/$base" && return 0
error "Cannot read file \"${base}\"" && exit 63 ;;
esac
Expand All @@ -299,13 +278,11 @@ target_fmt="${DISK_FMT:-}"
[[ "$target_fmt" != "raw" ]] && target_ext="qcow2"

case "${base,,}" in
*".raw" ) source_fmt="raw" ;;
*".vdi" ) source_fmt="vdi" ;;
*".vhd" ) source_fmt="vhd" ;;
*".vhd" ) source_fmt="vpc" ;;
*".vhdx" ) source_fmt="vpc" ;;
*".vmdk" ) source_fmt="vmdk" ;;
*".vhdx" ) source_fmt="vhdx" ;;
* )
error "Unknown file format, extension \".${base/*./}\" is not recognized!" && exit 33 ;;
* ) error "Unknown file extension, type \".${base/*./}\" is not recognized!" && exit 33 ;;
esac

dst="$STORAGE/${base%.*}.$target_ext"
Expand Down
27 changes: 17 additions & 10 deletions src/reset.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ echo "❯ For support visit $SUPPORT"
: "${RAM_SIZE:="1G"}" # Maximum RAM amount
: "${RAM_CHECK:="Y"}" # Check available RAM
: "${DISK_SIZE:="16G"}" # Initial data disk size
: "${BOOT_INDEX:="10"}" # Boot index of CD drive
: "${BOOT_MODE:=""}" # Boot system with UEFI
: "${BOOT_INDEX:="9"}" # Boot index of CD drive

# Helper variables

Expand Down Expand Up @@ -81,13 +82,18 @@ SPACE_GB=$(( (SPACE + 1073741823)/1073741824 ))
echo "❯ CPU: ${CPU} | RAM: $AVAIL_GB/$TOTAL_GB GB | DISK: $SPACE_GB GB (${FS}) | HOST: ${SYS}..."
echo

# Check compatibilty

if [[ "${FS,,}" == "ecryptfs" ]] || [[ "${FS,,}" == "tmpfs" ]]; then
DISK_IO="threads"
DISK_CACHE="writeback"
fi

# Check memory

if [[ "$RAM_CHECK" != [Nn]* ]]; then
if (( (RAM_WANTED + RAM_SPARE) > RAM_AVAIL )); then
error "Your configured RAM_SIZE of $WANTED_GB GB is too high for the $AVAIL_GB GB of memory available, please set a lower value."
exit 17
fi
if [[ "$RAM_CHECK" != [Nn]* ]] && (( (RAM_WANTED + RAM_SPARE) > RAM_AVAIL )); then
error "Your configured RAM_SIZE of $WANTED_GB GB is too high for the $AVAIL_GB GB of memory available, please set a lower value."
exit 17
fi

# Helper functions
Expand Down Expand Up @@ -194,13 +200,14 @@ addPackage() {

hasDisk() {

[ -b "/disk" ] && return 0
[ -b "/disk1" ] && return 0
[ -b "/dev/disk1" ] && return 0
[ -s "/boot.img" ] && return 0
[ -s "/boot.qcow2" ] && return 0
[ -b "${DEVICE:-}" ] && return 0
[ -s "$STORAGE/data.img" ] && return 0
[ -s "$STORAGE/data.qcow2" ] && return 0

[ -z "${DISK_NAME:-}" ] && DISK_NAME="data"
[ -s "$STORAGE/$DISK_NAME.img" ] && return 0
[ -s "$STORAGE/$DISK_NAME.qcow2" ] && return 0

return 1
}
Expand Down