Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCB - docker-in-docker build #202

Closed
wants to merge 4 commits into from
Closed

TCB - docker-in-docker build #202

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b5a6787
abs-path for tcb-build
DiarmuidKelly May 22, 2024
b4547e4
ubuntu version, check if working_directory is set
DiarmuidKelly May 22, 2024
5c5085b
install docker and powershell
DiarmuidKelly May 22, 2024
d4de384
changes to platform push
DiarmuidKelly Jun 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions scripts/bash/tcb-env-setup.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ tcb_env_setup_cleanup () {
unset under_windows
unset user_tag
unset storage
unset working_directory
unset volumes
unset network
unset remote_tags
Expand Down Expand Up @@ -95,6 +96,11 @@ tcb_env_setup_usage () {
echo " It must be an absolute directory or a Docker volume name. If this"
echo " flag is not set, the \"storage\" Docker volume will be used."
echo ""
echo " -wd: select working directory for docker mount to /workdir"
echo " Pass the directory explicitly for docker run torizon/torizoncore-builder"
echo " to mount as /workdir."
echo " It must be an absolute directory. If this is not set, \$(pwd) with be used"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$(pwd) with be used

I think this was a typo with should be will, right?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes ..will be used

echo ""
echo " -n: do not enable \"host\" network mode."
echo " Under Linux the tool runs in \"host\" network mode by default allowing"
echo " it to operate as a server without explicit port publishing. Under"
Expand Down Expand Up @@ -133,6 +139,7 @@ fi
# Parse flags
volumes=" -v /deploy "
storage="storage"
working_directory="working_directory"
network=" --network=host "
if [ $under_windows = "1" ]; then
# Do not use "host" network mode under Windows/WSL
Expand All @@ -144,6 +151,7 @@ do
-a) source=$2;[ "$2" ]||source="empty"; shift; shift;;
-t) user_tag="$2";[ "$2" ]||user_tag="empty"; shift; shift;;
-s) storage="$2";[ "$2" ]||storage="empty"; shift; shift;;
-wd) working_directory="$2";[ "$2" ]||working_directory=$(pwd); shift; shift;;
-d) volumes=" "; shift;;
-n) network=" "; shift;;
--) shift; break;;
Expand All @@ -162,7 +170,7 @@ then
tcb_env_setup_check_updated $SCRIPT_PATH
fi

if [[ $source = "empty" ]] || [[ $user_tag = "empty" ]] || [[ $storage = "empty" ]]
if [[ $source = "empty" ]] || [[ $user_tag = "empty" ]] || [[ $storage = "empty" ]] || [[ $working_directory ]]
then
tcb_env_setup_usage
tcb_env_setup_cleanup
Expand Down Expand Up @@ -293,7 +301,7 @@ function tcb_dynamic_params() {
# TODO Not compatible with ZSH
export -f tcb_dynamic_params

alias torizoncore-builder='docker run --rm'"$volumes"'-v "$(pwd)":/workdir -v '"$storage"':/storage -v /var/run/docker.sock:/var/run/docker.sock'"$network"'$(tcb_dynamic_params) '"$*"' torizon/torizoncore-builder:'"$chosen_tag"
alias torizoncore-builder='docker run --rm'"$volumes"' -v '"$working_directory"':/workdir -v '"$storage"':/storage -v /var/run/docker.sock:/var/run/docker.sock'"$network"' '"$(tcb_dynamic_params)"' '"$*"' torizon/torizoncore-builder:'"$chosen_tag"

[[ $storage =~ ^[a-zA-Z][a-zA-Z0-9_.-]*$ ]] && storage="Docker volume named '$storage'"

Expand Down
13 changes: 12 additions & 1 deletion scripts/tasks.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -649,9 +649,20 @@ function getCliInputs () {
# set the relative workspaceFolder (following the pattern that VS Code expects)
if (
($null -eq $env:APOLLOX_WORKSPACE) -and
($env:APOLLOX_CONTAINER -ne 1)
($env:APOLLOX_CONTAINER -ne 1) -and
(!$env:GITHUB_WORKSPACE)
) {
$Global:workspaceFolder = Join-Path $PSScriptRoot ..
}
elseif (
$env:GITHUB_WORKSPACE
) {
# If running in a github action, this ENV is set automatically,
# github actions sandboxes runners by re-writing their ABS path,
# so when we do a docker-in-docker build, the mount point for workdir is not found.
# This sets the ABS path from a file called abs-path
$Global:workspaceFolder = Join-Path $PSScriptRoot ..
$Global:working_directory = Get-Content -Path abs-path -ReadCount 1
} else {
$Global:workspaceFolder = $env:APOLLOX_WORKSPACE
}
Expand Down
14 changes: 14 additions & 0 deletions tcb/.github/workflows/build-torizoncore.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,27 @@ jobs:
build-deploy:
runs-on: ubuntu-latest
name: Build & Deploy TorizonCore
container: ubuntu:23.10
steps:
- uses: actions/checkout@v3
- uses: actions/cache@v3
id: cache
with:
path: storage/
key: tcb
- name: Set abs-path file
run: |
echo "${{ github.workspace }}" >> abs-path
cat abs-path
- name: Install build tools in container
run: |
apt-get update
apt-get install -y wget curl libicu-dev
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

wget -nv https://github.com/PowerShell/PowerShell/releases/download/v7.4.2/powershell_7.4.2-1.deb_amd64.deb
dpkg -i powershell_7.4.2-1.deb_amd64.deb
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, maybe this is needed on your end because you host your own runner. But, on Github Actions the default image already has these dependencies.

Would be nice to first identify whether the dependencies are already installed before installing it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is needed because I'm running the build inside a container, on the shared and self-hosted runners. Both will spin up a docker Ubuntu:23.04 container, mount the local filesystem, and run the build commands. It's good for reproducibility. If you used GitLab, running dind is standard.


- name: Initial Setup
shell: bash
Expand Down