Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: add definitions for devcontainer base image #2025

Merged
merged 2 commits into from
Mar 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .devcontainer/base-image/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM mcr.microsoft.com/devcontainers/rust:0.203.7-1

RUN apt-get update \
&& apt-get install -y --no-install-recommends \
cmake \
&& rm -rf /var/lib/apt/lists/* \
&& sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin

# ========= Install cargo-tools for non-root user (vscode) =========
COPY Taskfile.cargo-tools.yml /tmp/Taskfile.cargo-tools.yml
WORKDIR /tmp
USER vscode
RUN task -t Taskfile.cargo-tools.yml install

USER root
7 changes: 7 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,13 @@ updates:
commit-message:
prefix: "chore: "

- package-ecosystem: docker
directory: .devcontainer/base-image
schedule:
interval: daily
commit-message:
prefix: "chore: "

# For actions (rather than workflows), we need to list each directory, ref
# https://github.com/dependabot/dependabot-core/issues/5137, from https://github.com/dependabot/dependabot-core/issues/4178#issuecomment-1118492006
- directory: ".github/actions/build-prql-js"
Expand Down
64 changes: 64 additions & 0 deletions .github/workflows/devcontainer.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: devcontainer

on:
push:
paths:
- .devcontainer/base-image/Dockerfile
- Taskfile.cargo-tools.yml
pull_request:
paths:
- .devcontainer/base-image/Dockerfile
- Taskfile.cargo-tools.yml
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: docker/metadata-action@v4
id: meta
with:
images: ghcr.io/${{ github.repository_owner }}/prql-devcontainer-base
tags: |
type=raw,latest

- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- id: set-platforms
run: |
if [ "${{ github.ref }}" = "refs/heads/main" ]; then
echo "platforms=linux/amd64,linux/arm64" >>$GITHUB_OUTPUT
echo "push=true" >>$GITHUB_OUTPUT
else
echo "platforms=linux/amd64" >>$GITHUB_OUTPUT
echo "push=false" >>$GITHUB_OUTPUT
fi
Comment on lines +42 to +50
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eitsupi On reflection it's probably simpler to run both architectures on tests, not just amd64. Again — we should be liberal with workflows...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will fix this.
There is a disadvantage that multi-platform builds can't load images, so we can't run tests on containers, etc., but since we are only doing builds at this point, it shouldn't be a problem.


- name: Build
uses: docker/build-push-action@v4
with:
file: "./.devcontainer/base-image/Dockerfile"
tags: ${{ steps.meta.outputs.tags }}
platforms: ${{ steps.set-platforms.outputs.platforms }}
push: ${{ steps.set-platforms.outputs.push }}
cache-from: |
${{ env.IMAGE_NAME }}
type=gha
cache-to: |
type=inline
type=gha,mode=max
19 changes: 19 additions & 0 deletions Taskfile.cargo-tools.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# yaml-language-server: $schema=https://json.schemastore.org/taskfile.json

version: 3

tasks:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No stress at all, but out of interest — any reason to split this out into a different file?

Copy link
Member Author

@eitsupi eitsupi Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each time the Taskfile.yml (Files copied into the container at build time) is modified (even a mere modification of a comment, for example), the later layers' cache is invalidated.
By copying files that are less likely to be modified, I believe we can make better use of the cache.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. But the Taskfile doesn't change often — I would lean towards having the repo simpler even if it means running too many workflows. If we did this for everything we'd have many many tiny files and it would be difficult to work with the repo.

If we had a full build system like Bazel, then we could do this, but it's very approximate at best, and so somewhat fruitless to optimize it...

install:
# factored this out because it takes a long time to build
desc: Install cargo tools for PRQL development.
cmds:
# `--locked` installs from the underlying lock files (which is not the
# default?!)
- cargo install --locked bacon cargo-audit cargo-insta cargo-release
default-target mdbook mdbook-admonish mdbook-toc wasm-bindgen-cli
wasm-pack
# Can't install atm with `--locked`
- cargo install mdbook-footnote
- cmd: |
[ "$(which cargo-insta)" ] || echo "🔴 Can't find a binary that cargo just installed. Is the cargo bin path (generally at ~/.cargo/bin) on the \$PATH?"
silent: true
20 changes: 7 additions & 13 deletions Taskfile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@

version: "3"

includes:
cargo-tools: ./Taskfile.cargo-tools.yml

vars:
# Keep in sync with .vscode/extensions.json
vscode_extensions: |
Expand Down Expand Up @@ -38,7 +41,7 @@ tasks:

desc: Install tools for PRQL development.
cmds:
- task: install-cargo-tools
- task: cargo-tools:install
- task: install-brew-dependencies
- task: install-maturin
- task: install-pre-commit
Expand All @@ -54,18 +57,9 @@ tasks:

install-cargo-tools:
# factored this out because it takes a long time to build
desc: Install cargo tools for PRQL development.
cmds:
# `--locked` installs from the underlying lock files (which is not the
# default?!)
- cargo install --locked bacon cargo-audit cargo-insta cargo-release
default-target mdbook mdbook-admonish mdbook-toc wasm-bindgen-cli
wasm-pack
# Can't install atm with `--locked`
- cargo install mdbook-footnote
- cmd: |
[ "$(which cargo-insta)" ] || echo "🔴 Can't find a binary that cargo just installed. Is the cargo bin path (generally at ~/.cargo/bin) on the \$PATH?"
silent: true
desc: Alias of `cargo-tools:install`
deps:
- cargo-tools:install

install-maturin:
desc: Install maturin.
Expand Down