hlb 0.2

Welcome to HLB 0.2! We landed many important features to the language, so we're starting to stabilize and focus heavily on testing and polishing the whole experience.

Highlights

Heredocs

Thanks to @coryb, we landed heredocs in this release. Let's look into how we can use it.

A common pattern to start from a base image and install system packages required to build or run an application. In hlb v0.1, we had to write everything in one line.

fs default() {
	image "ubuntu"
	run "apt-get update && apt-get --no-install-recommends install -y git make gcc g++"
}

Using heredocs, we can make it easier to read and maintain. A heredoc begins with a prefix << followed by an identifier. This identifier is used to indicate the start and end of the heredoc and you can pick this identifier to ensure that it doesn't appear in the heredoc body. We recommend picking an identifier related to the contents of the heredoc body. In this case, we will use APT as our heredoc identifier.

There are three types of heredoc prefixes which are useful in different scenarios: <<, <<-, and <<~

The << prefix means everything after the identifier is literal, and that includes any white spaces, newlines, and tabs. Since the shell eliminates white spaces too, a better example is if you are writing a file where whitespace matters.

fs default() {
	scratch
	mkfile "/deployment.yaml" 0o644 <<YAML
apiVersion: apps/v1
kind: Deployment
metadata:
	name: nginx-deployment
spec:
	selector:
		matchLabels:
			app: nginx
	YAML
}

The <<- prefix means all leading tabs are stripped. So we can go back to the apt-get example and improve upon it. Using <<- allows us to write a reasonably tabulated and multi-line version.

fs default() {
	image "ubuntu"
	run <<-APT
		apt-get update && \
		apt-get install --no-install-recommends install -y \
			git \
			make \
			gcc \
			g++
	APT
}

The <<~ prefix means that all sequences of white spaces, newlines, and tabs are replaced with a single white space. This means that we no longer have to suffix \ at the end of every line! Note that we still need && to run multiple commands on the shell.

fs default() {
	image "ubuntu"
	run <<~APT
		apt-get update &&
		apt-get install --no-install-recommends install -y
			git
			make
			gcc
			g++
	APT
}

Multi-targets

We landed support for multi-targets! This means that you can specify multiple targets from the command line and it'll build all the targets in parallel!

hlb run -t unitTest -t integrationTest -t lint build.hlb

Outputs

In HLB 0.1, the only way to retrieve data from a build is to specify it from the command line. For example, if we wanted to download the result of a build as a tarball we can run:

hlb run -t myBuild --downloadTarball=myBuild.tar build.hlb

In HLB 0.2, now that we have landed multi-targets, we need to associate an output like downloadTarball to a specific target. After specifying the target name, you can optionally specify multiple outputs in a CSV fashion:

hlb run -t myBuild,downloadTarball=myBuild.tar \
	-t myOtherBuild,download=./other-build,dockerPush=openllb/other-build:latest \
	build.hlb

However, specifying outputs from the command line is usually kept for one-off tasks. In 0.2, you can now execute outputs in the middle of a build using the same outputs.

fs default() {
	image "alpine"
	run <<~APK
		apk add -U
			git
			curl
			alpine-sdk
	APK
	dockerPush "openllb/alpine:build-essentials"
}

Note that command dockerPush does not modify the fs, so you can continue chaining commands in the same function:

fs default() {
	image "alpine"
	run <<~APK
		apk add -U
			git
			curl
			alpine-sdk
	APK
	dockerPush "openllb/myimage:base"
	run "make && make install" with option {
		dir "/in"
		mount src "/in"
	}
	dockerPush "openllb/myimage:built"
}

Groups

When running multi-targets from the command line, the only choice is to run them in parallel. We landed the group type in 0.2 that allows us to define multi-targets in HLB itself, as well as the ability to control which targets to run sequentially, and which to run in parallel.

We can define a group function just like any other, and it can invoke other group functions too. When invoked one after another, the first group will finish executing before the second will start. This means that if any targets in test fails, then publish will never run.

group default() {
	test
	publish
}

Groups also have a builtin group parallel(variadic group groups) that allow you to define parallel multi-targets.

group test() {
	parallel unitTest integrationTest
}

Functions with fs type can be coerced to group (but not the other way around), so unitTest and integrationTest can be regular fs functions or other group functions.

For example, we can define a group that will publish all the microservices in parallel.

group publish() {
	parallel fs {
		buildApp
		dockerPush "openllb/app"
	} fs {
		buildDB
		dockerPush "openllb/db"
	} fs {
		buildUI
		dockerPush "openllb/ui"
	}
}

Import/export

One of the main themes of HLB is to write reusable containerized builds. We landed the import and export features to allow users to write and consume HLB as libraries.

HLB programs are file scoped, this means that functions defined in one file are not directly accessible by other files. In 0.2, you can now import another file so that you can break down your HLB files by function.

import go "./go.hlb"

fs default() {
	go.build fs { local "."; }
}

In the example above, the default target runs the function build from the module go. However, in order for this to work, go.hlb must also export the build function. This is how HLB modules declare their public API.

export build

fs build(fs input) {
	...
}

We also added the ability to consume external modules. Since we already have powerful primitives in HLB, it's only like HLB to import external modules with a target.

import go from fs {
	image "openllb/go.hlb"
}

In this example, we pushed to DockerHub an image openllb/go.hlb that contains a single module.hlb file. This file has a few exported functions, so we are able to import a HLB module from a container image. Since this is a regular fs function, you can retrieve the module from pretty much anywhere.

import gist from fs {
	http "https://gist.githubusercontent.com/hinshun/fdffbc3a1acb21558022f87b2f530817/raw/549cab5a5a467b34b30187ffc6db434afc575b4d/module.hlb"
}

fs default() {
	gist.message
}

We started a new repository openllb/modules to author some high-quality HLB modules for the community. You can think of it as the start of a docker/library equivalent but for Dockerfiles. Maybe soon enough, you'll see a registry of HLB modules that is indexed and searchable!