Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for musl based native alpine docker images. #535

Merged
merged 5 commits into from
Apr 1, 2021
Merged

Add support for musl based native alpine docker images. #535

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
fc94f8b
Add support for musl based native alpine docker images.
dinogun Mar 29, 2021
f77c651
Update alpine scc generation and fix inadvertant windows ver change.
dinogun Mar 29, 2021
7a834e1
Add debug statements, minor fixes.
dinogun Mar 30, 2021
9644a51
Make more vars local and remove debug statement.
dinogun Mar 30, 2021
0b970d0
Update README to indicate alpine musl images
dinogun Apr 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,12 @@ Note: Hotspot is not supported on Ubuntu 20.04 for s390x arch.
- 1809
- ltsc2016

# Official and Unofficial Images
AdoptOpenJDK Docker Images are available as both Official Images (Maintained by Docker) and Unofficial Images (Maintained by AdoptOpenJDK). Please choose based on your requirements.
# musl libc based Alpine Images

Starting from Java 16, hotspot builds are available natively built on musl libc instead of the regular glibc as part of the AdoptOpenJDK project. Currently these are available only for the x86_64 architecture. Accordingly we now have both regular and slim Docker Images for alpine musl based hotspot on x86_64.

# Official and Non-official Images
AdoptOpenJDK Docker Images are available as both Official Images (Maintained by Docker) and Non-official Images (Maintained by AdoptOpenJDK). Please choose based on your requirements.
* [Official Images](https://hub.docker.com/_/adoptopenjdk) are maintained by Docker and updated on every release from AdoptOpenJDK as well as when the underlying OSes are updated. Supported OSes and their versions and type of images are as below.
- Linux
- Ubuntu (20.04): Release
Expand Down
114 changes: 58 additions & 56 deletions build_latest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,24 +53,26 @@ set_runtype "$4"
# Get the image build time stored in the "build_time" array for the current arch
# Build time is stored as the time since 1-1-1970
function get_image_build_time() {
local osfamily=$2

if ! declare -p "$1" &>/dev/null; then
return;
fi

# shellcheck disable=SC2154,SC1083
btime=$(btarray=$1[${current_arch}]; eval btarch=\${"$btarray"}; echo "${btarch}");
btime=$(btarray=$1[${osfamily}_${current_arch}]; eval btarch=\${"$btarray"}; echo "${btarch}");

echo "${btime}"
}

# Check if the adopt image is available, if not need to build it.
function check_adopt_image_available() {
local tag=$1
local image_name=$1

echo "INFO: Checking when the adopt docker image ${tag} was built ..."
if ! docker pull "${tag}" &>/dev/null; then
echo "INFO: Checking when the adopt docker image ${image_name} was built ..."
if ! docker pull "${image_name}" &>/dev/null; then
# Adopt image not available currently, build needed
echo "INFO: AdoptOpenJDK docker image for ${tag} does not exist. Docker build needed"
echo "INFO: AdoptOpenJDK docker image for ${image_name} does not exist. Docker build needed"
build_needed=1
return;
fi
Expand All @@ -79,7 +81,7 @@ function check_adopt_image_available() {
# Check if we have a newer base OS docker image
# If a new base OS docker image was built in the last 24 hrs, then we need to rebuild the adopt docker image
function check_new_os_image_available() {
local tag=$2
local image_name=$2

# Get the date when the base image was created. Eg if the base OS is ubuntu, this
# translates as the exact date/time when the Ubuntu image was created on DockerHub
Expand All @@ -104,7 +106,7 @@ EOF
)"

# Get the shasums of the current Adopt Image layers
docker inspect "${tag}" > adopt_image.info
docker inspect "${image_name}" > adopt_image.info
adopt_sha_arr="$(python3 <<EOF
import sys, json;
input_file = open ('adopt_image.info');
Expand All @@ -118,7 +120,7 @@ EOF
for ssum in ${base_os_sha_arr}
do
if ! echo "${adopt_sha_arr}" | grep -q "${ssum}" ; then
echo "Base OS layer ${ssum} not found in Adopt Image: ${tag}"
echo "Base OS layer ${ssum} not found in Adopt Image: ${image_name}"
# Layer missing in the current Adopt Image, rebuild needed
build_needed=1
break;
Expand All @@ -130,10 +132,11 @@ EOF

# Check if we have a newer adopt build tarball
function check_new_adopt_build_available() {
local tag=$1
local osfamily=$1
local image_name=$2

# Get the last build date for the current arch from the "build_time" array
adopt_last_build_date=$(get_image_build_time "${build_time}")
adopt_last_build_date=$(get_image_build_time "${build_time}" "${osfamily}")
if [ -z "${adopt_last_build_date}" ]; then
echo "INFO: Unknown last tarball build time. Docker build needed"
build_needed=1
Expand All @@ -143,12 +146,12 @@ function check_new_adopt_build_available() {
adopt_last_build_date=$(( adopt_last_build_date + 86400 ))

# check when the adopt image was last built
adopt_image_creation="$(docker inspect "${tag}" | python3 -c "import sys, json; print(json.load(sys.stdin)[0]['Created'])")"
adopt_image_creation="$(docker inspect "${image_name}" | python3 -c "import sys, json; print(json.load(sys.stdin)[0]['Created'])")"
# Convert this to seconds since 1-1-1970
adopt_image_creation_date="$(date --date="${adopt_image_creation}" +%s)"

echo "INFO: Current docker image for ${tag} build date: $(date --date="@${adopt_image_creation_date}")"
echo "INFO: Current adopt build tarball related to ${tag} build date: $(date --date="@${adopt_last_build_date}")"
echo "INFO: Current docker image for ${image_name} build date: $(date --date="@${adopt_image_creation_date}")"
echo "INFO: Current adopt build tarball related to ${image_name} build date: $(date --date="@${adopt_last_build_date}")"
if [[ ${adopt_image_creation_date} -le ${adopt_last_build_date} ]]; then
# build needed
echo "INFO: Newer adopt build found. Docker build needed"
Expand All @@ -164,12 +167,12 @@ function check_new_adopt_build_available() {
# 3. If a new Adopt build is found
# 4. On any other error condition
function check_build_needed() {
local tag=$2
local osfamily=$2
local image_name=$3

echo "Checking build for image_name: ${image_name}"
build_needed=0

adopt_image_tag="${tag// -t /}"

# `runtype` flag specifies if the script is being run for `build` or `test` (PR checks)
# Checking for runtype if its `test` we proceed to build release images, as part of PR checks
if [ "${runtype}" == "test" ]; then
Expand All @@ -182,60 +185,56 @@ function check_build_needed() {
# For nightly images, check if a newer adopt nightly build is available.
if [ "${build}" == "nightly" ]; then
# Check if we have a newer adopt build tarball
check_new_adopt_build_available "${adopt_image_tag}"
check_new_adopt_build_available "${osfamily}" "${image_name}"
if [[ ${build_needed} -eq 1 ]]; then
return;
fi
fi

# Check if the adopt image is available, if not need to build it.
check_adopt_image_available "${adopt_image_tag}"
check_adopt_image_available "${image_name}"
if [[ ${build_needed} -eq 1 ]]; then
return;
fi

# Check if we have a newer base OS Image
check_new_os_image_available "$1" "${adopt_image_tag}"
check_new_os_image_available "$1" "${image_name}"
if [[ ${build_needed} -eq 1 ]]; then
return;
fi

# Check if we have a newer adopt build tarball
check_new_adopt_build_available "${adopt_image_tag}"
check_new_adopt_build_available "${osfamily}" "${image_name}"
if [[ ${build_needed} -eq 1 ]]; then
return;
fi

# build not needed
echo "INFO: Docker image for ${adopt_image_tag} exists and is latest. Docker build NOT needed"
echo "INFO: Docker image for ${image_name} exists and is latest. Docker build NOT needed"
}

# Build the Docker image with the given repo, build, build type and tags.
# Build the Docker image with the given repo, build, build type and tag.
function build_image() {
repo=$1; shift;
build=$1; shift;
btype=$1; shift;

local local_tags=("$@") # copy arguments to local array
for i in "${!local_tags[@]}"
do
echo "Tag - ${i} : ${local_tags[$i]}" # Adding an echo to check if jenkins build job is passing multiple tags
tags="${tags} -t ${repo}:${local_tags[$i]}"
done

auto_space_line=" "
image_name="${repo}:${tag}"
printf -v expanded_tags "%s ${repo}:%s " "-t" "${local_tags[@]}" # concatenate to single string : -t repo:tag -t repo:tag2
expanded_tags=${expanded_tags%?} # remove trailing space
dockerfile="Dockerfile.${vm}.${build}.${btype}"
local repo=$1;
local build=$2;
local btype=$3;
local osfamily=$4;
local tag=$5;

local dockerfile="Dockerfile.${vm}.${build}.${btype}"
local image_name="${repo}:${tag}"
check_build_needed "${dockerfile}" "${osfamily}" "${image_name}"
# Check if we need to build this image.
check_build_needed "${dockerfile}" "${tags}"
if [[ ${build_needed} -eq 0 ]]; then
# No build needed, we are done
return;
fi

echo "docker push ${repo}:${tag}" >> "${push_cmdfile}"
auto_space_line=" "
printf -v expanded_tags "%s ${repo}:%s " "-t" "${tag}" # concatenate to single string : -t repo:tag -t repo:tag2
expanded_tags=${expanded_tags%?} # remove trailing space

echo "docker push ${image_name}" >> "${push_cmdfile}"
echo "#####################################################"
echo "INFO: docker build --no-cache ${expanded_tags} -f ${dockerfile} ."
echo "#####################################################"
Expand Down Expand Up @@ -264,11 +263,8 @@ function build_image() {
echo "#####################################################"
echo " Scanning with snyk for vulnerabilities "
echo "#####################################################"
for i in "${!tags[@]}"
do
echo "...scanning ${repo}:${tags[$i]}"
snyk test --docker "${repo}:${tags[$i]}" --file="${dockerfile}"
done
echo "...scanning ${image_name}"
snyk test --docker "${image_name}" --file="${dockerfile}"
fi
echo "| ${image_name:0:80}${auto_space_line:0:$((76 - ${#image_name}))} | success |" >> ${summary_table_file}
echo "+------------------------------------------------------------------------------+----------+" >> ${summary_table_file}
Expand All @@ -295,11 +291,8 @@ function build_image() {
echo "#####################################################"
echo " Scanning with snyk for vulnerabilities "
echo "#####################################################"
for i in "${!tags[@]}"
do
echo "...scanning ${repo}:${tags[$i]}"
snyk test --docker "${repo}:${tags[$i]}" --file="${dockerfile}"
done
echo "...scanning ${image_name}"
snyk test --docker "${image_name}" --file="${dockerfile}"
fi
echo "| ${image_name:0:80}${auto_space_line:0:$((76 - ${#image_name}))} | success |" >> ${summary_table_file}
echo "+------------------------------------------------------------------------------+----------+" >> ${summary_table_file}
Expand All @@ -309,15 +302,23 @@ function build_image() {

# Build the docker image for a given VM, OS, BUILD and BUILD_TYPE combination.
function build_dockerfile {
vm=$1; pkg=$2; os=$3; build=$4; btype=$5;

local vm=$1;
local pkg=$2;
local build=$3;
local btype=$4;
local osfamily=$5;
local os=$6;

local tag=""
echo "INFO: current_arch: ${current_arch}, osfamily: ${osfamily}, os: ${os}"
if [ -z "${current_arch}" ]; then
jverinfo="${shasums}[version]"
else
jverinfo="${shasums}[version-${current_arch}]"
jverinfo="${shasums}[version-${osfamily}_${current_arch}]"
fi
# shellcheck disable=SC1083,SC2086
eval jrel=\${$jverinfo}
echo "INFO: release: ${jrel}"
# Docker image tags cannot have "+" in them, replace it with "_" instead.
# shellcheck disable=SC2154
rel=${jrel//+/_}
Expand Down Expand Up @@ -354,7 +355,7 @@ function build_dockerfile {
fi
echo "INFO: Building ${trepo} ${tag} from $file ..."
pushd "${dir}" >/dev/null || return
build_image "${trepo}" "${build}" "${btype}" "${tag}"
build_image "${trepo}" "${build}" "${btype}" "${osfamily}" "${tag}"
popd >/dev/null || return
}

Expand Down Expand Up @@ -386,6 +387,7 @@ do
# Type = Full or Slim
btypes=$(parse_vm_entry "${vm}" "${version}" "${package}" "${os}" "Type:")
dir=$(parse_vm_entry "${vm}" "${version}" "${package}" "${os}" "Directory:")
osfamily=$(parse_vm_entry "${vm}" "${version}" "${package}" "${os}" "OS_Family:")

for build in ${builds}
do
Expand All @@ -411,12 +413,12 @@ do
for btype in ${btypes}
do
file="${dir}/Dockerfile.${vm}.${build}.${btype}"
generate_dockerfile "${file}" "${package}" "${build}" "${btype}" "${os}"
generate_dockerfile "${file}" "${package}" "${build}" "${btype}" "${osfamily}" "${os}"
if [ ! -f "${file}" ]; then
continue;
fi
# Build the docker images for valid Dockerfiles
build_dockerfile "${vm}" "${package}" "${os}" "${build}" "${btype}"
build_dockerfile "${vm}" "${package}" "${build}" "${btype}" "${osfamily}" "${os}"
done
done
done
Expand Down
Loading