Skip to content

Commit

Permalink
Specify multiple architectures for Julia to precompile to (#2044)
Browse files Browse the repository at this point in the history
* Specify multiple architectures for Julia to precompile to

For amd64 (x86_64), we should specify what specific targets the
precompilation should be done for.  If we don't specify it,
it's *only* done for the target of the host doing the compilation.
When the container runs on a host that's still x86_64, but
a *different* generation of CPU than what the build host was, the
precompilation is useless and Julia takes a long long time to start
up. This specific multitarget comes from
https://docs.julialang.org/en/v1/devdocs/sysimg/#Specifying-multiple-system-image-targets,
and is the same set of options that the official Julia x86_64 build is
compiled with.  If the architecture the container runs on is
different, precompilation may still have to be re-done on first
startup - but this *should* catch most of the issues.

h/t to
https://discourse.julialang.org/t/is-it-possible-to-make-precompilation-portable-for-docker-images-built-with-a-different-cpu/95913
which helped point me towards `JULIA_CPU_TARGET`.

Fixes #2015 for more information

* Fix bash syntax issue

* Add JULIA_CPU_TARGET for aarch64 as well

- Don't need `export` as this is only used within this
  script
- Steal from upstream what should be setup for aarch64

* Re-add export for JULIA_CPU_TARGET

Quietens pre-commit
  • Loading branch information
yuvipanda authored Nov 29, 2023
1 parent d91bb62 commit 99d3064
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions images/minimal-notebook/setup-scripts/setup-julia-packages.bash
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,28 @@ set -exuo pipefail
# - The JULIA_PKGDIR environment variable is set
# - Julia is already set up, with the setup-julia.bash command


# If we don't specify what CPUs the precompilation should be done for, it's
# *only* done for the target of the host doing the compilation. When the
# container runs on a host that's the same architecture, but a *different*
# generation of CPU than what the build host was, the precompilation is useless
# and Julia takes a long long time to start up. This specific multitarget comes
# from https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L20-L76,
# and may need to be updated as new CPU generations come out.
# If the architecture the container runs on is different,
# precompilation may still have to be re-done on first startup - but this
# *should* catch most of the issues. See
# https://github.com/jupyter/docker-stacks/issues/2015 for more information
if [ "$(uname -m)" == "x86_64" ]; then
# See https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L24
# for an explanation of these options
export JULIA_CPU_TARGET="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)"
elif [ "$(uname -m)" == "aarch64" ]; then
# See https://github.com/JuliaCI/julia-buildkite/blob/70bde73f6cb17d4381b62236fc2d96b1c7acbba7/utilities/build_envs.sh#L54
# for an explanation of these options
export JULIA_CPU_TARGET="generic;cortex-a57;thunderx2t99;carmel"
fi

# Install base Julia packages
julia -e '
import Pkg;
Expand Down

0 comments on commit 99d3064

Please sign in to comment.