How to use GR_jll in Google Colab? #6089

Snimm · 2023-01-16T15:37:38Z

https://colab.research.google.com/drive/1RLc2DAnQ0pRguGasg2-nSmdRbsbLNYeb?usp=sharing#scrollTo=L5gRrpzpjs7A
Colab file

I am trying to use AlphaZero.jl in Colab but getting the this error:

[ Info: Precompiling AlphaZero [8ed9eb0b-7496-408d-8c8b-2119aeea02cd]
[ Info: Using the Flux implementation of AlphaZero.NetLib.
ERROR: LoadError: InitError: UndefVarError: GR_jll not defined
Stacktrace:
  [1] __init__()
    @ GR.GRPreferences ~/.julia/packages/GR/I1aTM/src/preferences.jl:50
  [2] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
    @ Base ./loading.jl:831
  [3] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt64)
    @ Base ./loading.jl:1039
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1315
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [6] macro expansion
    @ ./loading.jl:1180 [inlined]
  [7] macro expansion
    @ ./lock.jl:223 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
  [9] top-level scope
    @ ~/.julia/packages/Plots/nuwp4/src/backends.jl:395
 [10] eval
    @ ./boot.jl:368 [inlined]
 [11] _initialize_backend(pkg::Plots.GRBackend)
    @ Plots ~/.julia/packages/Plots/nuwp4/src/backends.jl:394
 [12] backend(pkg::Plots.GRBackend)
    @ Plots ~/.julia/packages/Plots/nuwp4/src/backends.jl:260
 [13] backend(sym::Symbol)
    @ Plots ~/.julia/packages/Plots/nuwp4/src/backends.jl:270
 [14] load_default_backend()
    @ Plots ~/.julia/packages/Plots/nuwp4/src/backends.jl:201
 [15] backend()
    @ Plots ~/.julia/packages/Plots/nuwp4/src/backends.jl:248
 [16] top-level scope
    @ ~/.julia/packages/Plots/nuwp4/src/init.jl:123
 [17] include(mod::Module, _path::String)
    @ Base ./Base.jl:419
 [18] include(x::String)
    @ Plots ~/.julia/packages/Plots/nuwp4/src/Plots.jl:1
 [19] top-level scope
    @ ~/.julia/packages/Plots/nuwp4/src/Plots.jl:176
 [20] include
    @ ./Base.jl:419 [inlined]
 [21] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1554
 [22] top-level scope
    @ stdin:1
during initialization of module GRPreferences
in expression starting at /root/.julia/packages/Plots/nuwp4/src/init.jl:123
in expression starting at /root/.julia/packages/Plots/nuwp4/src/Plots.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile Plots [91a5bcdd-55d7-5caf-9e0b-520d859cae80] to /root/.julia/compiled/v1.8/Plots/jl_s51kIN.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:1707
  [3] compilecache
    @ ./loading.jl:1651 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1337
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [6] macro expansion
    @ ./loading.jl:1180 [inlined]
  [7] macro expansion
    @ ./lock.jl:223 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
  [9] include(mod::Module, _path::String)
    @ Base ./Base.jl:419
 [10] include(x::String)
    @ AlphaZero ~/.julia/packages/AlphaZero/3oiyX/src/AlphaZero.jl:6
 [11] top-level scope
    @ ~/.julia/packages/AlphaZero/3oiyX/src/AlphaZero.jl:160
 [12] include
    @ ./Base.jl:419 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1554
 [14] top-level scope
    @ stdin:1
in expression starting at /root/.julia/packages/AlphaZero/3oiyX/src/ui/ui.jl:1
in expression starting at /root/.julia/packages/AlphaZero/3oiyX/src/AlphaZero.jl:6
in expression starting at stdin:1
Failed to precompile AlphaZero [8ed9eb0b-7496-408d-8c8b-2119aeea02cd] to /root/.julia/compiled/v1.8/AlphaZero/jl_L5f97Q.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1707
 [3] compilecache
   @ ./loading.jl:1651 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144

Using import GR_jll:

[ Info: Precompiling GR_jll [d2c73de3-f751-5644-a686-071e5b155ba9]
ERROR: LoadError: InitError: could not load library "/root/.julia/artifacts/37dda4e57d9de95c99d1f8c6b3d8f4eca88c39a2/lib/libGL.so"
/root/.julia/artifacts/37dda4e57d9de95c99d1f8c6b3d8f4eca88c39a2/lib/libGL.so: undefined symbol: _glapi_tls_Current
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ Libglvnd_jll ~/.julia/packages/Libglvnd_jll/6lXn3/src/wrappers/x86_64-linux-gnu.jl:22
  [5] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
    @ Base ./loading.jl:831
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt64)
    @ Base ./loading.jl:1039
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1315
  [8] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [9] macro expansion
    @ ./loading.jl:1180 [inlined]
 [10] macro expansion
    @ ./lock.jl:223 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
 [12] include(mod::Module, _path::String)
    @ Base ./Base.jl:419
 [13] top-level scope
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/toplevel_generators.jl:188
 [14] include
    @ ./Base.jl:419 [inlined]
 [15] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1554
 [16] top-level scope
    @ stdin:1
during initialization of module Libglvnd_jll
in expression starting at /root/.julia/packages/GLFW_jll/Rctxo/src/wrappers/x86_64-linux-gnu.jl:4
in expression starting at /root/.julia/packages/GLFW_jll/Rctxo/src/GLFW_jll.jl:2
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile GLFW_jll [0656b61e-2033-5cc2-a64a-77c0f6c09b89] to /root/.julia/compiled/v1.8/GLFW_jll/jl_eZZOku.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
    @ Base ./loading.jl:1707
  [3] compilecache
    @ ./loading.jl:1651 [inlined]
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1337
  [5] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [6] macro expansion
    @ ./loading.jl:1180 [inlined]
  [7] macro expansion
    @ ./lock.jl:223 [inlined]
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
  [9] include(mod::Module, _path::String)
    @ Base ./Base.jl:419
 [10] top-level scope
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/toplevel_generators.jl:188
 [11] include
    @ ./Base.jl:419 [inlined]
 [12] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1554
 [13] top-level scope
    @ stdin:1
in expression starting at /root/.julia/packages/GR_jll/cIPwk/src/wrappers/x86_64-linux-gnu-cxx11.jl:8
in expression starting at /root/.julia/packages/GR_jll/cIPwk/src/GR_jll.jl:2
in expression starting at stdin:1
Failed to precompile GR_jll [d2c73de3-f751-5644-a686-071e5b155ba9] to /root/.julia/compiled/v1.8/GR_jll/jl_pXwRVg.

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1707
 [3] compilecache
   @ ./loading.jl:1651 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144

How do I fix this?

The text was updated successfully, but these errors were encountered:

giordano · 2023-01-16T15:44:06Z

Is the environment variable LD_LIBRARY_PATH set? You can see that inside julia with

ENV["LD_LIBRARY_PATH"]

if it's set to a non-empty value you should see the string printed to string, if it isn't set at all you should get an error about the key LD_LIBRARY_PATH not existing in the ENV dictionary. If it's indeed set, does starting Julia with

LD_LIBRARY_PATH="" julia

help?

Snimm · 2023-01-16T17:11:05Z

@giordano ENV["LD_LIBRARY_PATH"] gives "/usr/lib64-nvidia" . I don't know what you mean by starting Julia with LD_LIBRARY_PATH="" julia mean but executing it in a cell gives

syntax: extra token "julia" after end of expression

Stacktrace:
 [1] top-level scope
   @ In[14]:1

giordano · 2023-01-16T17:12:26Z

When you start julia, start it with

LD_LIBRARY_PATH="" julia

instead of

julia

This has to be done outside of the julia process, not in the Julia REPL

Snimm · 2023-01-16T17:14:28Z

How do I do that in google colab? https://colab.research.google.com/drive/1RLc2DAnQ0pRguGasg2-nSmdRbsbLNYeb?usp=sharing#scrollTo=GIeFXS0F0zww

%%shell
set -e

#---------------------------------------------------#
JULIA_VERSION="1.8.2" # any version ≥ 0.7.0
JULIA_PACKAGES="IJulia BenchmarkTools"
JULIA_PACKAGES_IF_GPU="CUDA" # or CuArrays for older Julia versions
JULIA_NUM_THREADS=2
#---------------------------------------------------#

if [ -z `which julia` ]; then
  # Install Julia
  JULIA_VER=`cut -d '.' -f -2 <<< "$JULIA_VERSION"`
  echo "Installing Julia $JULIA_VERSION on the current Colab Runtime..."
  BASE_URL="https://julialang-s3.julialang.org/bin/linux/x64"
  URL="$BASE_URL/$JULIA_VER/julia-$JULIA_VERSION-linux-x86_64.tar.gz"
  wget -nv $URL -O /tmp/julia.tar.gz # -nv means "not verbose"
  tar -x -f /tmp/julia.tar.gz -C /usr/local --strip-components 1
  rm /tmp/julia.tar.gz

  # Install Packages
  nvidia-smi -L &> /dev/null && export GPU=1 || export GPU=0
  if [ $GPU -eq 1 ]; then
    JULIA_PACKAGES="$JULIA_PACKAGES $JULIA_PACKAGES_IF_GPU"
  fi
  for PKG in `echo $JULIA_PACKAGES`; do
    echo "Installing Julia package $PKG..."
    julia -e 'using Pkg; pkg"add '$PKG'; precompile;"' &> /dev/null
  done

  # Install kernel and rename it to "julia"
  echo "Installing IJulia kernel..."
  julia -e 'using IJulia; IJulia.installkernel("julia", env=Dict(
      "JULIA_NUM_THREADS"=>"'"$JULIA_NUM_THREADS"'"))'
  KERNEL_DIR=`julia -e "using IJulia; print(IJulia.kerneldir())"`
  KERNEL_NAME=`ls -d "$KERNEL_DIR"/julia*`
  mv -f $KERNEL_NAME "$KERNEL_DIR"/julia  

  echo ''
  echo "Successfully installed `julia -v`!"
  echo "Please reload this page (press Ctrl+R, ⌘+R, or the F5 key) then"
  echo "jump to the 'Checking the Installation' section."
fi

Where do I put it?

giordano · 2023-01-16T17:16:44Z

I have no clue, I never used Colab.

giordano · 2023-01-31T10:00:36Z

Sounds like Julia in Colab, which was never supported by Google, is broken everywhere: JuliaLang/julia#48461. Not really much we can do about.

giordano · 2023-02-02T08:13:41Z

See https://discourse.julialang.org/t/google-colab-compatibility-broken/93004/19. Closing in favour of googlecolab/colabtools#3385.

Snimm mentioned this issue Jan 18, 2023

Is there a way to use AlphaZero.jl without using Plots.jl and Gr_jll? jonathan-laurent/AlphaZero.jl#160

Open

MilesCranmer mentioned this issue Feb 2, 2023

Colab update breaks compatibility with JuliaLang googlecolab/colabtools#3385

Closed

giordano closed this as not planned Won't fix, can't repro, duplicate, stale Feb 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to use GR_jll in Google Colab? #6089

How to use GR_jll in Google Colab? #6089

Snimm commented Jan 16, 2023 •

edited

Loading

giordano commented Jan 16, 2023

Snimm commented Jan 16, 2023

giordano commented Jan 16, 2023

Snimm commented Jan 16, 2023

giordano commented Jan 16, 2023

giordano commented Jan 31, 2023

giordano commented Feb 2, 2023

How to use GR_jll in Google Colab? #6089

How to use GR_jll in Google Colab? #6089

Comments

Snimm commented Jan 16, 2023 • edited Loading

giordano commented Jan 16, 2023

Snimm commented Jan 16, 2023

giordano commented Jan 16, 2023

Snimm commented Jan 16, 2023

giordano commented Jan 16, 2023

giordano commented Jan 31, 2023

giordano commented Feb 2, 2023

Snimm commented Jan 16, 2023 •

edited

Loading