Enzyme_jll for 1.6

[vchuravy]

To capture the discussion I had with @staticfloat on the topic.

 julia> using Base.BinaryPlatforms
       platforms = Dict(
           Platform("x86_64", "linux"; julia_version=v"1.5") => "info for 1.5",
           Platform("x86_64", "linux"; julia_version=v"1.6") => "info for 1.6",
           Platform("x86_64", "linux"; julia_version=v"1.7") => "info for 1.7",
       )
       select_platform(platforms, Platform("x86_64", "linux"; julia_version=v"1.6"))

so what I’m showcasing here is that (1) you can have arbitrary tags assigned to Platform objects
(2) you can pattern-match with Base.BinaryPlatforms.select_platform to select the Platform key in a dictionary that best matches the platform you pass in to it
so if we have a JLL that has a bunch of mappings in its Artifacts.toml, each of which has a julia_version key, it will automatically choose the right one
and we don’t have to write any custom code on the JLL side because teh default HostPlatform already encodes Julia version:

julia> HostPlatform()
Linux x86_64 {cxxstring_abi=cxx11, julia_version=1.6.0, libc=glibc, libgfortran_version=4.0.0, libstdcxx_version=3.4.26}

We do this because I consider the julia version a property of the environment

from the BB perspective, I think what needs to happen is we need to do away with the julia_compat kwarg to build_tarballs() and instead present this information via the julia_version tag on the platform objects

and then we need to add an expand_julia_versions(platforms, julia_versions) function so that we can do e.g. expand_julia_versions(platforms, [v"1.6", v"1.7"]) also, (and here’s the real kicker) this will only work for v1.6+ because v1.5 doesn’t know how to deal with extended platforms

Okay so this is a kind of hacky strategy that Stefan and I talked about a long time ago
https://github.com/JuliaLang/Pkg.jl/blob/v1.5.3/src/Artifacts.jl#L554-L555
this is how v1.5 selects the Artifacts.toml block that gets downloaded
In v1.5, if you add extended attributes, they will be ignored
so unpack_platform() will give the same result, ignoring the julia_version key
so you’ll get two keys that are the same

julia> Dict("a" => 1, "b" => 2, "a" => 3)
Dict{String, Int64} with 2 entries:
  "b" => 2
  "a" => 3

So you need to make sure that the Julia v1.5 entry is the last one in the file
so basically, write out all the entries into the TOML file, but then ensure that the Julia v1.5 mappings are after the Julia v1.6 mappings I don’t really like it, but it should function

cc @giordano

[vchuravy]

1. Ensure that the sorted Artifacts trick works
2. Add special casing of such to BB
3. Change julia_compat to julia_version
4. Can we make Dependencies be choosable upon the julia_version? Enzyme_jll needs different LLVM versions depending on the platform tag.
5. ???
6. Profit.

[staticfloat]

4. Can we make Dependencies be choosable upon the julia_version? Enzyme_jll needs different LLVM versions depending on the platform tag.

No, but you can emit multiple build_tarballs() invocations. So you can construct your dependencies_jl16 = ... and dependencies_jl17 = ... and then pass them into two separate build_tarballs() that each have the platform objects set with the appropriate julia_version tags set. As long as all of your builds are output with the same name passed to build_tarballs(), they'll be published in the same JLL.

[giordano]

Remember that when you have multiple build_tarballs calls you need to do stuff like https://github.com/JuliaPackaging/Yggdrasil/blob/8f26b64df939bc17a51c76e1944636bd0656be74/M/MKL/build_tarballs.jl#L50-L65

[vchuravy]

Remember that when you have multiple build_tarballs calls you need to do stuff like https://github.com/JuliaPackaging/Yggdrasil/blob/8f26b64df939bc17a51c76e1944636bd0656be74/M/MKL/build_tarballs.jl#L50-L65

😢

[giordano]

Maybe the alternative is to implement JuliaPackaging/BinaryBuilder.jl#778 (comment)