[Docs] Raw Container Task Julia Docker Image Doesn't support Arm64

[Future-Outlier]

Description

When downloading ghcr.io/flyteorg/rawcontainers-julia:v2, and start the image as container, we will find that julia command is not available.

Reference: https://docs.flyte.org/en/latest/user_guide/customizing_dependencies/raw_containers.html#raw-container

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[dosubot]

Hey there, @Future-Outlier! 🚀 Great to see you diving into another adventure with us. How's the journey in the world of tech treating you today?

Thank you for bringing this to our attention. It seems like there might be an issue with the ghcr.io/flyteorg/rawcontainers-julia:v2 image. The julia command should be available if the image was built correctly from a Dockerfile that includes Julia.

The Dockerfile provided in the Flyte documentation shows how to create a custom Julia environment by starting from the official Julia image (julia:1.6.4-buster), setting a working directory, and copying a Julia script into the container. If the julia command is not available in the ghcr.io/flyteorg/rawcontainers-julia:v2 image, it suggests that this specific image might not be built using the provided Dockerfile or there could be an issue with how the image was built or configured.

Unfortunately, I wasn't able to find the exact Dockerfile used to build the ghcr.io/flyteorg/rawcontainers-julia:v2 image, any known issues with this image, or the exact process used to build this image within the repository. It's possible that this information is available elsewhere or I may have missed it.

To ensure the julia command is available, you would need to build a Docker image using the provided Dockerfile or ensure that the image you are using is correctly built from a Dockerfile that includes Julia. If you're using an image from a repository like ghcr.io/flyteorg/rawcontainers-julia:v2, you would need to check the documentation or the Dockerfile used to build that specific image version to understand why the julia command might not be available.

I hope this helps! If you have any additional information or context that could help us understand this issue better, please let us know.

Sources

• docs/user_guide/customizing_dependencies/raw-containers-supporting-files/per-language/julia/Dockerfile

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[eapolinario]

I can't repro this, @Future-Outlier. This is what I see:

❯ docker run -it --rm --platform linux/amd64 ghcr.io/flyteorg/rawcontainers-julia:v2  /bin/sh
# julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.4 (2021-11-19)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia>
#

~ 
❯ docker images | grep julia
ghcr.io/flyteorg/rawcontainers-julia          v2                                             810ea9ce17e6   16 months ago   484MB

This dockerfile was used to build that image. Can you try building it locally?

[Future-Outlier]

I can't repro this, @Future-Outlier. This is what I see:

❯ docker run -it --rm --platform linux/amd64 ghcr.io/flyteorg/rawcontainers-julia:v2  /bin/sh
# julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.6.4 (2021-11-19)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia>
#

~ 
❯ docker images | grep julia
ghcr.io/flyteorg/rawcontainers-julia          v2                                             810ea9ce17e6   16 months ago   484MB

This dockerfile was used to build that image. Can you try building it locally?

I think the main problem will be we need to support multi arch for the Docker Image, when I build by myself, it works, but when I use the official image, it doesn't.

[Future-Outlier]

cc @samhita-alla , Do you know how to update the image?
Here is the command I use

docker buildx build -t localhost:30000/julia:0321 -f DockerfileJulia . --load

[Future-Outlier]

@eapolinario Can you also take a look at my PR about local container task execution?
Thank you.
Also if you can provide input and output with List and Dict in Raw Container task remote execution, it will help me a lot.
I've traced the code in flytecopilot, but it still couldn't help me use List or Dict as input/output types.

Here's the PR link:
flyteorg/flytekit#2258

[eapolinario]

Let's separate these issues out: (1) support for multi-arch images for the julia example, and (2) local raw containers and other raw containers limitations.

For (1), since this image to run the julia examples doesn't change frequently, I pushed a multi-arch version of that image (in fact I pushed multiarch versions of all raw container images): https://github.com/orgs/flyteorg/packages/container/rawcontainers-julia/194176739?tag=v2

I'm going to review the PR related to (2) shortly.

[eapolinario]

We now have multi-arch images for the julia example.