Caching for Haskell projects

[commandodev]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

I'm trying to get caching working for Haskell projects.

Using the same basic haskell/servant project as #312 time I run

NIXPACKS_INSTALL_CACHE_DIRS=/root/.stack NIXPACKS_BUILD_CACHE_DIRS=/app/.stack-work nixpacks build --env NIXPACKS_INSTALL_CACHE_DIRS NIXPACKS_BUILD_CACHE_DIRS -o ../rw-out .

This gives me the following Dockerfile

FROM ghcr.io/railwayapp/nixpacks:debian-1657820962

WORKDIR /app/

# Setup
COPY environment.nix /app/
RUN nix-env -if environment.nix
RUN apt-get update && apt-get install -y libgmp-dev gcc binutils make zlib1g-dev

# Load environment variables
ARG NIXPACKS_BUILD_CACHE_DIRS NIXPACKS_INSTALL_CACHE_DIRS
ENV NIXPACKS_BUILD_CACHE_DIRS=$NIXPACKS_BUILD_CACHE_DIRS NIXPACKS_INSTALL_CACHE_DIRS=$NIXPACKS_INSTALL_CACHE_DIRS

# Install
COPY . /app/
RUN --mount=type=cache,id=zUXEhU0xtPA-/root/.stack,target=/root/.stack stack setup

# Build

RUN --mount=type=cache,id=zUXEhU0xtPA-/app/.stack-work,target=/app/.stack-work stack build

# Start
COPY . /app/
CMD stack exec railway-exe

This works building locally, but the cache doesn't appear to make any speed up (ghc is still downloaded and all library dependencies are rebuilt).

I think the haskell might work better if it did stack install and then called the executable from /root/local/bin (I think).

Or even better - use a two stage build and just copy the executable to the (much smaller) image that's pushed.

To reproduce

Use the command above to build a local docker file

Expected behavior

Caching should work

Environment

ubuntu

❯ nixpacks --version
nixpacks 0.2.4

[commandodev]

OK, looks like I've cracked it:

NIXPACKS_BUILD_CACHE_DIRS:   /root/.stack,/app/.stack-work
NIXPACKS_BUILD_CMD:          stack install
NIXPACKS_INSTALL_CACHE_DIRS: /root/.stack
NIXPACKS_START_CMD:          /root/.local/bin/railway-exe

Build time: 7.94 seconds

Key changes:

Cache the /root/.stack and /app/.stack-work directories
Change build to stack install (I think this should be the default behaviour)
Change the run command to /root/.local/bin/<exe-name> the $(head packageYaml.executables) trick should work here to set the default.

Optional improvements:

Multi stage build that just copies the /root/.local/bin directory to the image that's published.

cc @coffee-cup

[coffee-cup]

Incredible! I can make that change today (unless you want to take it on). Thanks for doing the hard work and figuring this out.

[commandodev]

Feel free - I'm not a rust native just yet I'm afraid 😁

…

On Fri, 15 Jul 2022, 17:45 Jake Runzer, ***@***.***> wrote: Incredible! I can make that change today (unless you want to take it on). Thanks for doing the hard work and figuring this out. — Reply to this email directly, view it on GitHub <#318 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAVNVCAZ4AJUNKO2XLEOUTVUGISLANCNFSM53U32U6Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[aarnphm]

actually contributing for this is relatively straightforward. You can take a look at src/providers/haskell.rs 😃