Validate pipeline for aws20 container

[scottwittenburg]

No description provided.

[scottwittenburg]

The regex-based branch automated build isn't working, and I'm wondering if it could be related to this issue.

[scottwittenburg]

The regex-based branch automated build isn't working, and I'm wondering if it could be related to this issue.

Although who knows, is it possible Docker Hub is waiting for this recently triggered gitlab check to finish? For some reason pushing the same sha to a different branch name (one that should match the autobuild rule) also re-triggered the gitlab pipeline, even though that new branch name is not associated with this PR.

[becker33]

A couple corrections on the compilers.

[becker33]

I think we need a gcc-6.5 in here. We need to have multiple system gccs.

[scottwittenburg]

Ok, I'll add 6.5.0

[becker33]

I think this should be gcc@6.5.0. There's no gcc@7.5.0 on the system when the users log in, and this isn't a compiler that we bootstrap during the tutorial.

[scottwittenburg]

I picked gcc@7.5.0, because that's what is installed when the Dockerfile installs gcc, g++ and gfortran. But if I'm going to specifically install 6.5.0 too, then I can change the 4.8 in the test script to 6.5.0...

[becker33]

Yeah the whole point of this one is it's demonstrating changing the compiler version, so it needs to be different from the spec of the default gcc.

[becker33]

In the most recent version of the tutorial this was /hmvj. It needs to be the hash of the zlib@1.2.8 cppflags=-O3 spec.

[scottwittenburg]

Ok, thanks for the heads up, I'll see what the hash is now and set it to that.

[scottwittenburg]

@becker33 IIRC the targets -> all -> [x86_64] in this spack.yaml was needed for compiler bootstrapping to work correctly. This is resulting in the buildcaches all being designated that way. Does that mean we need a packages.yaml in the container specifying the same thing? Right now, commands that should install from the binary mirror aren't doing it, I think because of the architecture mismatch ('haswellis the detected arch, but binaries in the mirror arex86_64`).

[becker33]

Yes, there should be a packages.yaml in /etc/spack/ with just that one constraint in it.

[becker33]

We target the whole thing for x86_64 because we can't know what machine they'll run the container on. When the new aarch64 macs become popular we may have to create two versions of the container.

[hartzell]

I wish I'd seen this earlier. Just got someone set up to run the tutorial and ran into a few roadblocks. Summarized it to the mailing list, but adding it here for posterity:

---

Their docker image is missing the file command. Build your own
derivative image that includes it.

mkdir foo
cd foo
vi Dockerfile
docker build -t spack-tutorial-updated .

Here's a functional Dockerfile:

FROM spack/tutorial:latest

USER root

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        file \
    && \
    apt-get autoremove --purge && \
    apt-get clean

USER spack

WORKDIR /home/spack

CMD ["bash"]

Start the new container like so:

docker run -it --rm spack-tutorial-updated

After you start the container and work through the tutorial, you'll
need to adjust a few things.

• After cloning the spack repository and cd-ing into it, you'll need
  to configure it to build for the same target that they used by
  adding this bit to etc/spack/packages.yaml:

  packages:
      all:
        target: [x86_64]

• Instead of adding their S3 bucket as a mirror, use the mirror that
  has been copied into the image:

  spack mirror add local file:///mirror/build_cache/

  and then go ahead and add the key as specified.

• When it first mentions installing zlib w/ gcc@6.5.0, you'll need to
  install gcc@6.5.0 and add it to the list of known compilers (it's
  quick, it'll install from the binary cache):

  spack install gcc@6.5.0
  spack compiler add $(spack location -i gcc@6.5.0)