[BUG] docker-compose run createbuckets fails

[jmcgranahan]

Describe the bug
A clear and concise description of what the bug is.
On an Ubuntu 24.04.1 LTS server within an AWS environment with no other applications running, we are attempting to install the newest version of Avalon since it is compatible with Fedora 6. However, at the step to createbuckets, it always fails:

mc: Unable to initialize new alias from the provided credentials. Get "http://minio:9000/probe-bsign-mk1dfiympem9dhqqjp2mmsftlegtvj/?location=": dial tcp 172.18.0.2:9000: connect: connection refused.

To Reproduce
Steps to reproduce the behavior, including the results:
Launch an Ubuntu 24.04.1 LTS server in AWS using their Graviton processor.
Install Docker - apt install docker
Install Docker-compose - curl -SL "https://github.com/docker/compose/releases/download/v2.32.4/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

check & verify Docker/Docker-compose -
Docker version 27.5.1, build 9f9e405
Docker Compose version v2.32.4

Then:

1. Go to https://github.com/avalonmediasystem/avalon and follow the "Quickstart development with Docker"
2. git clone https://github.com/avalonmediasystem/avalon
3. cd avalon
4. cp config/controlled_vocabulary.yml.example config/controlled_vocabulary.yml
5. docker-compose pull
6. docker-compose run createbuckets

docker-compose run createbuckets
[+] Creating 1/1
✔ Container avalon-minio-1 Created 0.0s
[+] Running 1/1
✔ Container avalon-minio-1 Started 0.5s
mc: Configuration written to /root/.mc/config.json. Please update your access credentials.
mc: Successfully created /root/.mc/share.
mc: Initialized share uploads /root/.mc/share/uploads.json file.
mc: Initialized share downloads /root/.mc/share/downloads.json file.
mc: Unable to initialize new alias from the provided credentials. Get "http://minio:9000/probe-bsign-mk1dfiympem9dhqqjp2mmsftlegtvj/?location=": dial tcp 172.18.0.2:9000: connect: connection refused.
Bucket created successfully myminio/fcrepo.
Bucket created successfully myminio/masterfiles.
Bucket created successfully myminio/derivatives.
Bucket created successfully myminio/supplementalfiles.
Bucket created successfully myminio/preserves.
Access permission for myminio/derivatives is set to download
Access permission for myminio/supplementalfiles is set to download

Expected behavior
Expected to be able to complete the quick start at a minimum in order to test this application.

Environment (please complete the following information):

• Device / platform: Laptop access AWS console
• OS: Laptop OS = Windows 11; OS of server through AWS = Ubuntu 24.04.1
• Browser: Edge Version 132.0.2957.140 (Official build) (64-bit)

Release:
Current release

[cjcolvar]

@jmcgranahan Thanks for submitting this bug!

I have a PR into avalon that should resolve this issue. For now you could make these changes in your docker-compose.yml file and createbuckets should run successfully.

Note that the docker-compose.yml in the avalon repository is meant for development environments. If you're looking for a production environment docker-compose setup then try the avalon-docker repository. Alternatively if you are deploying to AWS and want to make use of more AWS services then the avalon-terraform repository would be a better fit.

[jmcgranahan]

@cjcolvar Thanks for the suggestion. However, after making that change to the docker-compose.yml file, I get this whenever I attempt to run the command:

root@ip-10-55-58-158:/apps/avalon# docker-compose run createbuckets
[+] Creating 1/1
 ✔ Container avalon-minio-1  Created                                                                                                                                                                         0.1s
[+] Running 1/1
 ✔ Container avalon-minio-1  Started                                                                                                                                                                         0.7s
dependency failed to start: container avalon-minio-1 exited (2)

And here is my docker-compose.yml file - createbuckets snippet:

  createbuckets:
    image: minio/mc
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc config host add myminio http://minio:9000 minio minio123;
      /usr/bin/mc mb -p myminio/fcrepo myminio/masterfiles myminio/derivatives myminio/supplementalfiles myminio/preserves;
      /usr/bin/mc anonymous set download myminio/derivatives;
      /usr/bin/mc anonymous set download myminio/supplementalfiles;
      exit 0;
      "
    networks:
      internal:

[cjcolvar]

I would try changing the minio service image to minio/minio (latest tag) then running:

docker-compose down -v  #Shutdown and clean up anything previously created including volumes
docker-compose pull
docker-compose run createbuckets

Does that help?

[jmcgranahan]

@cjcolvar unfortunately, no:

docker-compose.yml:

  createbuckets:
    image: minio/minio
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc config host add myminio http://minio:9000 minio minio123;
      /usr/bin/mc mb -p myminio/fcrepo myminio/masterfiles myminio/derivatives myminio/supplementalfiles myminio/preserves;
      /usr/bin/mc anonymous set download myminio/derivatives;
      /usr/bin/mc anonymous set download myminio/supplementalfiles;
      exit 0;
      "
    networks:
      internal:

root@ip-10-55-58-158:/apps/avalon# docker-compose down -v
[+] Running 5/5
 ✔ Volume avalon_npms      Removed                                                                                                                                    0.0s
 ✔ Volume avalon_data      Removed                                                                                                                                    0.0s
 ✔ Volume avalon_database  Removed                                                                                                                                    0.0s
 ✔ Volume avalon_fedora    Removed                                                                                                                                    0.0s
 ✔ Volume avalon_solr      Removed                                                                                                                                    0.0s
root@ip-10-55-58-158:/apps/avalon# docker-compose pull
[+] Pulling 25/25
 ✔ test Skipped - Image is already being pulled by worker                                                                                                             0.0s
 ✔ avalon Skipped - Image is already being pulled by worker                                                                                                           0.0s
 ✔ solr-test Skipped - Image is already being pulled by solr                                                                                                          0.0s
 ✔ redis Skipped - Image is already being pulled by redis-test                                                                                                        0.0s
 ✔ db-test Skipped - Image is already being pulled by db                                                                                                              0.0s
 ✔ fedora Skipped - Image is already being pulled by fedora-test                                                                                                      0.0s
 ✔ hls Pulled                                                                                                                                                         0.6s
 ✔ db Pulled                                                                                                                                                          0.6s
 ✔ cypress Pulled                                                                                                                                                     0.7s
 ✔ worker Pulled                                                                                                                                                      0.7s
 ✔ createbuckets Pulled                                                                                                                                               6.0s
   ✔ 6e48e364a98d Already exists                                                                                                                                      0.0s
   ✔ 8fd4d1fe6b72 Already exists                                                                                                                                      0.0s
   ✔ 5189eab82378 Pull complete                                                                                                                                       0.8s
   ✔ 1f6ec44106d6 Pull complete                                                                                                                                       0.8s
   ✔ 7cb30f1b4b09 Pull complete                                                                                                                                       4.7s
   ✔ 236d64fc11cf Pull complete                                                                                                                                       4.8s
   ✔ b8351f9cbedd Pull complete                                                                                                                                       4.9s
   ✔ fcdafa6a3a86 Pull complete                                                                                                                                       4.9s
   ✔ 3e9007ae0b36 Pull complete                                                                                                                                       4.9s
   ✔ 7e7f18a032b8 Pull complete                                                                                                                                       4.9s
 ✔ fedora-test Pulled                                                                                                                                                 0.7s
 ✔ minio Pulled                                                                                                                                                       0.7s
 ✔ solr Pulled                                                                                                                                                        0.7s
 ✔ redis-test Pulled                                                                                                                                                  0.6s
root@ip-10-55-58-158:/apps/avalon# docker-compose run createbuckets
[+] Creating 4/4
 ✔ Network avalon_external   Created                                                                                                                                  0.1s
 ✔ Network avalon_internal   Created                                                                                                                                  0.1s
 ✔ Volume "avalon_data"      Created                                                                                                                                  0.0s
 ✔ Container avalon-minio-1  Created                                                                                                                                  0.1s
[+] Running 1/1
 ✔ Container avalon-minio-1  Started                                                                                                                                  0.3s
dependency failed to start: container avalon-minio-1 exited (2)

[cjcolvar]

@jmcgranahan
How about this in your docker-compose.yml?
(I changed the minio service image to minio/minio and the createbuckets service image to minio/mc.)

  minio:
    image: minio/minio
    command: minio server /data --console-address ":9090"
    environment:
      MINIO_ACCESS_KEY: minio
      MINIO_SECRET_KEY: minio123
    volumes:
      - data:/data
    ports:
      - 9000:9000
      - 9090:9090
    networks:
      internal:
      external:
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 20s
      retries: 3

  createbuckets:
    image: minio/mc
    depends_on:
      minio:
        condition: service_healthy
    entrypoint: >
      /bin/sh -c "
      /usr/bin/mc config host add myminio http://minio:9000 minio minio123;
      /usr/bin/mc mb -p myminio/fcrepo myminio/masterfiles myminio/derivatives myminio/supplementalfiles myminio/preserves;
      /usr/bin/mc anonymous set download myminio/derivatives;
      /usr/bin/mc anonymous set download myminio/supplementalfiles;
      exit 0;
      "
    networks:
      internal:

[jmcgranahan]

@cjcolvar Success - well, at least we are past that step. Now I'm running into a different problem. We're using AWS's Graviton processor which uses the aarm cores and apparently some of the packages are not compatible. I attempted adding this - platform: linux/arm64/v8 - under Services: but that didn't work

docker-compose down -v
[+] Running 5/5
✔ Volume avalon_npms      Removed                                                                                                                                    0.0s
✔ Volume avalon_data      Removed                                                                                                                                    0.0s
✔ Volume avalon_fedora    Removed                                                                                                                                    0.0s
✔ Volume avalon_database  Removed                                                                                                                                    0.0s
✔ Volume avalon_solr      Removed                                                                                                                                    0.0s
root@ip-10-55-58-158:/apps/avalon# docker-compose pull
[+] Pulling 15/15
✔ solr Skipped - Image is already being pulled by solr-test                                                                                                          0.0s
✔ db-test Skipped - Image is already being pulled by db                                                                                                              0.0s
✔ test Skipped - Image is already being pulled by worker                                                                                                             0.0s
✔ avalon Skipped - Image is already being pulled by worker                                                                                                           0.0s
✔ redis-test Skipped - Image is already being pulled by redis                                                                                                        0.0s
✔ fedora-test Skipped - Image is already being pulled by fedora                                                                                                      0.0s
✔ fedora Pulled                                                                                                                                                      0.7s
✔ worker Pulled                                                                                                                                                      0.8s
✔ solr-test Pulled                                                                                                                                                   0.7s
✔ minio Pulled                                                                                                                                                       0.7s
✔ db Pulled                                                                                                                                                          0.7s
✔ redis Pulled                                                                                                                                                       0.7s
✔ createbuckets Pulled                                                                                                                                               0.7s
✔ hls Pulled                                                                                                                                                         0.7s
✔ cypress Pulled                                                                                                                                                     0.7s
root@ip-10-55-58-158:/apps/avalon# docker-compose run createbuckets
[+] Creating 4/4
✔ Network avalon_internal   Created                                                                                                                                  0.1s
✔ Network avalon_external   Created                                                                                                                                  0.1s
✔ Volume "avalon_data"      Created                                                                                                                                  0.0s
✔ Container avalon-minio-1  Created                                                                                                                                  0.1s
[+] Running 1/1
✔ Container avalon-minio-1  Started                                                                                                                                  0.6s
mc: Configuration written to `/root/.mc/config.json`. Please update your access credentials.
mc: Successfully created `/root/.mc/share`.
mc: Initialized share uploads `/root/.mc/share/uploads.json` file.
mc: Initialized share downloads `/root/.mc/share/downloads.json` file.
Added `myminio` successfully.
Bucket created successfully `myminio/fcrepo`.
Bucket created successfully `myminio/masterfiles`.
Bucket created successfully `myminio/derivatives`.
Bucket created successfully `myminio/supplementalfiles`.
Bucket created successfully `myminio/preserves`.
Access permission for `myminio/derivatives` is set to `download`
Access permission for `myminio/supplementalfiles` is set to `download`
root@ip-10-55-58-158:/apps/avalon# docker-compose up avalon worker --remove-orphans
[+] Running 16/16
✔ Volume "avalon_npms"                                                                                                                                  Created      0.0s
✔ Volume "avalon_fedora"                                                                                                                                Created      0.0s
✔ Volume "avalon_solr"                                                                                                                                  Created      0.0s
✔ Volume "avalon_database"                                                                                                                              Created      0.0s
✔ Container avalon-createbuckets-run-ee633c6d2be3                                                                                                       Removed      0.0s
✔ Container avalon-hls-1                                                                                                                                Created      0.1s
✔ Container avalon-minio-1                                                                                                                              Running      0.0s
✔ Container avalon-redis-1                                                                                                                              Created      0.1s
✔ Container avalon-solr-1                                                                                                                               Created      0.1s
✔ Container avalon-db-1                                                                                                                                 Created      0.1s
✔ Container avalon-fedora-1                                                                                                                             Created      0.1s
**! hls The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested**                 0.0s
✔ Container avalon-worker-1                                                                                                                             Created      0.1s
✔ Container avalon-avalon-1                                                                                                                             Created      0.1s
! worker The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested              0.0s
! avalon The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested              **0.0s**
Attaching to avalon-1, worker-1
**worker-1  | exec /usr/bin/dumb-init: exec format error
avalon-1  | exec /bin/bash: exec format error
worker-1 exited with code 255
avalon-1 exited with code 255**

[cjcolvar]

This is new to me. My guess is that you'll probably need to rebuild those images in order for them to work with that platform: docker-compose build avalon. If that works for the avalon image (also used by the worker) then I'll figure out how you can rebuild the hls image.