Storage template stuck on default, no errors in logs

[Soukyuu]

The bug

Getting started with immich and I can't seem to make it apply my storage template with 1.120.1. There are no errors related to running the storage template job in the logs.

The OS that Immich Server is running on

Debian bookworm

Version of Immich Server

v1.120.1

Version of Immich Mobile App

v1.120.1

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich
services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:2283
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false
  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2ba50e1ac3a0ea17b736ce9db2b0a9f6f8b85d4c27d5f5accc6a416d8f42c6d5
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: --data-checksums
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' ||
        exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}'
        --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT
        COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo
        "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command:
      - postgres
      - -c
      - shared_preload_libraries=vectors.so
      - -c
      - search_path="$$user", public, vectors
      - -c
      - logging_collector=on
      - -c
      - max_wal_size=2GB
      - -c
      - shared_buffers=512MB
      - -c
      - wal_compression=on
    restart: always
volumes:
  model-cache: null
networks: {}

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./data
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Europe/Berlin

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=redacted

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

• start with a fresh instance
• run onboarding
• enable and adapt storage template from default (does not matter which)
• upload some photos
• check the library directory for folder structure

-> the default storage template is used

• run storage template migration job

-> the default storage template is still used

• change storage template to one of the pre-defined values
• run the storage template migration job

-> the default storage template is still being used

Relevant log output

[Nest] 6  - 11/10/2024, 11:59:59 AM     LOG [Microservices:SystemConfigService] LogLevel=log (set via system config)
[Nest] 6  - 11/10/2024, 12:00:09 PM     LOG [Microservices:StorageTemplateService] Starting storage template migration
[Nest] 6  - 11/10/2024, 12:00:19 PM     LOG [Microservices:StorageTemplateService] Finished storage template migration
[Nest] 6  - 11/10/2024, 12:04:46 PM     LOG [Microservices:StorageTemplateService] Starting storage template migration
[Nest] 6  - 11/10/2024, 12:04:56 PM     LOG [Microservices:StorageTemplateService] Finished storage template migration

Additional information

The setup is as follows:

• proxmox node running two VMs
  • trueNAS VM with SATA controller passed through handling the storage/NFS/SMB shares
  • debian bookworm VM with docker containers, also containing the immich container
• the library folder is hosted on an NFS share with no_root_squash option enabled
• permissions look fine to me, immich seems to be creating the files and folders with root:root and 755, so I expect it being able to modify/move the files

[bo0tzz]

Can you set the log level to verbose, run the job, and post the logs here? Also please share a screenshot of your storage template settings.

[Soukyuu]

Sure:

[Nest] 6  - 11/10/2024, 12:38:27 PM     LOG [Microservices:StorageTemplateService] Starting storage template migration
[Nest] 16  - 11/10/2024, 12:38:27 PM   DEBUG [Api:LoggingInterceptor~dpm93fya] PUT /api/jobs/storageTemplateMigration 200 4.34ms ::ffff:172.24.0.1
[Nest] 16  - 11/10/2024, 12:38:27 PM VERBOSE [Api:LoggingInterceptor~dpm93fya] {"command":"start","force":false}
[Nest] 16  - 11/10/2024, 12:38:28 PM   DEBUG [Api:LoggingInterceptor~sqsun5si] GET /api/jobs 200 50.31ms ::ffff:172.24.0.1
[Nest] 16  - 11/10/2024, 12:38:33 PM   DEBUG [Api:LoggingInterceptor~h3ko4ihr] GET /api/jobs 200 12.10ms ::ffff:172.24.0.1
[Nest] 6  - 11/10/2024, 12:38:36 PM   DEBUG [Microservices:StorageTemplateService] Cleaning up empty directories...
[Nest] 6  - 11/10/2024, 12:38:38 PM     LOG [Microservices:StorageTemplateService] Finished storage template migration
[Nest] 16  - 11/10/2024, 12:38:38 PM   DEBUG [Api:LoggingInterceptor~9cscqk6n] GET /api/jobs 200 11.77ms ::ffff:172.24.0.1

[Soukyuu]

Hmm, I don't know what happened, but after rebooting the whole PC I had some troubles with my mount order. After fixing them (and removing the bind mount) the storage template migration is now running correctly.

So yeah, if someone finds this: if you are using bind mounts, try removing them

[lpil]

I too am having this problem! Could we reopen please.

[Soukyuu]

@lpil: are you mounting directories via NFS and/or bind mounts as I was? Share your setup and versions please, maybe there is more to that, then.

[lpil]

I'm using a docker volume like so:

/data/immich/photos:/photos:rw,z

I am not using NFS or such.

After some testing I've found that Immich will use the template if I restart it (Immich, not the whole server) after changing the template in Immich's settings.

[bo0tzz]

@zackpollard I think your config (re)loading refactor may be the issue here.