App keeps forgetting backup albums

[madadam]

The bug

Whenever I restart the app, there are no selected backup albums despite me selecting some before the restart.

The OS that Immich Server is running on

OSMC (Debian) on Raspberry PI 4

Version of Immich Server

v1.65.0

Version of Immich Mobile App

v1.65.0 build.88

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    command: ["start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:release
    command: ["start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

# EDIT: Machine learning runs on a different machine
#  immich-machine-learning:
#    container_name: immich_machine_learning
#    image: ghcr.io/immich-app/immich-machine-learning:release
#    volumes:
#      - ${UPLOAD_LOCATION}:/usr/src/app/upload
#      - model-cache:/cache
#    env_file:
#      - .env
#    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:release
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:release
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    depends_on:
      - immich-server
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

###################################################################################
# Database
###################################################################################

# NOTE: The following four database variables support Docker secrets by adding a *_FILE suffix to the variable name
# See the docker-compose documentation on secrets for additional details: https://docs.docker.com/compose/compose-file/compose-file-v3/#secrets
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# REDIS_URL will be used to pass custom options to ioredis.
# Example for Sentinel
# {"sentinels":[{"host":"redis-sentinel-node-0","port":26379},{"host":"redis-sentinel-node-1","port":26379},{"host":"redis-sentinel-node-2","port":26379}],"name":"redis-sentinel"}
# REDIS_URL=ioredis://eyJzZW50aW5lbHMiOlt7Imhvc3QiOiJyZWRpcy1zZW50aW5lbDEiLCJwb3J0IjoyNjM3OX0seyJob3N0IjoicmVkaXMtc2VudGluZWwyIiwicG9ydCI6MjYzNzl9XSwibmFtZSI6Im15bWFzdGVyIn0=

# Optional Redis settings:

# Note: these parameters are not automatically passed to the Redis Container
# to do so, please edit the docker-compose.yml file as well. Redis is not configured
# via environment variables, only redis.conf or the command line

# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_USERNAME=
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=/media/sklad/photos/immich


###################################################################################
# Typesense
###################################################################################
TYPESENSE_API_KEY=eKr/DBr9zgOMCoi7XmQCbHAqiBKUC00kXx9jHxezxmV56Jhq3Uq9ELsYtbvXfKrT4VJjFBbgU9fB1oxmgLmFVw==
# TYPESENSE_ENABLED=false
# TYPESENSE_URL uses base64 encoding for the nodes json.
# Example JSON that was used:
# [
#      { 'host': 'typesense-1.example.net', 'port': '443', 'protocol': 'https' },
#      { 'host': 'typesense-2.example.net', 'port': '443', 'protocol': 'https' },
#      { 'host': 'typesense-3.example.net', 'port': '443', 'protocol': 'https' },
#  ]
# TYPESENSE_URL=ha://WwogICAgeyAnaG9zdCc6ICd0eXBlc2Vuc2UtMS5leGFtcGxlLm5ldCcsICdwb3J0JzogJzQ0MycsICdwcm90b2NvbCc6ICdodHRwcycgfSwKICAgIHsgJ2hvc3QnOiAndHlwZXNlbnNlLTIuZXhhbXBsZS5uZXQnLCAncG9ydCc6ICc0NDMnLCAncHJvdG9jb2wnOiAnaHR0cHMnIH0sCiAgICB7ICdob3N0JzogJ3R5cGVzZW5zZS0zLmV4YW1wbGUubmV0JywgJ3BvcnQnOiAnNDQzJywgJ3Byb3RvY29sJzogJ2h0dHBzJyB9LApd

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001

# IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003
IMMICH_MACHINE_LEARNING_URL=http://192.168.1.112:3003

####################################################################################
# Alternative API's External Address - Optional
#
# This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
# You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
# NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
# Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

Reproduction steps

1. Start the app
2. Go to "Backup" and select some backup albums
3. Stop the app (swipe it off)
4. Start the app again
5. Go to "Backup"
6. Observe no backup albums are selected

Additional information

App log:

created_at,level,context,message,stacktrace
2023-07-04 21:23:25.307307,LogLevel.SEVERE,"BackupNotifier","Failed to generate album from id","#0      AssetPathEntity.obtainPathFromProperties (package:photo_manager/src/types/entity.dart:139)
<asynchronous suspension>
#1      AssetPathEntity.fromId (package:photo_manager/src/types/entity.dart:50)
<asynchronous suspension>
#2      BackupNotifier._getBackupAlbumsInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:276)
<asynchronous suspension>
#3      BackupNotifier.getBackupInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:378)
<asynchronous suspension>
#4      BackupNotifier.startBackupProcess (package:immich_mobile/modules/backup/providers/backup.provider.dart:424)
<asynchronous suspension>
#5      BackupNotifier._resumeBackup (package:immich_mobile/modules/backup/providers/backup.provider.dart:573)
<asynchronous suspension>
"
2023-07-04 21:23:23.877612,LogLevel.INFO,"AlbumService","Syncing completed. Changes: true",""
2023-07-04 21:23:23.877420,LogLevel.INFO,"SyncService","Added a new local album to DB: Domcek",""
2023-07-04 21:23:23.860068,LogLevel.INFO,"SyncService","Upserted 1 assets into the DB",""
2023-07-04 21:23:23.851915,LogLevel.INFO,"SyncService","26 assets already existed in DB, to upsert 1",""
2023-07-04 21:23:23.809714,LogLevel.INFO,"SyncService","Syncing a new local album to DB: Domcek",""
2023-07-04 21:23:23.809544,LogLevel.INFO,"SyncService","Added a new local album to DB: DCIM",""
2023-07-04 21:23:23.774247,LogLevel.INFO,"SyncService","228 assets already existed in DB, to upsert 0",""
2023-07-04 21:23:23.557727,LogLevel.INFO,"SyncService","Syncing a new local album to DB: DCIM",""
2023-07-04 21:23:22.663699,LogLevel.INFO,"AlbumService","'Recents' is not selected, keeping only selected albums",""
2023-07-04 21:23:22.663386,LogLevel.INFO,"AlbumService","Found 17 device albums",""
2023-07-04 21:23:21.892299,LogLevel.INFO,"BackupNotifier","Found 17 local albums",""
2023-07-04 21:23:21.812240,LogLevel.INFO,"BackupNotifier","[_resumeBackup] Start back up",""
2023-07-04 21:23:12.144143,LogLevel.SEVERE,"BackupNotifier","Failed to generate album from id","#0      AssetPathEntity.obtainPathFromProperties (package:photo_manager/src/types/entity.dart:139)
<asynchronous suspension>
#1      AssetPathEntity.fromId (package:photo_manager/src/types/entity.dart:50)
<asynchronous suspension>
#2      BackupNotifier._getBackupAlbumsInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:276)
<asynchronous suspension>
#3      BackupNotifier.getBackupInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:378)
<asynchronous suspension>
"
2023-07-04 21:23:09.097362,LogLevel.INFO,"BackupNotifier","Found 17 local albums",""
2023-07-04 21:23:01.310300,LogLevel.SEVERE,"BackupNotifier","Failed to generate album from id","#0      AssetPathEntity.obtainPathFromProperties (package:photo_manager/src/types/entity.dart:139)
<asynchronous suspension>
#1      AssetPathEntity.fromId (package:photo_manager/src/types/entity.dart:50)
<asynchronous suspension>
#2      BackupNotifier._getBackupAlbumsInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:276)
<asynchronous suspension>
#3      BackupNotifier.getBackupInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:378)
<asynchronous suspension>
"
2023-07-04 21:22:59.817237,LogLevel.INFO,"AssetNotifier","Load assets: 14843ms",""
2023-07-04 21:22:59.803838,LogLevel.INFO,"AlbumService","Syncing completed. Changes: true",""
2023-07-04 21:22:59.803650,LogLevel.INFO,"SyncService","Removed 1 and updated 0 local assets from DB",""
2023-07-04 21:22:59.795279,LogLevel.INFO,"SyncService","Removed local album Domcek from DB",""
2023-07-04 21:22:59.786813,LogLevel.INFO,"SyncService","Removing local album Domcek from DB",""
2023-07-04 21:22:59.786639,LogLevel.INFO,"SyncService","Removed local album DCIM from DB",""
2023-07-04 21:22:59.672628,LogLevel.INFO,"SyncService","Removing local album DCIM from DB",""
2023-07-04 21:22:58.925626,LogLevel.INFO,"AlbumService","'Recents' is not selected, keeping only selected albums",""
2023-07-04 21:22:58.925375,LogLevel.INFO,"AlbumService","Found 17 device albums",""
2023-07-04 21:22:53.448789,LogLevel.INFO,"BackupNotifier","Found 17 local albums",""
2023-07-04 21:22:52.616524,LogLevel.INFO,"BackupNotifier","No Asset On Device - Abort Backup Process",""
2023-07-04 21:22:52.588554,LogLevel.INFO,"BackupNotifier","Not found albums or assets on the device to backup",""
2023-07-04 21:22:51.048576,LogLevel.SEVERE,"BackupNotifier","Failed to generate album from id","#0      AssetPathEntity.obtainPathFromProperties (package:photo_manager/src/types/entity.dart:139)
<asynchronous suspension>
#1      AssetPathEntity.fromId (package:photo_manager/src/types/entity.dart:50)
<asynchronous suspension>
#2      BackupNotifier._getBackupAlbumsInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:276)
<asynchronous suspension>
#3      BackupNotifier.getBackupInfo (package:immich_mobile/modules/backup/providers/backup.provider.dart:378)
<asynchronous suspension>
#4      BackupNotifier.startBackupProcess (package:immich_mobile/modules/backup/providers/backup.provider.dart:424)
<asynchronous suspension>
#5      BackupNotifier._resumeBackup (package:immich_mobile/modules/backup/providers/backup.provider.dart:573)
<asynchronous suspension>
"
2023-07-04 21:22:46.435558,LogLevel.INFO,"AlbumService","Syncing completed. Changes: false",""
2023-07-04 21:22:45.510646,LogLevel.INFO,"AlbumService","'Recents' is not selected, keeping only selected albums",""
2023-07-04 21:22:45.510402,LogLevel.INFO,"AlbumService","Found 17 device albums",""
2023-07-04 21:22:44.687665,LogLevel.INFO,"BackupNotifier","Found 17 local albums",""
2023-07-04 21:22:44.608238,LogLevel.INFO,"BackupNotifier","[_resumeBackup] Start back up",""
2023-07-04 21:22:39.855382,LogLevel.INFO,"BackupNotifier","No Asset On Device - Abort Backup Process",""
2023-07-04 21:22:39.843066,LogLevel.INFO,"BackupNotifier","Not found albums or assets on the device to backup",""

[justinvdk]

I'm having the same issue on a Samsung SM-A520F (Galaxy A5 (2017)).
Same repro steps and similar logs (same function that fails).

The flutter documentation about obtainPathFromProperties mentions "Method not recommended for users" but doesn't elaborate; perhaps this function is unstable for Immich's usecase?

Nevermind above; Immich is not using that function; I did not read the stack trace carefully enough.

[madadam]

This seems to be caused by the photo_manager library. For some reason AssetPathEntity.fromId throws even when called with an id from an AssetPathEntity returned from earlier call to PhotoManager.getAssetPathList(). Seems like a bug to me but haven't looked too deeply into it. There is a simple workaround though. Instead of calling AssetPathEntity.fromId (e.g. here: https://github.com/immich-app/immich/blob/main/mobile/lib/modules/backup/providers/backup.provider.dart#L276) we can look it up from the albums list created earlier here: https://github.com/immich-app/immich/blob/main/mobile/lib/modules/backup/providers/backup.provider.dart#L205. We can even speed it up a bit by creating a map of id => AssetPathEntity beforehand.

I'll create a PR in a bit.

[justinvdk]

Ah you're one step ahead of me. I just got around building a sample app that lists all albums and their IDs. On the phone I got this issue on it does list stable IDs so indeed, this is an actual bug in photo_manager.
Your change makes a lot of sense imho, no reason to do an extra lookup.

[julianmojico]

I can reproduce this bug, because anytime I start the app, it has to fetch the backup status and it delays a while.

But I'm also seeing discrepancies in the amount of backed up photos. Even negative numbers.

Are this the same bug??
This is a capture from latest Android app 1.69.0 build 92:

[justinvdk]

Hi julianmojico,

This bug should've been fixed with v1.68.0.
Both the Google Play Store and f-droid should have up-to-date version.
Can you reproduce the bug on that version (or newer) as well?

The negative remainder is something I have seen as well, but this should be unrelated. I don't think the exact cause is known yet according to #2567.

[justinvdk]

@madadam I haven't been able to reproduce the issue since v1.68.0. You fix has resolved it for me. Can you close the issue if it has resolved yours as well?

[androsoftweb]

this is still problem on 1.92.2+119.
seems like albums to backup is not stored locally, and when that page refreshes, selected and filtered backup albums resets.

[alextran1502]

@androsoftweb I don't believe the app forgot the album for backup; I think the problem is the state doesn't properly show on the app. Can you try to re-enter the backup screen a couple of times to see if it eventually pulls up the selected albums?

[androsoftweb]

Well i tried a lot and each time it resets to no albums selected at all.
And the reason behind my suspection is due to the fact that no albums are backing up.
My foreground and background backups are on.
It is behaving same in my other device as well, when i initially added camera folder for backup, it backs up correctly to this date, but I can't add or remove backup folders now. Adding new doesn't work, removing camera doesn't work.

[alextran1502]

@androsoftweb that sounds strange, can you record a video?

[androsoftweb]

backup.folder.selection.issue.demo.mp4

There you go!
there might be my fault! but as i said, it was working before cause i added my camera album before.

[alextran1502]

@androsoftweb, your server is extremely out of date. We can only guarantee everything works as expected when the app and the server are on the same version at this stage

[androsoftweb]

upgraded server and its fixes, thanks a lot, i will keep it in mind to upgrade both things regularly, before commenting.

[jdrewsteiner]

I have been experiencing the same behavior since installing immich several months ago. I have the version 1.97.0 build 125 installed on a Pixel 7 and server version 1.97.0. Periodically i notice that my photos aren't being backed up. When I first click on the backup (cloud) icon, it shows no folders being backed up. If I wait a bit and go back into the backup settings the correct backup folders are shown but I have to hit the "start backup" button again.

[thegabriele97]

This happens to me too, I am using latest version of Immich both on server side and andrid side.

It happens when I delete all content in an album (i.e. Camera). If I take a new photo it doesn't get uploaded because everything gets disabled