Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backup job container restarting constantly #190

Open
bread-wolf opened this issue Apr 1, 2024 · 7 comments
Open

Backup job container restarting constantly #190

bread-wolf opened this issue Apr 1, 2024 · 7 comments
Labels

Comments

@bread-wolf
Copy link

I use the default example docker compose to run backup, prune and check containers, the backup container creates a backup on startup, then exits when the backup is done, then restarts due to the "unless-stopped" restart policy. It creates backups in a loop.

@bread-wolf
Copy link
Author

backup-1 | Executing backup on startup ...
backup-1 |
backup-1 | Starting Backup at 2024-04-01 13:06:04
backup-1 | rclone: 2024/04/01 13:06:04 NOTICE: Chunks temporary upload directory: [...]
backup-1 | using parent snapshot 56875acc
check-1 | Load(<lock/907a512920>, 0, 0) returned error, retrying after 5.903144332s: <lock/907a512920> does not exist
backup-1 |
backup-1 | Files: 0 new, 7 changed, 16711 unmodified
backup-1 | Dirs: 0 new, 13 changed, 965 unmodified
backup-1 | Added to the repository: 5.189 MiB (213.669 KiB stored)
backup-1 |
backup-1 | processed 16718 files, 1.490 GiB in 0:04
backup-1 | snapshot f34c87a0 saved
backup-1 | Backup successful
backup-1 | Forget about old snapshots based on RESTIC_FORGET_ARGS = --keep-last 10 --keep-daily 7 --keep-weekly 5 --keep-monthly 12
backup-1 | rclone: 2024/04/01 13:06:10 NOTICE: Chunks temporary upload directory: [...]
backup-1 | repo already locked, waiting up to 0s for the lock
backup-1 | unable to create lock in backend: repository is already locked by PID 39 on docker by root (UID 0, GID 0)
backup-1 | lock was created at 2024-04-01 12:54:23 (11m48.604414721s ago)
backup-1 | storage ID 9884566f
backup-1 | the unlock command can be used to remove stale locks
backup-1 |
backup-1 exited with code 1

Here's container output when I attach, container exits due to an error, I am not sure how to solve that one.
Maybe the restart policy should be changed, as these errors can lead to a lot of data usage on the backupt server.

@5a5cha
Copy link

5a5cha commented Apr 2, 2024

Hi, did you solve your issue?

I ran into a nearly identical problem with the default compose settings.

For me as a newbie it looks like there is some sort of lock between the backup-1 and check-1.
Maybe because both services are RUN_ON_STARTUP: "true".

My solution was to only set the RUN_ON_STARTUP: "true" for the backup service.
Makes no sense to me to start a backup and check of the repo at same time.

After that i fired up the compose and connected to each container and checkt with the "unlock" command that there is no lock left in the repo.

Runs smoothly for weeks since that modification in the compose file.

@bread-wolf
Copy link
Author

I did the same thing, forgot to comment here, I also set RUN_ON_STARTUP to false for the check container because otherwise there is a conflict with restic's container locks.

This issue is new, I have been using resticker for a few years now, it has to be due to an update to restic?

In any case, I will prepare a PR with additional documentation as a lot of things are still sparse or no documented at all like the RESTIC_PASSWORD_FILE flag that enables the use of secrets to specify the repository password.

@djmaze
Copy link
Owner

djmaze commented Apr 9, 2024

I don't understand: RUN_ON_STARTUP is already false for the check container in the example configs.

@djmaze djmaze added the bug label Apr 9, 2024
@benjamin051000
Copy link

I believe this is a duplicate of #171

@benjamin051000
Copy link

I did the same thing, forgot to comment here, I also set RUN_ON_STARTUP to false for the check container because otherwise there is a conflict with restic's container locks.

This does not solve the problem for me. I have check and prune's RUN_ON_STARTUP: false in my docker-compose.yml and still get the error

@djmaze
Copy link
Owner

djmaze commented Aug 9, 2024

@benjamin051000 Can you show an error log? How old is the reported lock? Unlocking does not solve the issue for you?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants