Nextcloud Upgrade purges mounted external local storage via rsync #1528

mirisbowring · 2021-07-04T13:20:58Z

Hi,
I've reported this issue already but it got removed to the docker repository #1493 (which is false in this case). See details below!

Steps to reproduce

Vanilla Nextcloud instance
mount external local storage to nextcloud instance
add files
upgrade nextcloud to next version when available

Now, when starting the application, rsync delete gets called on EVERY file on my external local storage.
Due to this Bug I lost 2 Times about 16 TB of Data. (Upgrade to 21.0.1 and to 21.0.2).
After the second time i mounted the storage as read only. Rsync now throws the error that it cant delete the file. Therefore Nextcloud wont start.

WHY does nextcloud try to delete the external storage???

Expected behaviour

The files should NOT be touched or even DELETED!

Actual behaviour

Nextcloud tries to wipe the whole external storage!

Server configuration

Operating system: Ubuntu 20.04 (Docker-Compose - default config)

Web server: Apache

Database: MariaDB

PHP version: 7.4.21

Nextcloud version: 21.0.3

Updated from an older Nextcloud/ownCloud or fresh install: from 21.0.2 / 21.0.1

Where did you install Nextcloud from: official docker repo

Signing status: cannot login because app won't start due to failing rsync delete

List of activated apps:

App list

Enabled:
 - accessibility: 1.7.0
 - activity: 2.14.3
 - breezedark: 21.0.8
 - bruteforcesettings: 2.2.0
 - checksum: 1.1.2
 - cloud_federation_api: 1.4.0
 - comments: 1.11.0
 - contactsinteraction: 1.2.0
 - dav: 1.17.1
 - extract: 1.3.1
 - federatedfilesharing: 1.11.0
 - federation: 1.11.0
 - files: 1.16.0
  - files_downloadactivity: 1.10.0
  - files_external: 1.12.0
  - files_pdfviewer: 2.1.0
  - files_rightclick: 1.0.0
  - files_sharing: 1.13.1
  - files_trackdownloads: 1.10.0
  - files_trashbin: 1.11.0
  - files_versions: 1.14.0
  - files_videoplayer: 1.10.0
  - firstrunwizard: 2.10.0
  - group_everyone: 0.1.7
  - groupfolders: 9.0.2
  - logreader: 2.6.0
  - lookup_server_connector: 1.9.0
  - metadata: 0.13.0
  - notifications: 2.9.0
  - oauth2: 1.9.0
  - password_policy: 1.11.0
  - provisioning_api: 1.11.0
  - serverinfo: 1.11.0
  - settings: 1.3.0
  - sharebymail: 1.11.0
  - systemtags: 1.11.0
  - text: 3.2.0
  - theming: 1.12.0
  - twofactor_backupcodes: 1.10.0
  - twofactor_webauthn: 0.2.10
  - updatenotification: 1.11.0
  - viewer: 1.5.0
  - workflowengine: 2.3.0
Disabled:
  - admin_audit
  - dashboard
  - encryption
  - nextcloud_announcements
  - photos
  - previewgenerator
  - privacy
  - recommendations
  - support
  - survey_client
  - user_ldap
  - user_status
  - weather_status

Nextcloud configuration:

Config report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwritehost": "nextcloud.primboard.de",
        "overwriteprotocol": "https",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.homelab-01.primboard.de"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "21.0.2.1",
        "overwrite.cli.url": "https:\/\/nextcloud.homelab-01.primboard.de",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "PLAIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls",
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "loglevel": 2,
        "maintenance": false
    }
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: NO

Client configuration

Browser: Chrome

Operating system: MacOS

Logs

Configuring Redis as session handler
Initializing nextcloud 21.0.3.1 ...
Upgrading nextcloud from 21.0.2.1 ...
cannot delete non-empty directory: moviedb1/Movies/Movie 1/Sample
cannot delete non-empty directory: moviedb1/Movies/Movie 1
cannot delete non-empty directory: moviedb1/Movies/Movie 2/Sample
cannot delete non-empty directory: moviedb1/Movies/Movie 2
cannot delete non-empty directory: moviedb1/Movies/Movie 3/Sample
cannot delete non-empty directory: moviedb1/Movies/Movie 3
cannot delete non-empty directory: moviedb1/Movies/Movie 4
cannot delete non-empty directory: moviedb1/Movies/Movie 5
cannot delete non-empty directory: moviedb1/Movies/Movie 6
cannot delete non-empty directory: moviedb1/Movies/Movie 7/Sample
cannot delete non-empty directory: moviedb1/Movies/Movie 7
cannot delete non-empty directory: moviedb1/Movies/Movie 8
cannot delete non-empty directory: moviedb1/Movies/Movie 9
cannot delete non-empty directory: moviedb1/Movies/Movie 10/Sample
cannot delete non-empty directory: moviedb1/Movies/Movie 10
cannot delete non-empty directory: moviedb1/Movies
cannot delete non-empty directory: moviedb1/download/Movie100/Sample
cannot delete non-empty directory: moviedb1/download/Movie100
cannot delete non-empty directory: moviedb1/download
cannot delete non-empty directory: moviedb1/Shows/Show A/Episode-1.h264.mkv
cannot delete non-empty directory: moviedb1/Shows/Show A/Episode-2.h264.mkv
cannot delete non-empty directory: moviedb1/Shows/Show A/Episode-3.h264.mkv
cannot delete non-empty directory: moviedb1/Shows/Show A/Episode-4.h264.mkv
cannot delete non-empty directory: moviedb1/Shows/Show A/Episode-5.h264.mkv
cannot delete non-empty directory: moviedb1/Shows/Show A/Episode-6.h264.mkv
cannot delete non-empty directory: moviedb1/Shows/Show A/Episode-7.h264.mkv
...

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]

Web server error log

The above is the docker output log

Nextcloud log (data/nextcloud.log)

Nextcloud log

This logfile was empty

Browser log

cannot access nextcloud because it's retarting due to the enforced error

The text was updated successfully, but these errors were encountered:

JuniorJPDJ · 2021-07-11T04:24:53Z

Where is your external storage mounted?

mirisbowring · 2021-07-11T07:30:13Z

There are two mounts. One ist exfat formatted and mounted as read-write with user/group www-data:root to /media/mount1/

The second is ext4 formatted and mounted to /media/mount2

JuniorJPDJ · 2021-07-11T11:29:36Z

I'm asking about paths inside docker. That's it?

mirisbowring · 2021-07-11T13:34:06Z

Ah sry, misunderstood that.

they are mounted to /var/www/html/disk1 oder disk2 inside the container

JuniorJPDJ · 2021-07-11T13:41:17Z

Then well... It's your mistake.
Maybe containers behavior is not documented good enough, but it removes everything in /var/www/html/ to upgrade installed nextcloud clearly.
There are 4 exceptions: config data custom_apps themes
Put your directories out of /var/www/html/ or inside one of mentioned directories and everything will be ok ;)

Aaaand.. Next time just don't trust maintainers, read scripts you are installing ;p
https://github.com/nextcloud/docker/blob/master/docker-entrypoint.sh#L104

mirisbowring · 2021-07-11T14:08:14Z

Yeah I saw the line, but I wondered if this is really the expected behavior. Thank you for clarification!

Will this be added to documentation?

JuniorJPDJ · 2021-07-12T04:51:35Z

It should be, but someone would need to do this
I'm not maintainer, I'm just an user ;D

mabeett · 2021-07-18T10:09:16Z

Hello,
Niether am I maintainer but I'd suggest you to propose changes in README.md under the tittle Persistent Data.
Maybe a more explicit text results helpful people as @mirisbowring in the past.
When creating the PR please mention this issue, so the mantainer could get the context of the intervention.

ChristianKniep · 2021-07-22T08:47:32Z

On that one, I am wondering why do you even need to rsync the installation onto the persistent storage?
Another way of doing that would be to just augment the persistent paths.
Like within the container:

move /usr/src/nextcloud/ to /var/www/html
have volumes mounted as /var/www/html/{config, data, custom_apps, themes} from underneath
if those volumes are empty, seed them with a default (like you do in the entrypoint)
ideally make the container file-system read-only and create volumes for everything that should be r/w.

With the work that @f7o does in aws-samples/aws-serverless-nextcloud volumes are used for some of them, expect themes that is. 🤔

As a docker nutjob, I am wondering whether the persistance of /var/www/html as a whole is necessary?

ChristianKniep · 2021-07-22T14:18:55Z

I created a PR to illustrate the point #1548

stsecurity · 2022-03-04T11:47:25Z

Just experienced the same thing and found this disscussion, I think at least some changes should be made to the README.md as it clearly states in the 'Update to a newer version' section:

Since all data is stored in volumes, nothing gets lost.

It is natural to assume all the data under /var/www/html would be preseved as we have mounted that directory as volume.

J0WI · 2023-04-04T18:05:41Z

Note added in #1862

9-FS · 2024-03-06T17:48:05Z

Then well... It's your mistake. Maybe containers behavior is not documented good enough, but it removes everything in /var/www/html/ to upgrade installed nextcloud clearly. There are 4 exceptions: config data custom_apps themes Put your directories out of /var/www/html/ or inside one of mentioned directories and everything will be ok ;)

Aaaand.. Next time just don't trust maintainers, read scripts you are installing ;p https://github.com/nextcloud/docker/blob/master/docker-entrypoint.sh#L104

I wish I would have known this earlier. I have just lost my whole media collection.

szaimen transferred this issue from nextcloud/server Jul 4, 2021

This was referenced Nov 14, 2022

Add note about not placing files inside /var/www/html #1168

Closed

Add words to README to draw attention to the use of upgrades.exclude, potential for files to be removed during installation/upgrades. #1862

Merged

J0WI closed this as completed Apr 4, 2023

joshtrichards mentioned this issue Oct 21, 2023

(Docker) deleting bind cleares external storage #1493

Closed

creekorful mentioned this issue Oct 2, 2024

Add PHP-FPM variant Dolibarr/dolibarr-docker#4

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nextcloud Upgrade purges mounted external local storage via rsync #1528

Nextcloud Upgrade purges mounted external local storage via rsync #1528

mirisbowring commented Jul 4, 2021

JuniorJPDJ commented Jul 11, 2021

mirisbowring commented Jul 11, 2021

JuniorJPDJ commented Jul 11, 2021

mirisbowring commented Jul 11, 2021

JuniorJPDJ commented Jul 11, 2021

mirisbowring commented Jul 11, 2021

JuniorJPDJ commented Jul 12, 2021

mabeett commented Jul 18, 2021 •

edited

Loading

ChristianKniep commented Jul 22, 2021 •

edited

Loading

ChristianKniep commented Jul 22, 2021

stsecurity commented Mar 4, 2022

J0WI commented Apr 4, 2023

9-FS commented Mar 6, 2024

Nextcloud Upgrade purges mounted external local storage via rsync #1528

Nextcloud Upgrade purges mounted external local storage via rsync #1528

Comments

mirisbowring commented Jul 4, 2021

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

Nextcloud log (data/nextcloud.log)

Browser log

JuniorJPDJ commented Jul 11, 2021

mirisbowring commented Jul 11, 2021

JuniorJPDJ commented Jul 11, 2021

mirisbowring commented Jul 11, 2021

JuniorJPDJ commented Jul 11, 2021

mirisbowring commented Jul 11, 2021

JuniorJPDJ commented Jul 12, 2021

mabeett commented Jul 18, 2021 • edited Loading

ChristianKniep commented Jul 22, 2021 • edited Loading

ChristianKniep commented Jul 22, 2021

stsecurity commented Mar 4, 2022

J0WI commented Apr 4, 2023

9-FS commented Mar 6, 2024

mabeett commented Jul 18, 2021 •

edited

Loading

ChristianKniep commented Jul 22, 2021 •

edited

Loading