[Bug]: occ files:scan uses huge memory and throws php error

[roadrunnerjb]

⚠️ This issue respects the following points: ⚠️

• This is a bug, not a question or a configuration/webserver/proxy issue.
• This issue is not already reported on Github (I've searched it).
• Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
• Nextcloud Server is running on 64bit capable CPU, PHP and OS.
• I agree to follow Nextcloud's Code of Conduct.

Bug description

After upgrading NC from 24.0.7 to 25.0.1 we had several files related issues (some shares are missing) so I wanted to do a occ files:scan. First I tried with --all but it never comes to an end (even not after 24 hours). It stucks at one particular user so I tried with this user and gave plenty of php memory. That gives me the following result:

sudo -u www-data php8.0  -d memory_limit=16G /var/www/nextcloud/occ   files:scan A********k
Starting scan for user 1 out of 1 (A*********k)
PHP Fatal error:  Allowed memory size of 17179869184 bytes exhausted (tried to allocate 262144 bytes) in /var/www/nextcloud/apps/files_sharing/lib/SharedStorage.php on line 534
PHP Fatal error:  Allowed memory size of 17179869184 bytes exhausted (tried to allocate 262144 bytes) in /var/www/nextcloud/lib/private/Log.php on line 135

For me it seems there is a endless loop consuming memory over and over.
The user itself has no personal files in the filesystem:

root@s******01:/var/nextcloud-data/A*******k/files# find . -type f|wc -l
0

How can I dig deeper into this issue, is the an sql that I can use ?
Any Ideas are welcome

Steps to reproduce

Expected behavior

occ files:scan should not terminate with php error

Installation method

Community Web installer on a VPS or web space

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.0

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Updated to a major version (ex. 22.2.3 to 23.0.1)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "default_language": "de",
        "default_phone_region": "DE",
        "trusted_domains": [
            "svllcloud01",
            "svllcloud01.adfc-bw.local",
            "vcloud.adfc-bw.de",
            "192.168.123.29"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "activity_expire_days": 180,
        "overwrite.cli.url": "https:\/\/vcloud.adfc-bw.de",
        "overwritehost": "vcloud.adfc-bw.de",
        "overwriteprotocol": "https",
        "dbtype": "mysql",
        "version": "25.0.1.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "skeletondirectory": "\/var\/nextcloud-data\/adfc-skeleton",
        "share_folder": "\/Mit mir geteilt",
        "theme": "",
        "default_locale": "de_DE",
        "auth.bruteforce.protection.enabled": false,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls",
        "mysql.utf8mb4": true,
        "app_install_overwrite": [
            "mindmaps",
            "emlviewer",
            "w2g2",
            "whiteboard",
            "integration_whiteboard",
            "fulltextsearch",
            "fulltextsearch_elasticsearch",
            "files_fulltextsearch",
            "impersonate"
        ],
        "updater.release.channel": "stable",
        "loglevel": 0,
        "sort_groups_by_name": true
    }
}

List of activated Apps

Enabled:
  - activity: 2.17.0
  - admin_audit: 1.15.0
  - announcementcenter: 6.4.0
  - bookmarks: 11.0.4
  - calendar: 4.1.0
  - circles: 25.0.0
  - cloud_federation_api: 1.8.0
  - collectives: 2.0.0
  - comments: 1.15.0
  - contacts: 5.0.1
  - contactsinteraction: 1.6.0
  - dashboard: 7.5.0
  - dav: 1.24.0
  - deck: 1.8.1
  - emlviewer: 1.0.4
  - external: 5.0.0
  - federatedfilesharing: 1.15.0
  - federation: 1.15.0
  - files: 1.20.1
  - files_external: 1.17.0
  - files_fulltextsearch: 24.0.1
  - files_pdfviewer: 2.6.0
  - files_rightclick: 1.4.0
  - files_sharing: 1.17.0
  - files_trashbin: 1.15.0
  - files_versions: 1.18.0
  - firstrunwizard: 2.14.0
  - forms: 3.0.1
  - fulltextsearch: 24.0.0
  - fulltextsearch_elasticsearch: 24.0.1
  - group_everyone: 0.1.11
  - groupfolders: 13.0.0
  - impersonate: 1.11.0
  - logreader: 2.10.0
  - lookup_server_connector: 1.13.0
  - mail: 2.1.1
  - maps: 0.2.1
  - nextcloud_announcements: 1.14.0
  - notifications: 2.13.1
  - notify_push: 0.5.0
  - oauth2: 1.13.0
  - onlyoffice: 7.6.8
  - password_policy: 1.15.0
  - photos: 2.0.0
  - polls: 4.0.0
  - privacy: 1.9.0
  - provisioning_api: 1.15.0
  - recommendations: 1.4.0
  - related_resources: 1.0.3
  - serverinfo: 1.15.0
  - settings: 1.7.0
  - sharebymail: 1.15.0
  - spreed: 15.0.1
  - support: 1.8.0
  - survey_client: 1.13.0
  - systemtags: 1.15.0
  - tasks: 0.14.5
  - text: 3.6.0
  - theming: 2.0.1
  - twofactor_backupcodes: 1.14.0
  - updatenotification: 1.15.0
  - user_status: 1.5.0
  - viewer: 1.9.0
  - w2g2: 3.3.9
  - weather_status: 1.5.0
  - whiteboard: 0.0.3
  - workflowengine: 2.7.0
Disabled:
  - bruteforcesettings
  - caniupdate: 0.2.0
  - encryption
  - files_automatedtagging: 1.15.0
  - files_markdown: 2.3.6
  - files_readmemd: 1.2.2
  - gpxpod: 5.0.2
  - mindmaps: 0.1.0
  - mood: 0.5.0
  - notes: 4.6.0
  - socialsharing_email: 2.5.0
  - socialsharing_facebook: 2.5.0
  - socialsharing_twitter: 2.5.0
  - suspicious_login
  - talk_simple_poll: 1.3.1
  - twofactor_totp
  - user_ldap

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

No response

[PVince81]

you likely have an infinite loop of sorts for that user, usually caused by misplaced mount points

please check the following for this user:

1. check if the user can even log in and list their files
2. select * from oc_share where share_with="$userid" or share_with in ("group1", "group2", ...) and share_type in (0,1,2)
   • replace "$userid" with the user's id
   • replace the groups with the groups where the user is a member of
   • pay attention to the file_target value which shows where it's mounted for that user
3. run "occ groupfolders:list" and check if any of the file_target is sitting on top of a group folder mount point
4. in 3) also check if group folders overlap each other somehow
5. pay attention to see if any of the file_target of 2) look similar to a group folder's name because it could mean that someone shared it with another person, and the other person might have shared it back, hence causing a loop of sorts

[roadrunnerjb]

thank you, her in short the answers:

1. I can impersionate the user and browse all files. We have set 'share_folder' => '/Mit mir geteilt', in NC config so all shares are in this subfolder.
2. I can see that two subfolders of a groupfolder the user has access to are also shared with the user but in my point of view this deos not lead to an endless loop ?
3. groupfoldernames are totally different than other folders in the users context
4. what do you mean with overlapping we have one single filesystem an each of 108 groupfolders is a subfolder in __groupfolders on filesystem level and the groupfoldernames are unique ?
5. file_target starts with /Mit mir geteilt/ and we have no groupfolderr with this name.

I tried other users whos are in the same group of the first user and with each of them I have the same issue (looping until php memory is exhausted)

Is there a kind of trace mode for occ files:scan to see where the loop is ?
Or can we have a remote sesseion to take a look on this issue ?
So do you think this is not related to the upgrade we did last week ?

Regards
Joachim

[PVince81]

@roadrunnerjb thanks for checking.

With overlapping mount point it would be like having these two mount points:

• "/groupfolder1"
• "/groupfolder1/receivedshare": here received share mounted on top of the group folder

you have already confirmed that there were no such scenarios, so am unsure where the loop is

sadly there is no easy way to get a trace from the current code and require adding a lot of extra noisy logging statements

before we debug further, please keep an eye on #34752 and the WIP PR #35218 because that one will fix some behavior with mount points.

[PVince81]

@roadrunnerjb you can apply https://github.com/nextcloud/server/pull/35255.patch now and let me know if the scan still crashes

[nextcloud-command]

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.