[Bug]: cron / files_trashbin endless loop #55555
Description
⚠️ This issue respects the following points: ⚠️
- This is a bug, not a question or a configuration/webserver/proxy issue.
- This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
- Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
- I agree to follow Nextcloud's Code of Conduct.
Bug description
realising that the disk is filling up again, found again mariadb log is > 10GB (running for 3 days only). Investigating and when using 'tail -f ...' it endlessly filled the screen.
So I found the following SQL queries in an endless loop:
(Guessing the component as "cron files_trashbin", but hej no idea.
Config:
root@localhost [owncloud]> select * from oc_appconfig where appid = 'files_trashbin';
+----------------+------------------------------------+----------------+------+------+
| appid | configkey | configvalue | type | lazy |
+----------------+------------------------------------+----------------+------+------+
| files_trashbin | background_job_expire_trash_offset | 10 | 8 | 0 |
| files_trashbin | enabled | yes | 2 | 0 |
| files_trashbin | installed_version | 1.20.1 | 2 | 0 |
| files_trashbin | types | filesystem,dav | 2 | 0 |
+----------------+------------------------------------+----------------+------+------+
config.php:
...
'trashbin_retention_obligation' => 'auto, 14',
...
mariadb log:
59 Query UPDATE `oc_appconfig` SET `configvalue` = '0', `lazy` = 0, `type` = 8 WHERE (`appid` = 'files_trashbin') AND (`configkey` = 'background_job_expire_trash_offset')
59 Query SELECT `appid`, `configkey`, `configvalue`, `type` FROM `oc_appconfig` WHERE `lazy` = 0
59 Query UPDATE `oc_appconfig` SET `configvalue` = '10', `lazy` = 0, `type` = 8 WHERE (`appid` = 'files_trashbin') AND (`configkey` = 'background_job_expire_trash_offset')
59 Query SELECT `userid` FROM `oc_preferences` WHERE (`appid` = 'login') AND (`configkey` = 'lastLogin') AND (`configvalue` IS NOT NULL) LIMIT 10
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 1) AND (`path_hash` = 'd41d8cd98f00b204e9800998ecf8427e')
59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner` FROM `oc_share_external` WHERE (`user` = 'cUSER') AND (`accepted` = 1)
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'cUSER') AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '1') AND (`share_with` IN ('admin', 'c+mb')) AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT * FROM `oc_share` WHERE (`share_with` = 'cUSER') AND (`share_type` = '2') AND (`item_type` IN ('file', 'folder'))
59 Query SELECT `id`, `timestamp`, `location`, `deleted_by` FROM `oc_files_trash` WHERE `user` = 'cUSER'
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 1) AND (`path_hash` = '3014a771cbe30761f2e9ff3272110dbf')
59 Query SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `filecache`.`name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `filecache`.`permissions`, `checksum`, `unencrypted_size`, `metadata_etag`, `creation_time`, `upload_time`, `meta`.`json` AS `meta_json`, `meta`.`sync_token` AS `meta_sync_token` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` LEFT JOIN `oc_files_metadata` `meta` ON `filecache`.`fileid` = `meta`.`file_id` WHERE (`filecache`.`parent` = 94428) AND (`storage` = 1) ORDER BY `name` ASC
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 21) AND (`path_hash` = 'd41d8cd98f00b204e9800998ecf8427e')
59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner` FROM `oc_share_external` WHERE (`user` = 'cUSERtest') AND (`accepted` = 1)
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'cUSERtest') AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT * FROM `oc_share` WHERE (`share_with` = 'cUSERtest') AND (`share_type` = '2') AND (`item_type` IN ('file', 'folder'))
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 23) AND (`path_hash` = 'd41d8cd98f00b204e9800998ecf8427e')
59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner` FROM `oc_share_external` WHERE (`user` = 'kUSER') AND (`accepted` = 1)
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'kUSER') AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT * FROM `oc_share` WHERE (`share_with` = 'kUSER') AND (`share_type` = '2') AND (`item_type` IN ('file', 'folder'))
59 Query SELECT `id`, `timestamp`, `location`, `deleted_by` FROM `oc_files_trash` WHERE `user` = 'kUSER'
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 23) AND (`path_hash` = '3014a771cbe30761f2e9ff3272110dbf')
59 Query SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `filecache`.`name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `filecache`.`permissions`, `checksum`, `unencrypted_size`, `metadata_etag`, `creation_time`, `upload_time`, `meta`.`json` AS `meta_json`, `meta`.`sync_token` AS `meta_sync_token` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` LEFT JOIN `oc_files_metadata` `meta` ON `filecache`.`fileid` = `meta`.`file_id` WHERE (`filecache`.`parent` = 99850) AND (`storage` = 23) ORDER BY `name` ASC
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3) AND (`path_hash` = 'd41d8cd98f00b204e9800998ecf8427e')
59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner` FROM `oc_share_external` WHERE (`user` = 'mUSER') AND (`accepted` = 1)
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'mUSER') AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '1') AND (`share_with` IN ('c+mb')) AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT * FROM `oc_share` WHERE (`share_with` = 'mUSER') AND (`share_type` = '2') AND (`item_type` IN ('file', 'folder'))
59 Query SELECT `id`, `timestamp`, `location`, `deleted_by` FROM `oc_files_trash` WHERE `user` = 'mUSER'
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 3) AND (`path_hash` = '3014a771cbe30761f2e9ff3272110dbf')
59 Query SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `filecache`.`name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `filecache`.`permissions`, `checksum`, `unencrypted_size`, `metadata_etag`, `creation_time`, `upload_time`, `meta`.`json` AS `meta_json`, `meta`.`sync_token` AS `meta_sync_token` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` LEFT JOIN `oc_files_metadata` `meta` ON `filecache`.`fileid` = `meta`.`file_id` WHERE (`filecache`.`parent` = 95592) AND (`storage` = 3) ORDER BY `name` ASC
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 4) AND (`path_hash` = 'd41d8cd98f00b204e9800998ecf8427e')
59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner` FROM `oc_share_external` WHERE (`user` = 'oUSER') AND (`accepted` = 1)
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'oUSER') AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT * FROM `oc_share` WHERE (`share_with` = 'oUSER') AND (`share_type` = '2') AND (`item_type` IN ('file', 'folder'))
59 Query SELECT `id`, `timestamp`, `location`, `deleted_by` FROM `oc_files_trash` WHERE `user` = 'oUSER'
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 4) AND (`path_hash` = '3014a771cbe30761f2e9ff3272110dbf')
59 Query SELECT `filecache`.`fileid`, `storage`, `path`, `path_hash`, `filecache`.`parent`, `filecache`.`name`, `mimetype`, `mimepart`, `size`, `mtime`, `storage_mtime`, `encrypted`, `etag`, `filecache`.`permissions`, `checksum`, `unencrypted_size`, `metadata_etag`, `creation_time`, `upload_time`, `meta`.`json` AS `meta_json`, `meta`.`sync_token` AS `meta_sync_token` FROM `oc_filecache` `filecache` LEFT JOIN `oc_filecache_extended` `fe` ON `filecache`.`fileid` = `fe`.`fileid` LEFT JOIN `oc_files_metadata` `meta` ON `filecache`.`fileid` = `meta`.`file_id` WHERE (`filecache`.`parent` = 89614) AND (`storage` = 4) ORDER BY `name` ASC
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 22) AND (`path_hash` = 'd41d8cd98f00b204e9800998ecf8427e')
59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner` FROM `oc_share_external` WHERE (`user` = 'sUSER') AND (`accepted` = 1)
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'sUSER') AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT * FROM `oc_share` WHERE (`share_with` = 'sUSER') AND (`share_type` = '2') AND (`item_type` IN ('file', 'folder'))
59 Query SELECT `fileid` FROM `oc_filecache` WHERE (`storage` = 17) AND (`path_hash` = 'd41d8cd98f00b204e9800998ecf8427e')
59 Query SELECT `remote`, `share_token`, `password`, `mountpoint`, `owner` FROM `oc_share_external` WHERE (`user` = 'tUSER') AND (`accepted` = 1)
59 Query SELECT `s`.*, `f`.`fileid`, `f`.`path`, `f`.`permissions` as `f_permissions`, `f`.`storage`, `f`.`path_hash`, `f`.`parent` as `f_parent`, `f`.`name`, `f`.`mimetype`, `f`.`mimepart`, `f`.`size`, `f`.`mtime`, `f`.`storage_mtime`, `f`.`encrypted`, `f`.`unencrypted_size`, `f`.`etag`, `f`.`checksum`, `st`.`id` AS `storage_string_id` FROM `oc_share` `s` LEFT JOIN `oc_filecache` `f` ON `s`.`file_source` = `f`.`fileid` LEFT JOIN `oc_storages` `st` ON `f`.`storage` = `st`.`numeric_id` WHERE (`share_type` = '0') AND (`share_with` = 'tUSER') AND (`item_type` IN ('file', 'folder')) ORDER BY `s`.`id` ASC
59 Query SELECT * FROM `oc_share` WHERE (`share_with` = 'tUSER') AND (`share_type` = '2') AND (`item_type` IN ('file', 'folder'))
This part is endlessly repeated.
If anyone knows what part is responsible, thanks.
It mainly updates the configkey background_job_expire_trash_offset from 0 to 10 and back to 0 and back to 10 ....
Cheers
Christian
Steps to reproduce
1.dont know
2.
3.
Expected behavior
not executing 10 Mio times the same query a day in an endless loop
Nextcloud Server version
30
Operating system
Other
PHP engine version
PHP 8.3
Web server
Apache (supported)
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Upgraded to a MAJOR version (ex. 31 to 32)
Are you using the Nextcloud Server Encryption module?
None
What user-backends are you using?
- Default user-backend (database)
- LDAP/ Active Directory
- SSO - SAML
- Other
Configuration report
[root@beastly /usr/local/www/htdocs/nc]# php ./occ config:list system
{
"system": {
"instanceid": "***REMOVED SENSITIVE VALUE***",
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"cXX.ch"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"overwrite.cli.url": "https:\/\/cmb.ch\/nc",
"dbtype": "mysql",
"version": "30.0.16.1",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"default_phone_region": "CH",
"mail_smtpmode": "sendmail",
"forcessl": true,
"theme": "",
"maintenance": false,
"logfile": "\/var\/log\/nextcloud\/nextcloud.log",
"logfile_audit": "\/var\/log\/nextcloud\/nextcloud.audit.log",
"loglevel": 1,
"trashbin_retention_obligation": "auto, 14",
"log.condition": {
"users": [
"cUSER"
],
"apps": [
"files"
]
},
"updater.release.channel": "stable",
"filelocking.enabled": true,
"localstorage.allowsymlinks": true,
"enable_previews": true,
"preview_max_x": 384,
"preview_max_y": 512,
"memcache.locking": "\\OC\\Memcache\\Redis",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"timeout": 0,
"password": "***REMOVED SENSITIVE VALUE***"
},
"mysql.utf8mb4": true,
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_sendmailmode": "smtp",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"maintenance_window_start": 1
}
}List of activated Apps
[root@beastly /usr/local/www/htdocs/nc]# php ./occ app:list
Enabled:
- activity: 3.0.0
- admin_audit: 1.20.0
- app_api: 4.0.6
- bookmarks: 15.2.0
- calendar: 5.5.5
- cloud_federation_api: 1.13.0
- comments: 1.20.1
- contacts: 7.3.2
- dashboard: 7.10.0
- dav: 1.31.1
- federatedfilesharing: 1.20.0
- federation: 1.20.0
- files: 2.2.0
- files_downloadlimit: 3.0.0
- files_pdfviewer: 3.0.0
- files_reminders: 1.3.0
- files_sharing: 1.22.0
- files_trashbin: 1.20.1
- firstrunwizard: 3.0.0
- logreader: 3.0.0
- lookup_server_connector: 1.18.0
- nextcloud_announcements: 2.0.0
- notifications: 3.0.0
- oauth2: 1.18.1
- password_policy: 2.0.0
- photos: 3.0.2
- polls: 8.3.10
- privacy: 2.0.0
- provisioning_api: 1.20.0
- recommendations: 3.0.0
- related_resources: 1.5.0
- serverinfo: 2.0.0
- settings: 1.13.0
- sharebymail: 1.20.0
- support: 2.0.0
- systemtags: 1.20.0
- text: 4.1.0
- theming: 2.6.0
- twofactor_backupcodes: 1.19.0
- updatenotification: 1.20.0
- user_pwauth: 2.8.3
- user_status: 1.10.0
- viewer: 3.0.0
- webhook_listeners: 1.1.0-dev
- workflowengine: 2.12.0
Disabled:
- bruteforcesettings: 3.0.0 (installed 2.4.0)
- circles: 30.0.0 (installed 0.14.2)
- contactsinteraction: 1.11.0 (installed 1.9.0)
- encryption: 2.18.0
- files_external: 1.22.0
- files_versions: 1.23.0 (installed 1.23.0)
- survey_client: 2.0.0 (installed 1.8.0)
- suspicious_login: 8.0.0
- twofactor_nextcloud_notification: 4.0.0
- twofactor_totp: 12.0.0-dev
- user_ldap: 1.21.0
- weather_status: 1.10.0 (installed 1.0.0)
I had files_version installed when the problem occurred. But haven't found out (with documentation) where it is used. Never seen the version button in the gui on any file, so it is not needed by me therefore disabled it)Nextcloud Signing status
No errors have been found.Nextcloud Logs
Aehm, it logs 2 GB a day.. overwrites its own log (where is only one .1 file...) but only shows like 6 harmless items in the gui (after a one minute wait).
I have no clue how anyone could ever find something in there.
Eg:
[root@beastly /usr/local/www/htdocs/nc]# grep files_trashbin /var/log/nextcloud/nextcloud.log.1 | wc
12 3362932 91749250
[root@beastly /usr/local/www/htdocs/nc]# grep files_trashbin /var/log/nextcloud/nextcloud.log | wc
6 1373378 37189240
it is unreadable... sorry.. and then jsonAdditional info
Not sure if related to this issue, but the last 2 weeks (version 30) and macos client 3.17.x it has issues updating/uploading just one file. Once it uploaded it 100 times partly. Then it never showed the tick in the menu line. Same thing happening from another mac (M3 macbookair). Then you stop and restart the client, then it solves it. But next day same again....