-
-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allowed memory size of 536870912 bytes exhausted on activity stream #31528
Comments
Activities in the range: "app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_deleted",
"app": "files",
"type": "file_created",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_created",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_created",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_created",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed",
"app": "files",
"type": "file_changed", |
sadly no full stack trace, so harder to pinpoint the exact code location |
I debugged the used memory: \OC::$server->get(\Psr\Log\LoggerInterface::class)->error('Description#' . memory_get_usage(), ['app' => 'nickv']); First file in the activity loop
https://github.com/nextcloud/activity/blob/8481eb8aa8d688fd8127c7348f4dc6019cd8be5f/lib/Controller/APIv2Controller.php#L375 The a second file without preview
|
@icewind1991 do you remember what else we changed that could be related ? |
No idea what could cause this and can't reproduce |
are these entries connected to shares ? if yes there's a slight chance they could be connected to the wrong share owner values case ? ref #30791 and #28503 (comment) |
BaselinePoint of measurement: https://github.com/nextcloud/activity/blob/8481eb8aa8d688fd8127c7348f4dc6019cd8be5f/lib/ViewInfoCache.php#L114 Disabling Circles, Deck and Talk
|
$shares = array_merge($shares, $this->shareManager->getSharedWith($user->getUID(), IShare::TYPE_CIRCLE, null, -1)); | |
$shares = array_merge($shares, $this->shareManager->getSharedWith($user->getUID(), IShare::TYPE_ROOM, null, -1)); | |
$shares = array_merge($shares, $this->shareManager->getSharedWith($user->getUID(), IShare::TYPE_DECK, null, -1)); |
Before: 9.685.336 ~9.6 MB
After: 99.907.184 ~100 MB
Disabling Talk only if $userId === me
$shares = array_merge($shares, $this->shareManager->getSharedWith($user->getUID(), IShare::TYPE_ROOM, null, -1)); |
Before: 9.685.336 ~9.6 MB
After: 100.058.336 ~100 MB
DB ref
SELECT share_type, COUNT(id) FROM oc_share WHERE share_with = 'me =)' GROUP BY share_type;
+------------+-----------+
| share_type | COUNT(id) |
+------------+-----------+
| 0 | 23 |
| 2 | 27 |
| 11 | 1360 |
| 13 | 15 |
+------------+-----------+
Talk RoomShareProvider if $userId === me
Before: 10.815.296 ~10.8 MB
After: 18.674.088 ~18.6 MB
MountProvider total if $userId === me
server/apps/files_sharing/lib/MountProvider.php
Lines 95 to 151 in 8597d9c
// filter out excluded shares and group shares that includes self | |
$shares = array_filter($shares, function (\OCP\Share\IShare $share) use ($user) { | |
return $share->getPermissions() > 0 && $share->getShareOwner() !== $user->getUID(); | |
}); | |
$superShares = $this->buildSuperShares($shares, $user); | |
$mounts = []; | |
$view = new View('/' . $user->getUID() . '/files'); | |
$ownerViews = []; | |
$sharingDisabledForUser = $this->shareManager->sharingDisabledForUser($user->getUID()); | |
$foldersExistCache = new CappedMemoryCache(); | |
foreach ($superShares as $share) { | |
try { | |
/** @var \OCP\Share\IShare $parentShare */ | |
$parentShare = $share[0]; | |
if ($parentShare->getStatus() !== IShare::STATUS_ACCEPTED && | |
($parentShare->getShareType() === IShare::TYPE_GROUP || | |
$parentShare->getShareType() === IShare::TYPE_USERGROUP || | |
$parentShare->getShareType() === IShare::TYPE_USER)) { | |
continue; | |
} | |
$owner = $parentShare->getShareOwner(); | |
if (!isset($ownerViews[$owner])) { | |
$ownerViews[$owner] = new View('/' . $parentShare->getShareOwner() . '/files'); | |
} | |
$mount = new SharedMount( | |
'\OCA\Files_Sharing\SharedStorage', | |
$mounts, | |
[ | |
'user' => $user->getUID(), | |
// parent share | |
'superShare' => $parentShare, | |
// children/component of the superShare | |
'groupedShares' => $share[1], | |
'ownerView' => $ownerViews[$owner], | |
'sharingDisabledForUser' => $sharingDisabledForUser | |
], | |
$loader, | |
$view, | |
$foldersExistCache | |
); | |
$event = new ShareMountedEvent($mount); | |
$this->eventDispatcher->dispatchTyped($event); | |
$mounts[$mount->getMountPoint()] = $mount; | |
foreach ($event->getAdditionalMounts() as $additionalMount) { | |
$mounts[$additionalMount->getMountPoint()] = $additionalMount; | |
} | |
} catch (\Exception $e) { | |
$this->logger->logException($e); | |
$this->logger->error('Error while trying to create shared mount'); | |
} | |
} |
Before: 18.532.088 ~18.5 MB
After: 20.741.552 ~20.7 MB
Filesystem::getMountManager
server/lib/private/Files/View.php
Lines 1739 to 1741 in d69531a
$manager = Filesystem::getMountManager(); | |
$mounts = $manager->findIn($this->fakeRoot); | |
$mounts[] = $manager->find($this->fakeRoot); |
Actually this seems to loop and setup all users again:
For the first call to: $path = $this->view->getPath($fileId);
https://github.com/nextcloud/activity/blob/fee285e6496e7dde396cab573ab4113920601d87/lib/ViewInfoCache.php#L114
We end up logging 428 times inside the View::getPath
I basically have all the company employees and former employees logged with:
\OC::$server->get(\Psr\Log\LoggerInterface::class)->error('getPath1#' . json_encode([$this->fakeRoot]) . memory_get_usage(), ['app' => 'nickv']);
{"reqId":"J5eVMi3H4zguIogiQQZk","level":3,"time":"2022-03-14T21:18:04+00:00","remoteAddr":"…","user":"joas","app":"nickv","method":"GET","url":"/ocs/v2.php/apps/activity/api/v2/activity/all?format=json&previews=true&since=0","message":"getPath1#[\"\\/joas\\/files\"]9371808","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0","version":"23.0.3.0"}
{"reqId":"J5eVMi3H4zguIogiQQZk","level":3,"time":"2022-03-14T21:18:04+00:00","remoteAddr":"…","user":"joas","app":"nickv","method":"GET","url":"/ocs/v2.php/apps/activity/api/v2/activity/all?format=json&previews=true&since=0","message":"getPath1#[\"\\/lukas\\/files\"]27903304","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0","version":"23.0.3.0"}
{"reqId":"J5eVMi3H4zguIogiQQZk","level":3,"time":"2022-03-14T21:18:04+00:00","remoteAddr":"…","user":"joas","app":"nickv","method":"GET","url":"/ocs/v2.php/apps/activity/api/v2/activity/all?format=json&previews=true&since=0","message":"getPath1#[\"\\/anna\\/files\"]40533712","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0","version":"23.0.3.0"}
{"reqId":"J5eVMi3H4zguIogiQQZk","level":3,"time":"2022-03-14T21:18:05+00:00","remoteAddr":"…","user":"joas","app":"nickv","method":"GET","url":"/ocs/v2.php/apps/activity/api/v2/activity/all?format=json&previews=true&since=0","message":"getPath1#[\"\\/jos\\/files\"]49765304","userAgent":"Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:98.0) Gecko/20100101 Firefox/98.0","version":"23.0.3.0"}
…
Repeat 424 times with different and repeating users
each user adding their ~10 MB. When you mounted ~50 users the 512 MB of memory limit are up.
Tried reverting those two:
but the problem persisted. So might be coincidentally just occur now since someone shared something with me causing more users to be mounted for me? |
I have this issue, just got it with the latest snap version for 24. Whenever I try to upload a file more than 500Mb nextcloud just fails and produces that exact error in the log. This issue filled my trashbin so my server ran out of space. What can I do/provide to be of more help? Version: 8.0.23 Memory limit: 512 MB Max execution time: 3600 Upload max size: 16 GB Extensions: Core, date, libxml, openssl, pcre, zlib, bcmath, bz2, ctype, curl, dom, hash, fileinfo, filter, ftp, gd, gmp, SPL, iconv, intl, json, ldap, mbstring, pcntl, PDO, session, posix, Reflection, standard, SimpleXML, mysqlnd, exif, tokenizer, xml, xmlreader, xmlwriter, zip, pdo_mysql, cgi-fcgi, redis, Zend OPcache |
Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you! |
Seems to be not happening directly on activity anymore. |
closing because we already have another open issue about the general issue |
this can still / again happen on Nextcloud 28.0.2.
full log entry:
only one line off this time, so I'd guess it's the same issue. |
Bug description
Something in the recent version changed something which now causes memory issues on the activity stream.
Steps to reproduce
Try it another time same error in a different place
Expected behavior
No error and the page shows quickly
Installation method
Manual installation
Operating system
Debian/Ubuntu
PHP engine version
PHP 7.4
Web server
Apache (supported)
Database engine version
MariaDB
Is this bug present after an update or on a fresh install?
Updated from a minor version (ex. 22.2.3 to 22.2.4)
Are you using the Nextcloud Server Encryption module?
Encryption is Disabled
What user-backends are you using?
Configuration report
List of activated Apps
Nextcloud Signing status
No response
Nextcloud Logs
Additional info
Error started occuring after update from 23.0.2 to 23.0.3rc1
The text was updated successfully, but these errors were encountered: