Wrong or no sizes of files/folders #5031
Comments
This comment was marked as spam.
This comment was marked as spam.
|
Confirmed. I also have this issue and on NC11 everything was fine. I'm running NC12 on a raspberrypi with PostgreSQL. This kind makes me remember about the PHP problem in retrieving file sizes greater than 2GB when running on 32bit architectures. |
|
Confirmed! Same problem here! |
|
Same issue here, Linux client and Rasberry Pi Server. |
|
Same issue here. A fresh install on Raspberry Pi 3. |
|
Same here. Raspberry Pi 3, PHP 7, MariaDB, Apache2. |
|
The same happens to me when a folder contains more than 2GB of files. Summed size of content: 2,835,430,391 bytes Example2: |
|
same here. |
|
Same here on a Raspberry Pi 3 running nextcloud 12.0.0 |
|
can confirm on Cubieboard 4 (CC-80) maybe this only affects armv7l? judging from the comments here everyone who reported their architecture was on arm |
|
Have same issue on 32-bit operating system. Is everyone having this issue using 32-bit hardware? Or is this issue occurring with both 64-bit and 32-bit operating systems? Architecture is not arm. |
|
Ah! Interesting to note that. Seems like all the people here reporting are the ones noticing this on 32bit OS. |
|
Yep.. I'm having the same issue running LAMP in a Rpi 2... No solution yet? Even in the nextcloud client shows 0 B of 43 GB in use... I have 50 GB so that means that it's assuming in a wrong way that I have consumed 7 GB. |
|
I'm having the same issue, system: Raspberry Pi 3, PHP 7, MariaDB, Apache2 and Debian 9 |
|
The problem, as far as I can see, is due to the 32-bit version of PHP. Apparently, until NC11, developers found a workaround for this issue. |
|
The console (occ) is still working.... so the problem is not in PHP - it is from the source of NC12. Also NC11 is working..... |
|
Is there a way to raise this problem to the NC developers? |
|
I will try in #nextcloud-dev |
|
I would assume this to be a problem with However, some of you are suggesting this error is new in NC12, and I could not find any relevant / detrimental commit. (Last patch to these lines was by @DeepDiver1975 in 2015...) |
|
on NC11 everything worked fine and I only updated to NC12 and did not change or update anything else (RPi3, PHP, MySQL, etc... is still the same) Maybe it is not the file summary, but the calculation of the single files (don't know where to find that code snippet) |
|
I have the same problem. Folder size shown as pending for a folder larger then 7GB in the root directory. Opening the folder however it shows the exact size of 7.1GB at the bottom (all files and subfolder are smaller then 2GB). This all started right after the upgrade from NextCloud 11.03 to 12.0. Also on the personal page the used space is shown as 0B while the max available space is shown as the MaxQuota-UsedSpace. Nextcloud 12.0 on raspbian (raspberry pi 3) updated from version 11.0.3 |
|
Just saw that sabre-http had a similiar problem that was solved by this workaround. Nexcloud - DAV also was affected by this sabe issue, which was fixed by @nickvergessen for NC13. |
|
@akki42 correct. But still, even with these two fixes, the file size is still capped at 2GB. I think you found the correct location, max integer at 32 Bit is 2147483647, so this may be the reason (see my screenshot). Note that javascript and thus parseInt can handle more then 32 Bit. Also with the latest patches, the file is downloaded correctly, so i think it is a matter of visualization. I can confirm this issue on an odroid SBC (32 Bit machine). @nickvergessen , @MorrisJobke : here is another small bug, which should be solved with 12.0.1 or 13: File sizes shown in the GUI are not above 2 GB on 32 Bit machines.
|
|
Update: I think the function "humanFileSize" is the problem. It takes an integer as argument, which is obviously capped at 32 Bits here. The function is used also at other places. I will investigate further .. |
|
Nothing new here??? |
|
No :( |
|
Experiencing the same issues. In the directory list, large dirs have a "pending" size, while the correct size is shown at the bottom when going into that directory. I've got no problems with a max filesize of 2GB (as is expectable on 32 bit systems), but the problem is that large directories are bringing the system down: I've set a quotum of 100 GB but NC says I've only got a quotum of 28 GB after transferring my files. I've already updated sabre-http manually but that obviously doesn't solve the problem. |
|
I did not find the root cause, i still think it has to do with humanFileSize, but as i am not used to templates nor to php that much, i cannot help further here. |
I also use Odroid HC2 with Openmediavault and use Nextcloud as a docker. It was working since like version 10, with no issues (with files as big as 70GB). Like you, I use it only for syncing with different computers and family members. Extremely frustrating. I guess I can find and do a fresh install of an older version like 18, but what is the point if this thing persists in the new versions and I am stuck with getting no updates and/or fixes... I just gave up at this point, unplugged the Odroid and stopped using the system... I am manually copying files between my computers, bad bad experience. I hope developers figure the problem out and fix it with an explanation as what was the issue for this big bug. |
|
The issue occures with files of more than 2147483647 Bytes in Nextcloud 19.0.2 on Raspberry PI OS 10 32 bit. Version 17.0.9 and 18.0.8. are not affected. So in my opinion the reason is not the php version. |
|
How could we get attention on an issue this old? Should we try and mention someone maybe? I believe many, many people are affected by it. |
|
Is this really install breaking? I mean accessing files still works, its just annoying to have to run an occ command to update large folders. |
Updating large folders (or folders with large files) with an occ command does not work for me, unfortunately. In my case Nextcloud isn't able to sync large files. When I try to sync a large file Nextcloud creates a copy of the file (on the client side). The file with the "conflicted copy" in it's name has the right size, but the "original" files has now a smaller size. So sync doesn't work, and this is install breaking... BTW: I also have this issue here (nextcloud/desktop#2279). Maybe this is somehow correlated? |
|
@dasaweb Ah I see, I don't have that issue, I can still sync files even if nextcloud cant display the size to me. |
|
running the command does not fix the issue anymore. In my case i'm still able to upload and download larger files, the size shown is just wrong, and therefore the quota and used space calculations are also off by quite a bit if you have a couple of larger files. For systems which rely on correctly reported Quota i would describe this as install breaking. Single User systems which do not rely on quota are probably still fine though |
|
At least in my case, no matter how many occ commands I run on my NC 19 installation, the size stays as Pending. On NC 18 that was not the case. Be it --unscanned or --all, it stays the same. Just tried it on a small folder to make sure I wasn't mixing things up. My files are on external storage, and since I upgraded to NC19 from NC18 I also found that occ doesn't trigger a scan when opening the external storage, which may or may not be related (i.e. it might trigger it but it may fail silently as I see no log of it). I can access and manage files as usual, though. |
|
The size access is completely broken on my side, the file on the storage (nextcloud) side is 2 gigs, while the real files should be way more than that. The file is then completely corrupted and it makes no sense to access it. |
|
Same issue here on Pi 4 B (32-bit Raspberry Pi OS). The instance worked like a charm before upgrading to Nextcloud 19. I will propably backup all data and downgrade to 18. Hopefully a fix is here soon. Maybe we should open another issue since the current problems are connected to Nextcloud 19 only? |
|
I installed 18 from scratch and moved all the files. Everything works as before. Seems like I will not be upgrading to 19 unless there is a fix for this. |
|
@MegaMarkey I'm up for a new issue being opened, and happy to chime in there. For what it's worth, I just downgraded to a previous NC18.0.8.2 backup I had, and after a few adjustments everything seems to be up and running as intended. I'm not biting the NC19 (or any major upgrade version) bullet anytime soon, certainly not until I am confident that this is solved. |
|
Downgrading to NC18 also resolves this problem on XU4 32-bit system. |
I believe this is the right way to do it. Since it seems like this is a new issue that started with the new version 19. Would you be able to take the lead and open a new issue? |
|
Hello, i developed a patch to fix the issue and opened a PR here: #23257 |
|
Wonderful, |
|
If the PR will get merged, i'll try to have it backported to stable19. |
|
thanks for the fix, i just made the same changes to my 19.0.2 installation and after a |
|
Sounds promising! Unfortunately I have to wait until the fix reaches Nextcloud Snap. But at least there is hope :D |
|
I successfully applied the patch on Nextcloud 20.0.0, everything works fine now. Thanks a lot, I hope it gets merged! |
|
Hello, |
|
You need to edit the file "lib/private/Files/Storage/Local.php" as seen in the pull request (removing three lines and adding two). Of course maintenance mode should be enabled before. After you are done, disable maintenance mode and rescan all files via |
Thank you a lot! Worked like a charm! |
|
I still have the problem with Nextcloud Webdav |
and others
After upgrading from NC11 to NC12, I noticed incorrect folder and file sizes. All >2GB files are pending. I checked the oc_filechache table - it looks like the sizes are fine. I trunc the oc_filechache table and executed the files: scan --all again and again filled it with the exact size, but the Files app is rendered incorrectly. There are no errors in the NC logos, there is no error in appache logos too.
Steps to reproduce
Attach external (local folder) with big files in it. Browse it in WebUI and the folder size will be incorrect or pending. Navigate to some sub folder that contains more then 2GB or have at least one file with size more than 2GB and will see "pending" in the file size column.
In my case file with 3.9G (4143738495 bytes) is shown as "pending" size. The upper folder also is with "Pending size". The same file in oc_filechache is with 4143738495 size so it is indexted currect.
Expected behaviour
Should see 3.9G
Actual behaviour
See "pending"
Server configuration
Operating system: OSMC (Debian 8.8)
Web server: Apache/2.4.10 (Debian)
Database: mysql Ver 14.14 Distrib 5.5.55, for debian-linux-gnu (armv7l) using readline 6.3
PHP version: php5 with fpm (PHP 5.6.30-0+deb8u1)
Nextcloud version: NC 12.0.0
Updated from an older Nextcloud/ownCloud or fresh install: Updated from NC 11.0.3
Where did you install Nextcloud from: from the official website
Signing status:
Signing status
No errors have been found.List of activated apps:
App list
Enabled: - calendar: 1.5.3 - contacts: 1.5.3 - dav: 1.3.0 - external: 2.0.2 - federatedfilesharing: 1.2.0 - files: 1.7.2 - files_external: 1.3.0 - files_pdfviewer: 1.1.1 - files_sharing: 1.4.0 - files_texteditor: 2.4.1 - files_videoplayer: 1.1.0 - gallery: 17.0.0 - gpxpod: 2.1.2 - logreader: 2.0.0 - lookup_server_connector: 1.0.0 - nextcloud_announcements: 1.1 - notifications: 2.0.0 - oauth2: 1.0.5 - password_policy: 1.2.2 - previewgenerator: 1.0.6 - provisioning_api: 1.2.0 - serverinfo: 1.2.0 - sharebymail: 1.2.0 - theming: 1.3.0 - twofactor_backupcodes: 1.1.1 - updatenotification: 1.2.0 - workflowengine: 1.2.0 Disabled: - activity - admin_audit - admin_notifications - bruteforcesettings - comments - encryption - federation - files_downloadactivity - files_trashbin - files_versions - firstrunwizard - mail - socialsharing_email - socialsharing_facebook - socialsharing_googleplus - survey_client - systemtags - user_external - user_ldapNextcloud configuration:
Config report
{
"system": {
"instanceid": "ocdhcp0sm80x",
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [
"XXX.XXX.X.XXX",
"X.XX.XXX.XX",
"XXXXXXXXX.XX"
],
"preview_libreoffice_path": "/usr/lib/libreoffice/program/soffice.bin",
"datadirectory": "/var/www/NCData",
"overwrite.cli.url": "https://XXX.XXX.X.XXX",
"dbtype": "mysql",
"version": "12.0.0.28",
"dbname": "NextCloud",
"dbhost": "127.0.0.1",
"dbtableprefix": "oc_",
"dbuser": "REMOVED SENSITIVE VALUE",
"dbpassword": "REMOVED SENSITIVE VALUE",
"logtimezone": "UTC",
"installed": true,
"memcache.local": "\OC\Memcache\APCu",
"allow_user_to_change_display_name": true,
"enable_avatars": false,
"quota_include_external_storage": false,
"updater.release.channel": "beta",
"appstore.experimental.enabled": true,
"mail_from_address": "test",
"mail_smtpmode": "php",
"mail_domain": "xxxxxxxxx.xx",
"mail_smtpsecure": "ssl",
"mail_smtpauthtype": "LOGIN",
"mail_smtpauth": 1,
"mail_smtphost": "xxxxl.xxx",
"mail_smtpport": "465 ",
"mail_smtpname": "REMOVED SENSITIVE VALUE",
"mail_smtppassword": "REMOVED SENSITIVE VALUE",
"loglevel": 0,
"maintenance": false,
"theme": "",
"htaccess.RewriteBase": "/",
"debug": true,
"updater.secret": "REMOVED SENSITIVE VALUE"
}
}
Are you using external storage, if yes which one: local
Are you using encryption: no
Are you using an external user-backend, if yes which one: No
Client configuration
Browser: 58.0.3029.110 (64-bit)
Operating system: Windows 7
Logs
Web server error log
Web server error log
no errorsNextcloud log (data/nextcloud.log)
Nextcloud log
no errorsBrowser log
Browser log
no errorsThe text was updated successfully, but these errors were encountered: