Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Call to a member function getId() on array at lib/private/Files/Cache/Updater.php on line 160 #26544

Closed
jknockaert opened this issue Apr 13, 2021 · 20 comments · Fixed by #30631
Closed
Labels
1. to develop Accepted and waiting to be taken care of bug feature: encryption (server-side)

Comments

@jknockaert
Copy link
Contributor

Happened upon deleting a file over DAV (using the official client) using NC 21.0.1. The same error has been reported by @doc75 for NC 20.0.9: #26473 (comment) and by @axheli
It's my best guess that the issue is caused by recent changes in the file cache code.
I would have include a conveniently formatted stack dump if someone finally got around fixing nextcloud/logreader#301

@jknockaert jknockaert added bug 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Apr 13, 2021
@jknockaert
Copy link
Contributor Author

A formatted trace dump is in #26473 (comment)
Not sure who can have a look into this. Provided that the issue seems to be with a change in the cache code there may be a relation with #26473.

@skid9000
Copy link

I have the same problem.

Fatal	webdav	Error: Call to a member function getId() on array

    /var/www/nextcloud/lib/private/Files/Cache/Updater.php - line 160:
    OC\Files\Cache\Cache->remove("files/Nope")
    /var/www/nextcloud/lib/private/Files/View.php - line 330:
    OC\Files\Cache\Updater->remove("files/Nope")
    /var/www/nextcloud/lib/private/Files/View.php - line 1180:
    OC\Files\View->removeUpdate(OCA\Files_Tr ... }}, "files/Nope")
    /var/www/nextcloud/lib/private/Files/View.php - line 725:
    OC\Files\View->basicOperation("unlink", "/Nope", [ "delete"])
    /var/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php - line 466:
    OC\Files\View->unlink("/Nope")
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php - line 179:
    OCA\DAV\Connector\Sabre\File->delete()
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php - line 281:
    Sabre\DAV\Tree->delete("Nope")
    /var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php - line 89:
    Sabre\DAV\CorePlugin->httpDelete(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 472:
    Sabre\DAV\Server->emit("method:DELETE", [ Sabre\HTTP ... }])
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 253:
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
    /var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php - line 321:
    Sabre\DAV\Server->start()
    /var/www/nextcloud/apps/dav/appinfo/v1/webdav.php - line 84:
    Sabre\DAV\Server->exec()
    /var/www/nextcloud/remote.php - line 167:
    require_once("/var/www/ne ... p")

@phanky5
Copy link

phanky5 commented Apr 17, 2021

same issue here

@heminei
Copy link

heminei commented Apr 19, 2021

I have the same issue with WebDAV on Windows 10 with files greater than 100MB:

Log:

[webdav] Fatal: Error: Call to a member function getId() on array at <<closure>>

 0. /home/nextcloud/example.com/lib/private/Files/Cache/Updater.php line 160
    OC\Files\Cache\Cache->remove("files/Photos/Pa ... 4")
 1. /home/nextcloud/example.com/lib/private/Files/View.php line 330
    OC\Files\Cache\Updater->remove("files/Photos/Pa ... 4")
 2. /home/nextcloud/example.com/lib/private/Files/View.php line 1180
    OC\Files\View->removeUpdate(OCA\Files_Trashb ... }}, "files/Photos/Pa ... 4")
 3. /home/nextcloud/example.com/lib/private/Files/View.php line 725
    OC\Files\View->basicOperation("unlink", "/Photos/Pamporo ... 4", ["delete"])
 4. /home/nextcloud/example.com/apps/dav/lib/Connector/Sabre/File.php line 466
    OC\Files\View->unlink("/Photos/xxxxx ... 4")
 5. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Tree.php line 179
    OCA\DAV\Connector\Sabre\File->delete()
 6. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 281
    Sabre\DAV\Tree->delete("files/xxxxx/P ... 4")
 7. /home/nextcloud/example.com/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->httpDelete(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
 8. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Server.php line 472
    Sabre\DAV\Server->emit("method:DELETE", [Sabre\HTTP\Requ ... }])
 9. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Server.php line 253
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
10. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Server.php line 321
    Sabre\DAV\Server->start()
11. /home/nextcloud/example.com/apps/dav/lib/Server.php line 332
    Sabre\DAV\Server->exec()
12. /home/nextcloud/example.com/apps/dav/appinfo/v2/remote.php line 35
    OCA\DAV\Server->exec()
13. /home/nextcloud/example.com/remote.php line 167
    require_once("/home/nextcloud ... p")

DELETE /remote.php/dav/files/xxxxx/Photos/xxxxxxx/xxxxxxxxxxxxxxxx.MP4
from 212.XX.XX.XXX by xxxxxxxx at 2021-04-19T18:35:17+00:00

@solracsf solracsf changed the title [21.0.1] "Error: Call to a member function getId() on array" at lib/private/Files/Cache/Updater.php on line 160 [21.0.1] Error: Call to a member function getId() on array at lib/private/Files/Cache/Updater.php on line 160 Apr 19, 2021
@realizelol
Copy link

Also the same here when deleting a single file on server (which isn't in a folder?).

Console error:

DELETE    https://example.com/remote.php/dav/files/user/testfile.txt 
[HTTP/2 500 Internal Server Error 803ms]

Also I get this error notification in webgui:
Error deleting file "testfile.txt".✖

Error in nextcloud.log:

{"reqId":"vnZgNtGyl7qO0yoGXzra","level":4,"time":"2021-04-21 10:40:30","remoteAddr":"XXX.XXX.XXX.XXX","user":"user","app":"webdav","method":"DELETE","url":"/remote.php/dav/files/testfile.txt","message":{"Exception":"Error","Message":"Call to a member function getId() on array","Code":0,"Trace":[{"file":"/var/www/html/nextcloud/lib/private/Files/Cache/Updater.php","line":160,"function":"remove","class":"OC\\Files\\Cache\\Cache","type":"->"},{"file":"/var/www/html/nextcloud/lib/private/Files/View.php","line":330,"function":"remove","class":"OC\\Files\\Cache\\Updater","type":"->"},{"file":"/var/www/html/nextcloud/lib/private/Files/View.php","line":1180,"function":"removeUpdate","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/html/nextcloud/lib/private/Files/View.php","line":725,"function":"basicOperation","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/html/nextcloud/apps/dav/lib/Connector/Sabre/File.php","line":466,"function":"unlink","class":"OC\\Files\\View","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php","line":179,"function":"delete","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":281,"function":"delete","class":"Sabre\\DAV\\Tree","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpDelete","class":"Sabre\\DAV\\CorePlugin","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/apps/dav/lib/Server.php","line":332,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/var/www/html/nextcloud/remote.php","line":167,"args":["/var/www/html/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/html/nextcloud/lib/private/Files/Cache/Cache.php","Line":541,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0","version":"21.0.1.1"}

Installed and enabled 3rd-party apps:

  • External user authentication (user_external)

best regards
realizeloll

@RafalLukawiecki
Copy link

I get the same error when deleting the file using the browser UI. Error is displayed "Error deleting file" and the file shows, but it disappears after page reload. Log contains the error as above:

[webdav] Fatal: Error: Call to a member function getId() on array at <<closure>>

 0. /usr/local/www/nextcloud/lib/private/Files/Cache/Updater.php line 160
    OC\Files\Cache\Cache->remove("files/xxx/xxxx  ... v")
 1. /usr/local/www/nextcloud/lib/private/Files/View.php line 330
    OC\Files\Cache\Updater->remove("files/xxx/xxxx  ... v")
 2. /usr/local/www/nextcloud/lib/private/Files/View.php line 1180
    OC\Files\View->removeUpdate(OCA\Files_Trashb ... }}, "files/xxx/xxxx  ... v")
 3. /usr/local/www/nextcloud/lib/private/Files/View.php line 725
    OC\Files\View->basicOperation("unlink", "/xxx/xxxx Prese ... v", ["delete"])
 4. /usr/local/www/nextcloud/apps/dav/lib/Connector/Sabre/File.php line 466
    OC\Files\View->unlink("/Ham/IRTS Prese ... v")
 5. /usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Tree.php line 179
    OCA\DAV\Connector\Sabre\File->delete()
 6. /usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 281
    Sabre\DAV\Tree->delete("files/rafal/Ham ... v")
 7. /usr/local/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->httpDelete(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
 8. /usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 472
    Sabre\DAV\Server->emit("method:DELETE", [Sabre\HTTP\Requ ... }])
 9. /usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 253
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
10. /usr/local/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php line 321
    Sabre\DAV\Server->start()
11. /usr/local/www/nextcloud/apps/dav/lib/Server.php line 332
    Sabre\DAV\Server->exec()
12. /usr/local/www/nextcloud/apps/dav/appinfo/v2/remote.php line 35
    OCA\DAV\Server->exec()
13. /usr/local/www/nextcloud/remote.php line 167
    require_once("/usr/local/www/ ... p")

DELETE /remote.php/dav/files/xxx/xxxx/practical%20machine%20learning%20.mov
from **** by rafal at 2021-04-21T14:28:57+01:00

@doc75
Copy link
Contributor

doc75 commented May 1, 2021

Any news on this ? I happens together with the message mentioned in #26473 at any delete.
WIth the official client this raise an error at every change of file synced when it involves a delete (with red cross icon). This is scary for most users even if at the end it seems to work fine despite this error message.

@solracsf solracsf changed the title [21.0.1] Error: Call to a member function getId() on array at lib/private/Files/Cache/Updater.php on line 160 Call to a member function getId() on array at lib/private/Files/Cache/Updater.php on line 160 May 3, 2021
@doobry-systemli
Copy link

doobry-systemli commented May 6, 2021

We have the same problem on our Nextcloud instance (21.0.1). Each DELETE via WebDAV API (even if triggered through the Nextcloud webinterface) results in the reported error Call to a member function getId() on array.

The file is removed from filesystem, but apparently something goes wrong when trying to remove it from filecache. When browsing the folder with the deleted file shortly afterwards in the web app (without hard-reloading the webinterface), the file is listed again. After a hard reload, it's gone.

Here again a full backtrace:

{
  "Exception": "Error",
  "Message": "Call to a member function getId() on array",
  "Code": 0,
  "Trace": [
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/lib/private/Files/Cache/Updater.php",
      "line": 160,
      "function": "remove",
      "class": "OC\\Files\\Cache\\Cache",
      "type": "->",
      "args": [
        "files/Talk/report.pdf"
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/lib/private/Files/View.php",
      "line": 330,
      "function": "remove",
      "class": "OC\\Files\\Cache\\Updater",
      "type": "->",
      "args": [
        "files/Talk/report.pdf"
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/lib/private/Files/View.php",
      "line": 1180,
      "function": "removeUpdate",
      "class": "OC\\Files\\View",
      "type": "->",
      "args": [
        {
          "cache": null,
          "scanner": {
            "__class__": "OC\\Files\\Cache\\Scanner"
          },
          "watcher": null,
          "propagator": null,
          "updater": {
            "__class__": "OC\\Files\\Cache\\Updater"
          },
          "__class__": "OCA\\Files_Trashbin\\Storage"
        },
        "files/Talk/report.pdf"
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/lib/private/Files/View.php",
      "line": 725,
      "function": "basicOperation",
      "class": "OC\\Files\\View",
      "type": "->",
      "args": [
        "unlink",
        "/Talk/report.pdf",
        [
          "delete"
        ]
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/apps/dav/lib/Connector/Sabre/File.php",
      "line": 466,
      "function": "unlink",
      "class": "OC\\Files\\View",
      "type": "->",
      "args": [
        "/Talk/report.pdf"
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/3rdparty/sabre/dav/lib/DAV/Tree.php",
      "line": 179,
      "function": "delete",
      "class": "OCA\\DAV\\Connector\\Sabre\\File",
      "type": "->",
      "args": []
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
      "line": 281,
      "function": "delete",
      "class": "Sabre\\DAV\\Tree",
      "type": "->",
      "args": [
        "files/user/Talk/report.pdf"
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
      "line": 89,
      "function": "httpDelete",
      "class": "Sabre\\DAV\\CorePlugin",
      "type": "->",
      "args": [
        {
          "__class__": "Sabre\\HTTP\\Request"
        },
        {
          "__class__": "Sabre\\HTTP\\Response"
        }
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/3rdparty/sabre/dav/lib/DAV/Server.php",
      "line": 472,
      "function": "emit",
      "class": "Sabre\\DAV\\Server",
      "type": "->",
      "args": [
        "method:DELETE",
        [
          {
            "__class__": "Sabre\\HTTP\\Request"
          },
          {
            "__class__": "Sabre\\HTTP\\Response"
          }
        ]
      ]
    },
    {
      "file": "/var/www/cloud.example3.org/nextcloud-21.0.1/3rdparty/sabre/dav/lib/DAV/Server.php",
      "line": 253,
      "function": "invokeMethod",
      "class": "Sabre\\DAV\\Server",
      "type": "->",
      "args": [
        {
          "__class__": "Sabre\\HTTP\\Request"
        },
        {
          "__class__": "Sabre\\HTTP\\Response"
        }
      ]
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/3rdparty/sabre/dav/lib/DAV/Server.php",
      "line": 321,
      "function": "start",
      "class": "Sabre\\DAV\\Server",
      "type": "->",
      "args": []
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/apps/dav/lib/Server.php",
      "line": 332,
      "function": "exec",
      "class": "Sabre\\DAV\\Server",
      "type": "->",
      "args": []
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/apps/dav/appinfo/v2/remote.php",
      "line": 35,
      "function": "exec",
      "class": "OCA\\DAV\\Server",
      "type": "->",
      "args": []
    },
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/remote.php",
      "line": 167,
      "args": [
        "/var/www/cloud.example.org/nextcloud-21.0.1/apps/dav/appinfo/v2/remote.php"
      ],
      "function": "require_once"
    }
  ],
  "File": "/var/www/cloud.example.org/nextcloud-21.0.1/lib/private/Files/Cache/Cache.php",
  "Line": 541,
  "CustomMessage": "--"
}

@heminei
Copy link

heminei commented May 11, 2021

I have the same issue with WebDAV on Windows 10 with files greater than 100MB:

Log:

[webdav] Fatal: Error: Call to a member function getId() on array at <<closure>>

 0. /home/nextcloud/example.com/lib/private/Files/Cache/Updater.php line 160
    OC\Files\Cache\Cache->remove("files/Photos/Pa ... 4")
 1. /home/nextcloud/example.com/lib/private/Files/View.php line 330
    OC\Files\Cache\Updater->remove("files/Photos/Pa ... 4")
 2. /home/nextcloud/example.com/lib/private/Files/View.php line 1180
    OC\Files\View->removeUpdate(OCA\Files_Trashb ... }}, "files/Photos/Pa ... 4")
 3. /home/nextcloud/example.com/lib/private/Files/View.php line 725
    OC\Files\View->basicOperation("unlink", "/Photos/Pamporo ... 4", ["delete"])
 4. /home/nextcloud/example.com/apps/dav/lib/Connector/Sabre/File.php line 466
    OC\Files\View->unlink("/Photos/xxxxx ... 4")
 5. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Tree.php line 179
    OCA\DAV\Connector\Sabre\File->delete()
 6. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 281
    Sabre\DAV\Tree->delete("files/xxxxx/P ... 4")
 7. /home/nextcloud/example.com/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->httpDelete(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
 8. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Server.php line 472
    Sabre\DAV\Server->emit("method:DELETE", [Sabre\HTTP\Requ ... }])
 9. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Server.php line 253
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
10. /home/nextcloud/example.com/3rdparty/sabre/dav/lib/DAV/Server.php line 321
    Sabre\DAV\Server->start()
11. /home/nextcloud/example.com/apps/dav/lib/Server.php line 332
    Sabre\DAV\Server->exec()
12. /home/nextcloud/example.com/apps/dav/appinfo/v2/remote.php line 35
    OCA\DAV\Server->exec()
13. /home/nextcloud/example.com/remote.php line 167
    require_once("/home/nextcloud ... p")

DELETE /remote.php/dav/files/xxxxx/Photos/xxxxxxx/xxxxxxxxxxxxxxxx.MP4
from 212.XX.XX.XXX by xxxxxxxx at 2021-04-19T18:35:17+00:00

My problem was with Cloudflare max upload file limit. When I disabled the proxy, I can upload files greater than 100MB with WebDav.

@prismopensource
Copy link

Same exact problem here also on 21.0.1

@skid9000
Copy link

Problem still there on 21.0.2

@korkosson
Copy link

Is there any solution for this issue (even some dirty one)? Due to this issue, sync is not working for some SW using WebDav access (eg. KeePass database sync).
Has anyone tried to use older version of the Cache code?

@korkosson
Copy link

korkosson commented Jun 16, 2021

Here is some (very brief) analysis:

  • File: lib\private\Files\Cache\Cache.php
  • Line 536: $entry = $this->get($file);

This is content of the $entry variable on this line: array(1) { ["encrypted"]=> bool(false) }

I have modified the line 538 to this: if ($entry && !is_array($entry)) {

There is a new error reported in the log: Error: Undefined index: size at /var/www/nextcloud/lib/private/Files/View.php#1340
but at least it is not visible to the user. Also WebDav sync is not reporting errors after deletion.

I know that it is ugly "fix" but I have seen the code for the first time today and I really wanted to get it working asap ;)

@frahi
Copy link

frahi commented Jun 25, 2021

I have a quite similar problem:

  • When deleting a file via DAV (from Windows nextcloud client) the client gets an internal server error 500.
  • This happens since I updated from 20.0.5 to 20.0.10 and it seems that it happens with every delete
  • Differs from the description of heminei as it happens to files of any size when deleting them here
  • Different to the description of korkosson $entry is an empty array here

Some debugging:
The function Cache::get in file lib\private\Files\Cache\Cache.php seems to return $this->partial[$file] sometimes even when there is no partially uploaded file. This results in the follow-up error in calling functions Cache::remove as well as in View.php.

public function get($file) {
	$query = $this->getQueryBuilder();
	$query->selectFileCache();

	if (is_string($file) or $file == '') {
		// normalize file
		$file = $this->normalize($file);

		$query->whereStorageId()
			->wherePath($file);
	} else { //file id
		$query->whereFileId($file);
	}

	$result = $query->execute();
	$data = $result->fetch();
	$result->closeCursor();

	//merge partial data
	if (!$data and is_string($file) and isset($this->partial[$file])) {
		return $this->partial[$file];
	} elseif (!$data) {
		return $data;
	} else {
		return self::cacheEntryFromData($data, $this->mimetypeLoader);
	}
}

Is there any parallel processing involved that manipulates Cache::partial?
I could not reproduce this with a fresh installation of 20.0.10 on a debian 10 in a virtual machine.

@szaimen szaimen added 1. to develop Accepted and waiting to be taken care of feature: encryption (server-side) and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels Aug 8, 2021
@mejo-
Copy link
Member

mejo- commented Aug 10, 2021

I experience the same issue on a Nextcloud 22.1.0 instance (and already with 21) - but only when deleting files from non-default storages. The collectives app brings its own storages for collectives and mounts them into the home folder. Deleting a file from this mountpoints throws the Call to a member function getId() on array exception. The instance doesn't have storage encryption enabled. Will try to further investigate.

@azul
Copy link
Contributor

azul commented Aug 18, 2021

Trying to figure out what's going on here based on the backlock posted by @mejo- above:

{
  "Exception": "Error",
  "Message": "Call to a member function getId() on array",
  "Code": 0,
  "Trace": [
    {
      "file": "/var/www/cloud.example.org/nextcloud-21.0.1/lib/private/Files/Cache/Updater.php",
      "line": 160,
      "function": "remove",
      "class": "OC\\Files\\Cache\\Cache",
      "type": "->",
      "args": [
        "files/Talk/report.pdf"
      ]
    },
  ]
}

Looking at lib/private/Files/Cache/Updater.php there's no call to getId. However the remove function in lib/private/Files/Cache/Cache.php has quite a few:

	public function remove($file) {
		$entry = $this->get($file);

		if ($entry) {
			$query = $this->getQueryBuilder();
			$query->delete('filecache')
				->whereFileId($entry->getId());
			$query->execute();

So it looks like $this->get($file); returns an array.
Chances are this comes from the store for partial data:

	/**
	 * get the stored metadata of a file or folder
	 *
	 * @param string | int $file either the path of a file or folder or the file id for a file or folder
	 * @return ICacheEntry|false the cache entry as array of false if the file is not found in the cache
	 */
	public function get($file) {
		$query = $this->getQueryBuilder();
		$query->selectFileCache();

		if (is_string($file) or $file == '') {
			// normalize file
			$file = $this->normalize($file);

			$query->whereStorageId()
				->wherePath($file);
		} else { //file id
			$query->whereFileId($file);
		}

		$result = $query->execute();
		$data = $result->fetch();
		$result->closeCursor();

		//merge partial data
		if (!$data and is_string($file) and isset($this->partial[$file])) {
			return $this->partial[$file];
		} elseif (!$data) {
			return $data;
		} else {
			return self::cacheEntryFromData($data, $this->mimetypeLoader);
		}
	}

mejo- added a commit that referenced this issue Oct 15, 2021
On external storages, removing files sometimes tries to remove partial
data, represented in an array.

`lib/private/Files/Cache/Cache.php:remove()` doesn't support to process
arrays.

According to @icewind1991, we can ignore this partial data in
`remove()`.

Fixes: #26544
mejo- added a commit that referenced this issue Oct 15, 2021
On external storages, removing files sometimes tries to remove partial
data, represented in an array.

`lib/private/Files/Cache/Cache.php:remove()` doesn't support to process
arrays.

According to @icewind1991, we can ignore this partial data in
`remove()`.

Fixes: #26544
Signed-off-by: Jonas Meurer <jonas@freesources.org>
@cartman29
Copy link

error

{"reqId":"YkmKaFQJhcwlNWtJAOWc","level":3,"time":"2021-12-06T09:10:02+01:00","remoteAddr":"","user":"--","app":"files","method":"","url":"--","message":"File entry could not be inserted but could also not be selected with getId() in order to perform an update. Please try again.","userAgent":"--","version":"22.2.3.0","exception":{"Exception":"RuntimeException","Message":"File entry could not be inserted but could also not be selected with getId() in order to perform an update. Please try again.","Code":0,"Trace":[{"file":"/var/www/nextcloud/lib/private/Files/Cache/Wrapper/CacheWrapper.php","line":135,"function":"insert","class":"OC\\Files\\Cache\\Cache","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":294,"function":"insert","class":"OC\\Files\\Cache\\Wrapper\\CacheWrapper","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":224,"function":"addToCache","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":427,"function":"scanFile","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":388,"function":"handleChildren","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":340,"function":"scanChildren","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":510,"function":"scan","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":521,"function":"OC\\Files\\Cache\\{closure}","class":"OC\\Files\\Cache\\Scanner","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/lib/private/Files/Cache/Scanner.php","line":509,"function":"runBackgroundScanJob","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/lib/private/Files/Utils/Scanner.php","line":185,"function":"backgroundScan","class":"OC\\Files\\Cache\\Scanner","type":"->"},{"file":"/var/www/nextcloud/apps/files/lib/BackgroundJob/ScanFiles.php","line":89,"function":"backgroundScan","class":"OC\\Files\\Utils\\Scanner","type":"->"},{"file":"/var/www/nextcloud/apps/files/lib/BackgroundJob/ScanFiles.php","line":125,"function":"runScanner","class":"OCA\\Files\\BackgroundJob\\ScanFiles","type":"->"},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/Job.php","line":51,"function":"run","class":"OCA\\Files\\BackgroundJob\\ScanFiles","type":"->"},{"file":"/var/www/nextcloud/lib/private/BackgroundJob/TimedJob.php","line":58,"function":"execute","class":"OC\\BackgroundJob\\Job","type":"->"},{"file":"/var/www/nextcloud/cron.php","line":127,"function":"execute","class":"OC\\BackgroundJob\\TimedJob","type":"->"}],"File":"/var/www/nextcloud/lib/private/Files/Cache/Cache.php","Line":340,"CustomMessage":"--"},"id":"61add307359bd"}

@cartman29
Copy link

2021-12-07 09:10:02.564 CET [39333] XXXXXX@YYYYYY ERREUR:  la valeur d'une clé dupliquée rompt la contrainte unique « fs_storage_path_hash »
2021-12-07 09:10:02.564 CET [39333] XXXXXX@YYYYYY DÉTAIL:  La clé « (storage, path_hash)=(3, 13c337fec*************1cf86af2c) » existe déjà.
2021-12-07 09:10:02.564 CET [39333] XXXXXX@YYYYYY INSTRUCTION :  INSERT INTO "oc_filecache" ("mimepart", "mimetype", "mtime", "size", "etag", "storage_mtime", "permissions", "name", "parent", "checksum", "path_hash", "path", "storage") VALUES($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)

@cartman29
Copy link

I don't want to downgrade security I want to fix this.

@mejo-
Copy link
Member

mejo- commented Jan 8, 2022

I now have a reproducer on a fresh Nextcloud installation:

  1. enable the encryption app
  2. enable server-side encryption in settings
  3. enable the Collectives app
  4. create a collective
  5. create a page in the collective an delete it

instead of 5. (creating + deleting a page in the collective), you can also navigate to Collectives/<your_collective> in the files app and create + delete a file there.

And the issue is only reproducible if the files_trashbin app is enabled. It seems to be related to the fact that the source storage (a collective mountpoint) is unencrypted when moving the file from the collective mountpoint to the trashbin storage.

mejo- added a commit that referenced this issue Jan 12, 2022
When moving a file to trash with encryption enabled, the cache gets
moved before the actual file. According to @icewind1991 this is in order
to not break object storage.

When moving a file from an unencrypted storage (e.g. a collectives
storage) to the encrypted trashbin storage, this causes errors, see

This commit fixes it by doing `updateEncryptedVersion()` on the target
cache entry *if* the source cache entry doesn't exist anymore, but the
corresponding target cache entry does exist already.

Fixes: #26544

Signed-off-by: Jonas Meurer <jonas@freesources.org>
backportbot-nextcloud bot pushed a commit that referenced this issue Jan 14, 2022
When moving a file to trash with encryption enabled, the cache gets
moved before the actual file. According to @icewind1991 this is in order
to not break object storage.

When moving a file from an unencrypted storage (e.g. a collectives
storage) to the encrypted trashbin storage, this causes errors, see

This commit fixes it by doing `updateEncryptedVersion()` on the target
cache entry *if* the source cache entry doesn't exist anymore, but the
corresponding target cache entry does exist already.

Fixes: #26544

Signed-off-by: Jonas Meurer <jonas@freesources.org>
backportbot-nextcloud bot pushed a commit that referenced this issue Jan 14, 2022
When moving a file to trash with encryption enabled, the cache gets
moved before the actual file. According to @icewind1991 this is in order
to not break object storage.

When moving a file from an unencrypted storage (e.g. a collectives
storage) to the encrypted trashbin storage, this causes errors, see

This commit fixes it by doing `updateEncryptedVersion()` on the target
cache entry *if* the source cache entry doesn't exist anymore, but the
corresponding target cache entry does exist already.

Fixes: #26544

Signed-off-by: Jonas Meurer <jonas@freesources.org>
mejo- added a commit that referenced this issue Jan 15, 2022
When moving a file to trash with encryption enabled, the cache gets
moved before the actual file. According to @icewind1991 this is in order
to not break object storage.

When moving a file from an unencrypted storage (e.g. a collectives
storage) to the encrypted trashbin storage, this causes errors, see

This commit fixes it by doing `updateEncryptedVersion()` on the target
cache entry *if* the source cache entry doesn't exist anymore, but the
corresponding target cache entry does exist already.

Fixes: #26544

Signed-off-by: Jonas Meurer <jonas@freesources.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. to develop Accepted and waiting to be taken care of bug feature: encryption (server-side)
Projects
None yet