occ files:scan error undefined method get #31077

felixboehm · 2018-04-10T15:57:47Z

https://github.com/owncloud/core/blob/v10.0.7/lib/private/Files/Node/Root.php#L338

assumes to have user home like: userId/files
but with homeFolderNamingRule the user home is not like that.
https://github.com/owncloud/user_ldap/blame/master/lib/User/UserEntry.php#L258

getHome() is not used in Root.php ??

occ files:scan throws error:

Starting scan for user 1 out of 1 (416e1740-2aa1-4b34-bc41-12113af99f56) 
PHP Fatal error: Call to undefined method OC\Files\Node\File::get() 
in /var/www/html/owncloud/lib/private/Files/Node/Root.php on line 360

Server v10.0.7
LDAP v0.10.0

The text was updated successfully, but these errors were encountered:

felixboehm · 2018-04-10T15:58:44Z

@butonic please check

ownclouders · 2018-04-10T16:03:02Z

GitMate.io thinks the contributor most likely able to help you is @PVince81.

Possibly related issues are #14681 (Internal Server Error after attempting to do "occ files:scan"), #23333 (occ file:scan reports error if user never logged on), #15877 (When occ had an error, move back the files), #19115 (occ files:scan's memory consumption grows with amount of files scanned), and #26902 (error).

felixboehm · 2018-04-10T16:09:35Z

next: provides three lines of patch to log the pathes
@IljaN

cdamken · 2018-04-10T17:50:36Z

We have this two patches to have better logs:

Index: lib/private/Files/Node/Root.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- lib/private/Files/Node/Root.php	(date 1519056277000)
+++ lib/private/Files/Node/Root.php	(date 1523378469000)
@@ -355,6 +355,17 @@
 			$folder = $this->newFolder($dir);
 		}
 
+		$folderType = get_class($folder);
+
+		$l = \OC::$server->getLogger();
+		$l->debug("Type of folder-var $folderType");
+
+		if ($folder instanceof Node) {
+			$l->debug("Node Path: ". $folder->getPath() . ' Internal Path: ' . $folder->getInternalPath());
+		}
+
+		# $l->debug('DUMP of $folder: ' . print_r($folder), true);
+
 		$dir = '/files';
 		try {
 			$folder = $folder->get($dir);

Index: lib/User/UserEntry.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- lib/User/UserEntry.php	(date 1513787270000)
+++ lib/User/UserEntry.php	(date 1523380741000)
@@ -271,6 +271,7 @@
 				$path = $this->config->getSystemValue('datadirectory',
 						\OC::$SERVERROOT.'/data' ) . '/' . $path;
 			}
+			\OC::$server->getLogger()->debug("DEBUG LDAP getHome path: $path");
 			return $path;
 		}

butonic · 2018-04-10T18:54:04Z

In https://github.com/owncloud/core/blob/v10.0.7/lib/private/Files/Node/Root.php#L338 you are looking at the virtual filesystem inside owncloud, which for users files always starts with /{userid}/files/

the actual file location is determined by the mount points.

butonic · 2018-04-10T18:59:32Z

scanning files for ldap users works fine with oc 10.0.7 and ldap 0.10.0

pako81 · 2018-04-10T19:01:46Z

@butonic indeed this does not happen for all users, but randomly for some of them. 2 LDAP users are affected until now

butonic · 2018-04-10T19:05:26Z

Reading the error message properly helps:

Call to undefined method OC\Files\Node\File::get()

true OC\Files\Node\File is a file and has no get method to get children like a directory has.

Let us have a look at /var/www/html/owncloud/lib/private/Files/Node/Root.php on line 360:

core/lib/private/Files/Node/Root.php

Lines 338 to 367 in f10d105

 public function getUserFolder($userId) { 

 $userObject = \OC::$server->getUserManager()->get($userId); 

 if (is_null($userObject)) { 

 \OCP\Util::writeLog('files', 'Backends provided no user object for ' . $userId, \OCP\Util::ERROR); 

 throw new \OC\User\NoUserException('Backends provided no user object for ' . $userId); 

 } 

 $userId = $userObject->getUID(); 

 \OC\Files\Filesystem::initMountPoints($userId); 

 $dir = '/' . $userId; 

 $folder = null; 

 try { 

 $folder = $this->get($dir); 

 } catch (NotFoundException $e) { 

 $folder = $this->newFolder($dir); 

 } 

 $dir = '/files'; 

 try { 

 $folder = $folder->get($dir); 

 } catch (NotFoundException $e) { 

 $folder = $folder->newFolder($dir); 

 } 

 return $folder; 

 }

in line 360 we assume that $folder is a folder. but we got a file, likely because $folder = $this->get($dir); in line 353 returned a folder. What does the filecache for this users storage look like. I assume the root entry got corrupted somehow.

butonic · 2018-04-10T20:07:37Z

an intact filecache should allow scanning, even if home folder naming rule was used:

% echo 'SELECT user_id, home FROM oc_accounts WHERE user_id="B5275C13-6466-43FD-A129-A12A6D3D9A0D"' | mysql -u root --password=123 -h 127.0.0.1 core
user_id	home
B5275C13-6466-43FD-A129-A12A6D3D9A0D	/var/www/html/core/data2/alicia@example.org
% echo 'SELECT s.id, storage, fileid, path, m.mimetype FROM oc_filecache f JOIN oc_storages s ON f.storage=s.numeric_id JOIN oc_mimetypes m ON f.mimetype = m.id WHERE s.id="home::B5275C13-6466-43FD-A129-A12A6D3D9A0D" ORDER BY fileid' | mysql -u root --password=123 -h 127.0.0.1 core
id	storage	fileid	path	mimetype
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	230		httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	231	cache	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	232	files	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	233	files/welcome.txt	text/plain
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	234	thumbnails	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	235	thumbnails/233	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	236	thumbnails/233/2048-2048-max.png	image/png
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	237	thumbnails/233/32-32.png	image/png
predator% echo 'SELECT user_id, home FROM oc_accounts WHERE user_id="B5275C13-6466-43FD-A129-A12A6D3D9A0D"' | mysql -u root --password=123 -h 127.0.0.1 core
user_id	home
B5275C13-6466-43FD-A129-A12A6D3D9A0D	/var/www/html/core/data2/alicia@example.org
% echo 'SELECT s.id, storage, fileid, path, m.mimetype FROM oc_filecache f JOIN oc_storages s ON f.storage=s.numeric_id JOIN oc_mimetypes m ON f.mimetype = m.id WHERE s.id="home::B5275C13-6466-43FD-A129-A12A6D3D9A0D" ORDER BY fileid' | mysql -u root --password=123 -h 127.0.0.1 core
id	storage	fileid	path	mimetype
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	230		httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	231	cache	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	232	files	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	233	files/welcome.txt	text/plain
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	234	thumbnails	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	235	thumbnails/233	httpd/unix-directory
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	236	thumbnails/233/2048-2048-max.png	image/png
home::B5275C13-6466-43FD-A129-A12A6D3D9A0D	41	237	thumbnails/233/32-32.png	image/png
% ./occ file:scan B5275C13-6466-43FD-A129-A12A6D3D9A0D -vvv                                                                                            

Starting scan for user 1 out of 1 (B5275C13-6466-43FD-A129-A12A6D3D9A0D)
	Folder /B5275C13-6466-43FD-A129-A12A6D3D9A0D/
	Folder /B5275C13-6466-43FD-A129-A12A6D3D9A0D/files
	File   /B5275C13-6466-43FD-A129-A12A6D3D9A0D/files/welcome.txt
	Folder /B5275C13-6466-43FD-A129-A12A6D3D9A0D/thumbnails
	Folder /B5275C13-6466-43FD-A129-A12A6D3D9A0D/thumbnails/233
	File   /B5275C13-6466-43FD-A129-A12A6D3D9A0D/thumbnails/233/32-32.png
	File   /B5275C13-6466-43FD-A129-A12A6D3D9A0D/thumbnails/233/2048-2048-max.png
	Folder /B5275C13-6466-43FD-A129-A12A6D3D9A0D/cache

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 5       | 3     | 00:00:00     |
+---------+-------+--------------+
% tree data2                                                                                                                                     
data2
└── alicia@example.org
    ├── cache
    ├── files
    │   └── welcome.txt
    └── thumbnails
        └── 233
            ├── 2048-2048-max.png
            └── 32-32.png

5 directories, 3 files

butonic · 2018-04-10T20:12:14Z

manually corrupting the filecache by setting the mimtype and mimepart of the root of the home storage to application/octet-stream is corrected correctly by occ file:scan

butonic · 2018-04-10T20:19:29Z

same with the files folder

without a filecache dump for the user cannot continue. please also double check the path in oc_accounts points to the correct location in the filesystem.

trying to put a file in place of the users home folder does not break but produces an error message:

% tree data2
data2
├── alicia@example.org
└── alicia@example.org.bak
    ├── cache
    ├── files
    │   └── welcome.txt
    └── thumbnails
        └── 233
            ├── 2048-2048-max.png
            └── 32-32.png

5 directories, 4 files
% ./occ file:scan B5275C13-6466-43FD-A129-A12A6D3D9A0D -vvv

Starting scan for user 1 out of 1 (B5275C13-6466-43FD-A129-A12A6D3D9A0D)
Home storage for user B5275C13-6466-43FD-A129-A12A6D3D9A0D not writable
Make sure you're running the scan command only as the user the web server runs as

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 0       | 0     | 00:00:00     |
+---------+-------+--------------+

same for replacing the file folder with a file:

% tree data2
data2
├── alicia@example.org
│   └── files
└── alicia@example.org.bak
    ├── cache
    ├── files
    │   └── welcome.txt
    └── thumbnails
        └── 233
            ├── 2048-2048-max.png
            └── 32-32.png

6 directories, 4 files
% ./occ file:scan B5275C13-6466-43FD-A129-A12A6D3D9A0D -vvv

Starting scan for user 1 out of 1 (B5275C13-6466-43FD-A129-A12A6D3D9A0D)
Home storage for user B5275C13-6466-43FD-A129-A12A6D3D9A0D not writable
Make sure you're running the scan command only as the user the web server runs as

+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 0       | 0     | 00:00:00     |
+---------+-------+--------------+

PVince81 · 2018-04-24T14:19:55Z

what's the next steps ? is this not reproducible ?

IljaN · 2018-04-24T14:29:33Z

@PVince81 AFAIR the problem was caused by filecache corruption caused by a storage-error.
@pako81 Close?

pako81 · 2018-05-11T15:04:30Z

this was caused by filecache corruption due to storage outage. Yes, closing - thanks

lock · 2019-07-30T13:00:04Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

felixboehm assigned butonic Apr 10, 2018

felixboehm added Type:Bug blue-ticket labels Apr 10, 2018

felixboehm added this to the planned milestone Apr 10, 2018

felixboehm assigned pako81 and unassigned butonic Apr 10, 2018

IljaN self-assigned this Apr 10, 2018

pako81 added sev2-high p2-high Escalation, on top of current planning, release blocker labels Apr 11, 2018

ownclouders mentioned this issue Apr 17, 2018

make easier the occ files:scan --repair #31166

Closed

PVince81 changed the title ~~occ files:scan error~~ occ files:scan error undefined method get Apr 23, 2018

pako81 closed this as completed May 11, 2018

PVince81 modified the milestones: development, QA Jun 13, 2018

lock bot locked as resolved and limited conversation to collaborators Jul 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

occ files:scan error undefined method get #31077

occ files:scan error undefined method get #31077

felixboehm commented Apr 10, 2018 •

edited

Loading

felixboehm commented Apr 10, 2018

ownclouders commented Apr 10, 2018

felixboehm commented Apr 10, 2018

cdamken commented Apr 10, 2018

butonic commented Apr 10, 2018

butonic commented Apr 10, 2018

pako81 commented Apr 10, 2018

butonic commented Apr 10, 2018 •

edited

Loading

butonic commented Apr 10, 2018

butonic commented Apr 10, 2018

butonic commented Apr 10, 2018

PVince81 commented Apr 24, 2018

IljaN commented Apr 24, 2018

pako81 commented May 11, 2018

lock bot commented Jul 30, 2019

occ files:scan error undefined method get #31077

occ files:scan error undefined method get #31077

Comments

felixboehm commented Apr 10, 2018 • edited Loading

felixboehm commented Apr 10, 2018

ownclouders commented Apr 10, 2018

felixboehm commented Apr 10, 2018

cdamken commented Apr 10, 2018

butonic commented Apr 10, 2018

butonic commented Apr 10, 2018

pako81 commented Apr 10, 2018

butonic commented Apr 10, 2018 • edited Loading

butonic commented Apr 10, 2018

butonic commented Apr 10, 2018

butonic commented Apr 10, 2018

PVince81 commented Apr 24, 2018

IljaN commented Apr 24, 2018

pako81 commented May 11, 2018

lock bot commented Jul 30, 2019

felixboehm commented Apr 10, 2018 •

edited

Loading

butonic commented Apr 10, 2018 •

edited

Loading