occ files:scan -all Makes all folders show that they were just changed

[kwstone]

Steps to reproduce

1. Setup Nextcloud 10
2. Connect to external SMB folders
3. Use occ files:scan -all to scan for changes

Expected behaviour

New files or folders should be shown but no changes to their last update time and date unless it did change.

Actual behaviour

All folders on SMB shares show that they were updated or modified when the scan took place.

Server configuration

Ubuntu 16.4

Web server:
Apache

Database:
MySQL

PHP version:
7.0.8

Nextcloud version: (see Nextcloud admin page)
10

Updated from an older Nextcloud/ownCloud or fresh install:
Fresh install

Where did you install Nextcloud from:
Downloaded from Nextcloud.com

Signing status:

Signing status

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results here.

List of activated apps:

App list

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your Nextcloud installation folder

The content of config/config.php:

Config report

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your Nextcloud installation folder

or 

Insert your config.php content here
(Without the database password, passwordsalt and secret)

Are you using external storage, if yes which one: local/smb/sftp/...
SMB
Are you using encryption: yes/no
No
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
No

LDAP configuration (delete this part if not used)

LDAP config

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your Nextcloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';


Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

### Client configuration

Browser:

Operating system:

Logs

Web server error log

Web server error log

Insert your webserver log here

#### Nextcloud log (data/nextcloud.log)

Nextcloud log

Insert your Nextcloud log here

#### Browser log

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

[nickvergessen]

Well the files:scan command is not recommended for daily use.

[kwstone]

It's the only way I have been able to get Nextcloud to pick up or recognize new files placed on the smb server. With only a few files it works without but after several gigs of files have been accessed and hundreds of folders, It just doesn't seem to look any further. I do have Check for Changes set to Once every direct access. But does that mean once every access via a sync client? Because that is the access most of my clients are using. If the check for changes should be working then I believe that is also listed as a bug that I found yesterday.

[it-consult]

Same problem. 37 GiG of data, the only way ist to use "occ files:scan -all"
Version 10.0.2

[nickvergessen]

But does that mean once every access via a sync client? Because that is the access most of my clients are using.

I think it does yes. But let's confirm by @icewind1991

[icewind1991]

The sync client only accesses the users home folder and will thus only find top level changes.

With SMB, you can run an occ command that actively listen for smb changes.

This issue will be fixed with #3449

[bestouff]

#3449 is merged, is that bug fixed ?

[seblu]

I scanned (occ files:scan -all) new local folders with 12.0.4 and there modification date is not correctly reported.

[nickvergessen]

Well as per it's description: will rescan all files of all known users

Maybe --unscanned helps preventing that?

[seblu]

I dropped oc_filecache and launched occ files:scan -all --unscanned. All directory are 1s hold. I don't get why directory are managed differently than files on this.

[nickvergessen]

You should not drop oc_filecache, it's not a cache but an index.
The naming can't be changed easily which is why we are still using it.

[seblu]

Yes thanks, I know that not a cache. I should have tested by creating a new directory. I got excited.

[LibreGit]

Had this issue with nc 12 too. I saw that there was a storage_mtime field with the correct values in the oc_filecache table.

Just execute this sql statement after each occ files:scan :

UPDATE oc_filecache SET mtime=storage_mtime

or from the shell :

mysql -u root -p -e "UPDATE oc_filecache SET mtime=storage_mtime" nextcloud

[dvergeylen]

Had this issue with nc 12 too. I saw that there was a storage_mtime field with the correct values in the oc_filecache table.

Just execute this sql statement after each occ files:scan :

UPDATE oc_filecache SET mtime=storage_mtime

or from the shell :

mysql -u root -p -e "UPDATE oc_filecache SET mtime=storage_mtime" nextcloud

Briliant, works like a charm as a workaround 👌 Thanks!

[hboetes]

What would be the postgresql command that does the same?

Edit:

su - postgres
postgres@example ~ % psql
postgres=# \l
                              List of databases
   Name    |  Owner   | Encoding | Collate |  Ctype  |   Access privileges
-----------+----------+----------+---------+---------+-----------------------
 nextcloud | something   | UTF8     | C.UTF-8 | C.UTF-8 | something=CTc/something
[snip]
(4 rows)
postgres=# \connect nextcloud
You are now connected to database "nextcloud" as user "postgres".
nextcloud=# UPDATE oc_filecache SET mtime=storage_mtime;
UPDATE 38594

[alpsayin]

I just had this issue with Nextcloud 23.0.2, Ubuntu 20.04, MariaDB 10.3.32, apache2.
Solution mentioned by @LibreGit works.
Are there any logs I can provide?
Edit:
Server error log is literally empty.
Nextcloud log also didn't have anything new after the scan --all command.

[viulian]

I too confirm the bug with the latest 25.0.1.1 version. While migrating the data between two virtual machines, I have used:

scp -rp source destination

to preserve the modification times. The folder stats are correct.

However, after triggering the scan, all folders appear to have been last modified at the moment of the copy. Values in the storage_mtime column are correct, but the mtime defaulted to what seem to be the time of the operation.

[FlexMcMurphy]

Thank you all for this very useful thread.
I also see this behaviour in Nextcloud 25.0.0 RC3

When a folder is added to Nexcloud either by dropping it in to the WebUI OR rsync'ing it in to the datadir and then running occ files:scan --all that folder is given a current Timestamp and not the os-level modified TS of the folder.

However, under normal usage perhaps that behavior makes sense? It's only a problem when you are setting up a new NC instance with your OLD data and for the first sync only you want to keep all the timestamps from the "old" NC instance.

So I wonder if the fix here is not to change the existing code but to add a new parameter to occ files:scan -all?
e.g: occ files:scan --all --preserve_os_mod_ts that tells it to use the folders os-level modified TS in the mtime and storage_mtime columns of the NC oc_filecache table. And then just using occ files:scan --all could continue to work as it currently does?

Just my two cents!

Flex