File deleted, but still visible in interface #741

ebogaard · 2016-08-05T07:41:09Z

In certain cases a file can get in a certain state where it's deleted, but still visible in the web interface.
The activities show it as deleted, but it's still visible in the web interface.
In the owncloud windows app, the file isn't visible, but the logging shows errors.

This only happens when you shuffle a file to- and from a 'Shared with you'-folder.

Steps to reproduce

Create, for example, an .odt file and put this in the root folder of the Files app
Drag and drop the file to a folder of your own
Drag and drop the file to from this folder back to the root, using the home-icon in the breadcrumb.
Drag and drop the file to the 'Shared with you'-folder
Drag and drop the file to from this folder back to the root, using the home-icon in the breadcrumb.
Now you get the error.

Sometimes you need to repeat this cycle.

Expected behaviour

The file is moved from location to location as usual and is still fully accessible afterwards.

Actual behaviour

The file shows it's deleted in the activities.
When trying to open the file, the following error is shown:

Service Unavailable
Could not open file
Technical details
• Remote Address: x.x.x.x
• Request ID: V6NgGqrNXzX0sE7McbjqDwAAAAc

The desktop app doesn't sync this file anymore and shows the following error in the log:

14:36:32|618|Test.odt|INST_RENAME|Up|1470148141|3851dea77e2b4d2719551e2e8418734e|32782|00000150o7UiddusMtKv|4||201|0|0|||INST_NONE|
|0|test_sharing.odt|INST_RENAME|Up|1470321128|45b8eccb37873e8ee0406e0e4ea3e0fd|9240|00000358o7UiddusMtKv|2|Error downloading https://URL/remote.php/webdav/test_sharing.odt - server replied: Forbidden|403|0|0|||INST_NONE|
#=#=#=# Syncrun started  until  (0 msec)
|0|test_sharing.odt|INST_RENAME|Up|1470321128|45b8eccb37873e8ee0406e0e4ea3e0fd|9240|00000358o7UiddusMtKv|6|Continue blacklisting: Error downloading https://URL/remote.php/webdav/test_sharing.odt - server replied: Forbidden|403|0|0|||INST_NONE|
#=#=#=# Syncrun started  until  (0 msec)
|0|test_sharing.odt|INST_RENAME|Up|1470321128|45b8eccb37873e8ee0406e0e4ea3e0fd|9240|00000358o7UiddusMtKv|6|The item is not synced because of previous errors: Error downloading https://URL/remote.php/webdav/test_sharing.odt - server replied: Forbidden|0|0|0|||INST_NONE|
#=#=#=# Syncrun started  until  (0 msec)
#=#=#=# Syncrun started  until  (0 msec)
|0|Documents|INST_SYNC|Up|1470321538||0||4||0|0|1470321392|57a352f07d3b9|00000062o7UiddusMtKv|INST_NONE|
|0|test_sharing.odt|INST_RENAME|Up|1470321128|45b8eccb37873e8ee0406e0e4ea3e0fd|9240|00000358o7UiddusMtKv|2|Error downloading https://URL/remote.php/webdav/test_sharing.odt - server replied: Forbidden|403|0|0|||INST_NONE|
#=#=#=# Syncrun started  until  (0 msec)
|0|Documents|INST_SYNC|Up|1470321538||0||4||0|0|1470321392|57a352f07d3b9|00000062o7UiddusMtKv|INST_NONE|
|0|test_sharing.odt|INST_RENAME|Up|1470321128|45b8eccb37873e8ee0406e0e4ea3e0fd|9240|00000358o7UiddusMtKv|6|Continue blacklisting: Error downloading https://URL/remote.php/webdav/test_sharing.odt - server replied: Forbidden|403|0|0|||INST_NONE|
#=#=#=# Syncrun started  until  (0 msec)

You can't do anything with this file anymore.

Server configuration

Operating system: Centos 7

Web server: Httpd 2.4

Database: MariaDB 10

PHP version: 5.6

Nextcloud version: 9.0.53

Updated from an older Nextcloud/ownCloud or fresh install: 9.0.51 and 52

Where did you install Nextcloud from: nextcloud.com - tar.gz

Signing status: No relevant files modified

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP

Client configuration

Browser: Chrome and Firefox

Operating system: Windows 7

Logs

Web server error log

x.x.x.x - user@name [04/Aug/2016:17:32:39 +0200] "PROPFIND /remote.php/webdav/ HTTP/1.1" 207 373 "-" "Mozilla/5.0 (Windows) mirall/2.2.2 (build 6192)"
x.x.x.x - user@name [04/Aug/2016:17:32:42 +0200] "GET /remote.php/webdav/testdocument_2.odt HTTP/1.1" 503 8738 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0"
x.x.x.x - user@name [04/Aug/2016:17:32:42 +0200] "GET /remote.php/webdav/testdocument_2.odt HTTP/1.1" 503 8738 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0"
x.x.x.x - - [04/Aug/2016:17:32:42 +0200] "GET /core/js/oc.js?v=517a1bf90ae552132656a98069c3d3a9 HTTP/1.1" 200 1129 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0"
x.x.x.x - - [04/Aug/2016:17:32:57 +0200] "GET /ocs/v2.php/apps/notifications/api/v1/notifications?format=json HTTP/1.1" 200 74 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0"

#### Nextcloud log (data/nextcloud.log)

Nextcloud log

{"reqId":"V6NgGqrNXzX0sE7McbjqDwAAAAc","remoteAddr":"x.x.x.x","app":"PHP","message":"fopen(\/opt\/repo\/nextcloud\/afc945d8-ea58-1034-8b97-2755ab2f238e\/files\/testdocument_2.odt): failed to open stream: No such file or directory at \/var\/www\/html\/nextcloud-9.0.53\/lib\/private\/files\/storage\/local.php#261","level":3,"time":"2016-08-04 17:32:42","method":"GET","url":"\/remote.php\/webdav\/testdocument_2.odt","user":"afc945d8-ea58-1034-8b97-2755ab2f238e"}
{"reqId":"V6NgGqrNXzX0sE7McbjqDwAAAAc","remoteAddr":"x.x.x.x","app":"webdav","message":"Exception: {\"Message\":\"HTTP\\\/1.1 503 Could not open file\",\"Exception\":\"Sabre\\\\DAV\\\\Exception\\\\ServiceUnavailable\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/html\\\/nextcloud-9.0.53\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(83): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->get()\\n#1 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpGet(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#2 \\\/var\\\/www\\\/html\\\/nextcloud-9.0.53\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#3 \\\/var\\\/www\\\/html\\\/nextcloud-9.0.53\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:GET', Array)\\n#4 \\\/var\\\/www\\\/html\\\/nextcloud-9.0.53\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#5 \\\/var\\\/www\\\/html\\\/nextcloud-9.0.53\\\/apps\\\/dav\\\/appinfo\\\/v1\\\/webdav.php(55): Sabre\\\\DAV\\\\Server->exec()\\n#6 \\\/var\\\/www\\\/html\\\/nextcloud-9.0.53\\\/remote.php(138): require_once('\\\/var\\\/www\\\/html\\\/n...')\\n#7 {main}\",\"File\":\"\\\/var\\\/www\\\/html\\\/nextcloud-9.0.53\\\/apps\\\/dav\\\/lib\\\/connector\\\/sabre\\\/file.php\",\"Line\":306,\"User\":\"afc945d8-ea58-1034-8b97-2755ab2f238e\"}","level":4,"time":"2016-08-04 17:32:42","method":"GET","url":"\/remote.php\/webdav\/testdocument_2.odt","user":"afc945d8-ea58-1034-8b97-2755ab2f238e"}
{"reqId":"V6NgGqrNXzX0sE7McbjqDwAAAAc","remoteAddr":"x.x.x.x","app":"PHP","message":"Cannot modify header information - headers already sent by (output started at \/var\/www\/html\/nextcloud-9.0.53\/3rdparty\/sabre\/http\/lib\/Sapi.php:83) at \/var\/www\/html\/nextcloud-9.0.53\/3rdparty\/sabre\/http\/lib\/Sapi.php#58","level":3,"time":"2016-08-04 17:32:42","method":"GET","url":"\/remote.php\/webdav\/testdocument_2.odt","user":"afc945d8-ea58-1034-8b97-2755ab2f238e"}
{"reqId":"V6NgGqrNXzX0sE7McbjqDwAAAAc","remoteAddr":"x.x.x.x","app":"PHP","message":"Cannot modify header information - headers already sent by (output started at \/var\/www\/html\/nextcloud-9.0.53\/3rdparty\/sabre\/http\/lib\/Sapi.php:83) at \/var\/www\/html\/nextcloud-9.0.53\/3rdparty\/sabre\/http\/lib\/Sapi.php#63","level":3,"time":"2016-08-04 17:32:42","method":"GET","url":"\/remote.php\/webdav\/testdocument_2.odt","user":"afc945d8-ea58-1034-8b97-2755ab2f238e"}

The text was updated successfully, but these errors were encountered:

rullzer · 2016-08-06T18:42:27Z

@jancborchardt this is what happend to you as well right?

jancborchardt · 2016-08-11T11:16:43Z

Not sure. For me it happened on a rename after moving it out of a federated share. @LukasReschke and @MorrisJobke investigated so they might have more info.

MorrisJobke · 2016-08-11T12:21:46Z

Not sure. For me it happened on a rename after moving it out of a federated share.

It was not a federated share. It was a plain share. Seems to be the same issue.

milgner · 2016-09-05T08:11:22Z

Just experiencing the same problem. Are there any workarounds for this? (Situation and logs are equivalent, only difference is that Caddy/PHP7-FPM is used as the web server)

MorrisJobke · 2016-09-05T12:13:03Z

@LukasReschke @rullzer @icewind1991 What was the outcome of this issue we had in Stuttgart? Any details you maybe want to share with all? 😉

ctvera · 2016-12-06T22:19:10Z

I have the same problem on just installed server.

I uploaded some data, shared them. Second user moved them from shared folder and then I see some items twice.

falolaf · 2017-01-04T18:25:11Z

I believe that I have managed to trigger this twice the last week. In my case it has been moving shared folders, within same share and outside the share.

All documents in the folders have been duplicated after move. Not possible to open or remove duplicated files. I had to move original files out of the folder and then delete the folder. Then Create the folder again an put original files in the new folder.

openSUSE Leap 42.1
NC 11.0.0 (stable)
PHP 5.6.29
Apache 2.4
Mariadb 10.0.28

I should add that this is a migration from OC 9.1.2 directly to NC 11, if that matters.

MorrisJobke · 2017-01-05T14:08:42Z

cc @rullzer @icewind1991

plrunner · 2017-02-14T17:52:34Z

@falolaf
I am experiencing the same problem when cross moving a folder from a shared folder to another shared folder, if it may help. I am on NC 10.0.3 (upgraded from 10.0.0, 10.0.1, 10.0.2...).

falolaf · 2017-02-14T17:59:54Z

The workaround I use is to move the folder out of NC sync, i.e. NC sees it as the folder has been removed totally, then move the folder to correct place. One loose the history by doing this but there are no duplicates.

plrunner · 2017-02-15T12:24:47Z

This bug might be the same reported here on help.nextcloud.com, where files and folders get inexplicably duplicated and the only way to clean up that mess is to move the whole parent folder away from NC and put it in place again later.

I could not precisely reproduce it though, but if it may help, a recap of the workflow to reproduce it is the following:

all users are on win7 with desktop client v2.2.4.2, server version is 10.0.3.

userA shares folder1 and folder2 with userA, userB, userC
userB creates subfolder1 in folder1 and put several files and subsubfolders in there
userC renames subfolder1 in subfolder2
userD cut-paste subfolder2 from folder1 to folder2
elements in subfolder2 get duplicated after the move

Another effect probably related to this bug is that the users keep getting prompted with the "backup detected" dialog.

falolaf · 2017-02-15T12:44:26Z

It looks like the same problem indeed! I'm on NC11.0.1 though and use desktop clients on win7 and openSuse leap 42.1. The same happens when moving folders in the web gui too. 2017-02-15 13:24 GMT+01:00 plrunner <notifications@github.com>:

…

This bug might be the same reported here on help.nextcloud.com <https://help.nextcloud.com/t/duplicate-files-delete-not-possible/8869>, where files and folders get inexplicably duplicated and the only way to clean up that mess is to move the whole parent folder away from NC and put it in place again later. I could not precisely reproduce it though, but if it may help, a recap of the workflow to reproduce it is the following: all users are on win7 with desktop client v2.2.4.2, server version is 10.0.3. - userA shares folder1 and folder2 with userA, userB, userC - userB creates subfolder1 in folder1 and put several files and subsubfolders in there - userC renames subfolder1 in subfolder2 - userD cut-paste subfolder2 from folder1 to folder2 - elements in subfolder2 get duplicated after the move Another effect probably related to this bug is that the users keep getting prompted with the "backup detected" dialog. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#741 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJYmX6-T3MTqcK0HWtvzs9pD4-x8X0YXks5rcu6SgaJpZM4JdbnB> .

bat553 · 2017-09-15T19:57:51Z

Hi ! Is there any fix in sight ?

ccerrillo · 2018-01-11T08:51:25Z

Hi, any workaround to this?

ccerrillo · 2018-02-09T08:41:17Z

@icewind1991 Maybe this could help here?

dabenzel · 2018-02-21T23:08:19Z

Have the same issue. Didn't help. Folder is not in Filesystem Anymore, but on the frontend it exists. I also can't find that folder in any database.

jospoortvliet · 2018-08-10T13:43:35Z

Hi, can you guys reproduce this with Nextcloud 14?

falolaf · 2018-08-14T06:55:48Z

I have not tested with NC 14 but in NC 13 I can't any longer reproduce this. I believe there was another issue fixed during the spring. Can't find that issue now though.

patricksebastien · 2018-09-05T17:11:25Z

I have the same issue on 13.05, the file doesn't exist on the server anymore but it's still showing in the interface, when trying to download or delete the file:

DELETE https://xxx/remote.php/webdav/hjhjh/file.abc 404 ()

What can I do to remove the listing? I tried to files:scan --all but it didn't fix this issue.

MBfromOK · 2018-10-05T03:15:32Z

@jospoortvliet

Hi, can you guys reproduce this with Nextcloud 14?

Debian 9.1
NextCloud 14.0.1
Php7.1
Apache 2.4.25
Mariadb 10.1.26

I can verify that non-existent files still show in web ui. I just rebuilt my NextCloud VM then used the Webmin file manager to move the data files to the group folders and user folders in the new NC data directory.

That file copy tool was apparently buggy and put some folders in the wrong level of the tree, after I cut & pasted them to the correct location the Web ui still shows them (even though the folder & files are non-existent on the drive).

Things I have tried:

wwwuser: $ php occ files:scan --all
wwwuser: $ php occ files:scan --path=/userid/files
wwwuser: $ php occ maintenance:repair --include-expensive

SunyataZero · 2018-10-08T12:17:41Z

I'm a user and i've also had a similar problem. The differences i know of are these:

For me the problem appeared after moving a file
In my case the problem didn't involve the "Shared with you" directory

Do you want me to write a separate issue for this?

Server: cloud.disroot.org

Client: Nextcloud-2.3.3-x86_64.AppImage (OS: Lubuntu 18.04 64-bit)

The problem causes the sync download process to stop in my client. This is what it looks like in my client (on Ubuntu):

Some thoughts i've had on what can be improved:

The error message says "service unavailble" but that's not really true right? Rather the error (as i understand it after talking with the disroot support) has to do with a mismatch between the file entries in the database and the actual data that is stored on the server
Also the sync process could continue despite the error
Not sure if this is realistic but maybe selecting "force sync" in the client could rebuild/clean the database?
(Best of all would of course be if the problem didn't arise in the first place!)

JulienLaclaverie · 2018-10-17T09:27:44Z

I encountered a similar problem and found a workaround to fix it, this might help you too : #4786 (comment)

skjnldsv · 2019-06-06T13:33:22Z

As there is no feedback since a while and since I cannot reproduce the original issue anymore, I will close this ticket. If this is still happening please make sure to upgrade to the latest version. After that, feel free to reopen.

AstroGD · 2020-01-25T14:40:04Z

I'm having the ghostfile issue on the newest version of nextcloud in group-folders from groupfolders-plugin. Is this the right place to report it as groupfolder-contributors seem to not be responding to issues? (Error ocurred while trying to give information to nextcloud/groupfolders#243 )

robertmx · 2021-08-07T11:10:50Z

Still happening on nextcloud 22.

nickvergessen added bug needs info feature: sharing labels Sep 28, 2016

plrunner mentioned this issue Feb 16, 2017

Duplicated files and folders when moving from one shared folder to another (can reproduce it) #3502

Closed

MorrisJobke removed the needs info label Aug 31, 2017

MorrisJobke mentioned this issue Aug 31, 2017

Ghost files in cache due to fast operations #5622

Closed

nextcloud-bot added the stale Ticket or PR with no recent activity label Jun 20, 2018

nextcloud-bot removed the stale Ticket or PR with no recent activity label Aug 10, 2018

skjnldsv closed this as completed Jun 6, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

File deleted, but still visible in interface #741

File deleted, but still visible in interface #741

ebogaard commented Aug 5, 2016

rullzer commented Aug 6, 2016

jancborchardt commented Aug 11, 2016

MorrisJobke commented Aug 11, 2016

milgner commented Sep 5, 2016 •

edited

Loading

MorrisJobke commented Sep 5, 2016

ctvera commented Dec 6, 2016

falolaf commented Jan 4, 2017

MorrisJobke commented Jan 5, 2017

plrunner commented Feb 14, 2017

falolaf commented Feb 14, 2017

plrunner commented Feb 15, 2017

falolaf commented Feb 15, 2017 via email

bat553 commented Sep 15, 2017

ccerrillo commented Jan 11, 2018

ccerrillo commented Feb 9, 2018

dabenzel commented Feb 21, 2018

jospoortvliet commented Aug 10, 2018

falolaf commented Aug 14, 2018

patricksebastien commented Sep 5, 2018

MBfromOK commented Oct 5, 2018 •

edited

Loading

SunyataZero commented Oct 8, 2018 •

edited

Loading

JulienLaclaverie commented Oct 17, 2018

skjnldsv commented Jun 6, 2019

AstroGD commented Jan 25, 2020

robertmx commented Aug 7, 2021

File deleted, but still visible in interface #741

File deleted, but still visible in interface #741

Comments

ebogaard commented Aug 5, 2016

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

rullzer commented Aug 6, 2016

jancborchardt commented Aug 11, 2016

MorrisJobke commented Aug 11, 2016

milgner commented Sep 5, 2016 • edited Loading

MorrisJobke commented Sep 5, 2016

ctvera commented Dec 6, 2016

falolaf commented Jan 4, 2017

MorrisJobke commented Jan 5, 2017

plrunner commented Feb 14, 2017

falolaf commented Feb 14, 2017

plrunner commented Feb 15, 2017

falolaf commented Feb 15, 2017 via email

bat553 commented Sep 15, 2017

ccerrillo commented Jan 11, 2018

ccerrillo commented Feb 9, 2018

dabenzel commented Feb 21, 2018

jospoortvliet commented Aug 10, 2018

falolaf commented Aug 14, 2018

patricksebastien commented Sep 5, 2018

MBfromOK commented Oct 5, 2018 • edited Loading

SunyataZero commented Oct 8, 2018 • edited Loading

JulienLaclaverie commented Oct 17, 2018

skjnldsv commented Jun 6, 2019

AstroGD commented Jan 25, 2020

robertmx commented Aug 7, 2021

milgner commented Sep 5, 2016 •

edited

Loading

MBfromOK commented Oct 5, 2018 •

edited

Loading

SunyataZero commented Oct 8, 2018 •

edited

Loading