External Storage not refreshing file list (tested in Dropbox)

[pokoli]

When you mount and external Storage you can see all the files on the external storage but the files are not synced. For example: If you mount a Dropbox account to your owncloud instance you can access and upload files and they are copied to Dropbox. But it doesn't work the other way arround. If you upload a file to Dropbox after it has been mounted on owncloud it never appears on the owncloud interface.

Steps to reproduce

1. Mount your Dropbox account to your owncloud instance.
2. After you can see the files on the owncloud interface go to Dropbox web and upload a file.
3. You can see the file on dropbox but not on owncloud.

Expected Behaviour

The files uploaded to the external storage must be accessbile from owncloud, whether it have uploaded from Owncloud or directly on the external storage

I have only tested it with Dropbox, i don't know if it's a Dropbox specifict issue or it affects all external storages types.

If only affects Dropbox, could you please provide some examples to implement the file syncronization.

[humplemann]

Same problem here with external FTP Storage. File upload via owncloud webinterface works very well and uploaded files are shown.
Uploaded files via FTP to the same dir are not shown in the webinterface. Those files have the same owner/group and file rights.

Thanks for any help.

[DeepDiver1975]

@icewind1991 Is this also file cache related or a problem within the files_external implementation? THX

[pokoli]

I updated to 4.5.3 and after a (very long long long time) login the files appeared.

I Uploaded a new file to Dropbox to test if it was working, and the files was missing. Signed out and SingedIn and the file is still missing.

I entered the Photos, clicked the Dropbox album (I have photos on DropBox) and after a long long page load it showed all the fotos. Then I went to the Files apps, and saw the new uploaded folder. So it's seems that is working now.

Do you know when the sync to dropbox is run?

There is some method to improve the performance? (it takes up to 5 minutes every time it syncs)

[karlitschek]

This will be fixed with ownCloud 5

[pokoli]

What's the solution for this bug?

Can be backported to 4.5?

[icewind1991]

Can be backported to 4.5?

Unfortunately not

[pokoli]

Can you elaborate why not?

icewind1991 notifications@github.com ha escrit:

Can be backported to 4.5?

Unfortunately not

---

Reply to this email directly or view it on GitHub:
#377 (comment)

[icewind1991]

There have been large changes to the way caching the filelist works for 5.0, this issue was caused by some underlying problems with the caching.

[pokoli]

And backporting the bug would imply backporting all those large changes. I understood!

icewind1991 notifications@github.com ha escrit:

There have been large changes to the way caching the filelist works for
5.0, this issue was caused by some underlying problems with the
caching.

---

Reply to this email directly or view it on GitHub:
#377 (comment)

[heitorsilva]

This problem is happening with me, on ownCloud 6.0.2 and 3 external storages from DreamObjects, using S3 storage.

My file list shows the same size for every storage folder and the same files on every storage folder as well... Like if it has some cache that never purges.

What should I do?

[lurdan]

Same here with dropbox (using oc 6.0.3).

Ugly workaround:
truncate table oc_filecache;

[torbennehmer]

I can confirm this problem with OC 6.0.2 and Dropbox as backend. I am using O'reilly ebook Sync to sync new ebooks I bought into my Dropbox (this is a "Dropbox App"). New files are available on Dropbox for two days now and they still do not appear in OC. cron.php is running every 15 minutes (just confirmed this). Even going to the directory in the OC Web Interface doesn't seem to change anything. This essentially makes this plugin useless for me (and probably most others), as my primary idea was to avoid having two sync clients running on my desktops.

[te-online]

I have this problem with OC 6.0.3, too. Dropbox as External Storage doesn't refresh, when putting files via Dropbox-client or Dropbox-website into the Dropbox. Everything else works great. Looks like some filecache problem again, maybe? Could someone have a quick look at it? @icewind1991?

[antzag]

I have some problem with OC 6.0.3. Dropbox as External Storage doesn't refresh, when putting files via Dropbox-client or Dropbox-client-Android into the Dropbox subfolder (shared and not shared with other Dropbox User), but when putting file into root folder of Dropbox (via client Dropbox or other) the file is synchronized into Owncloud. What should I do to fix it?

[te-online]

I can confirm, with Drobox and OC 6.0.3 the problem has multiple scenarios. Here ist what I observed:

• Files and Folders are affected the same
• Problem affects only content that is not created through OC
• Content created in subfolders of root (and content created in root itself), after initially syncing Dropbox and OC, is in sync immediately
• Content created in subfolders of root that have been there before initally syncing Drobox and OC (regarding the subfolders) appears after an unkown value of days (after 3 weeks I see it in sync now, but don't know how long it took actually).

So there might be some problem indexing folders that where created before there was a connection between OC and Dropbox. Maybe that helps someone working on this @icewind1991 @DeepDiver1975 @karlitschek?

[datapharmer]

Why was this issue closed? I am still having this problem in owncloud 7 with local external storage! We have entire folders that simply won't show up!

[te-online]

Same problem here – gave up external storage in combination with Dropbox for now because of this.
All my above described observations can still be reproduced (#377 (comment))

[spilss]

same here.. when connecting to FTP using External Storage, the new files uploaded to FTP does not show in OC 7.0. waited for a very long time and still no updates, even the deleted files are still there.

[datapharmer]

@spilss I rescanned the file system with /var/www/owncloud/console.php files:scan , which caused the files to appear again, though this is a workaround and not a fix. The issue seems to be permissions related, as I can reproduce it manually by changing the files/folders to not have permissions for the owncloud external storage user, clicking on the folder causes it to return to the home directory listing, change permissions back and owncloud will never show that file or folder again unless the manual filescan is performed. There may be more at play here (such as external server being temporarily unavailable and causing the same behavior) but that might at least expose the underlying problem.

[RobinMcCorkell]

@spilss @datapharmer @te-online The problem comes down to when ownCloud should consider its cache invalidated. Some (most?) external storage backends do not propagate modification times up the directory tree, so without a full rescan oC cannot determine that anything has changed. These problems are highlighted in #11797, so please move discussions there 😄.