Ideas to enhance current auto upload

[JonasMayerDev]

I would say there are 4 issues with this implementation currently:

1. Worker runs without auto upload activated when auto upload was activated before (issue probably existed before but was never visible).
2. I think the sync worker runs too often
3. The new foreground worker consumes too much energy
4. The notification is stuck sometimes

Originally posted by @JonasMayerDev in #12596 (comment)

[JonasMayerDev]

So Ideas to fix these issues would be to:

1. Make sync worker only check things changed on file system, not whole folder each time. Make a whole rescan of the folder only once per day instead of every time.
2. Try to get rid of the foreground worker (not ideal and requires the notification, so getting rid of it would be good practice and the notification can be hidden) by saving what was already scanned -> Worker could be killed by android and will pick up at the place where it quit.
3. Find bugs that make auto upload / file sync worker execute without any reason (e.g. when upload is only activated when device is plugged in / auto upload is deactivated)
4. If notification is needed give more feedback (show progress) else get rid of the notification.
   4.1 (More a feature then bugfix): To replace notification feedback, add information to auto uploads tab, when file system was last scanned successfully.

[JonasMayerDev]

1. Make sync worker only check things changed on file system, not whole folder each time. Make a whole rescan of the folder only once per day instead of every time.

Only partially works because it seems like executing a worker on file system changes only works for content uris (so only images and videos).

So this seems to be a good solution for images and videos, but other files can not be synced that way. A delay between full scans of 1 day is too much for some applications, so maybe it would make sense to make 1 day full sync delay the default and add the option to have a custom delay (let the user choose if they prioritize the sync or battery consumption)

[regs01]

1. Make sync worker only check things changed on file system, not whole folder each time. Make a whole rescan of the folder only once per day instead of every time.

But how is this whole rescan is working, so takes days. Does it check everything on server or does it calculate checksum on every file? Or is it just keep restarting over? As diskusage does whole disk scan in mere seconds.

[JonasMayerDev]

@regs01 it goes through every autoupload folder recursively and checks if the last modified timestamp is larger than the last scan (at least that's the behavior with the next release. Previously, it checked if the last modified timestamp is different to an saved modified timestamp in a local database). That's basically it. I am not sure if there is an better way on android but with the next release on my device it scanned 8000 Images in 60 sec. I would guess file explorers load those information only when needed to display them fast... The reason the notification suggest it sometimes takes ages to scan an directory is most likely the android background restriction that delay the work. If you know more about this toppic please share it. I am currently trying to optimize so every idea is welcome.

[regs01]

Still it takes a little long. Notification is up there for over 2 minutes, while DiskUsage does that folder in just a second. And in still experiencing hourly long usages in stand by.
https://github.com/IvanVolosyuk/diskusage

I also set up auto upload only via WiFi. But it keeps rescanning almost every unlock even if not connected to WiFi. So there is may be another thing to optimize. At least for those using WiFi only.

[JonasMayerDev]

@regs01 one factor why it takes so long could be that the background work not gets enough computing power as compared to an application in the foreground.

As far as I can tell, DiskUsage only shows size of folders and not scans that folder for all files. But if you have a concrete code snippet how to improve scanning in android, please share it. I have been working on it for multiple weeks now and always think about improvements, but everything I thought could work, didn't...

[jogerj]

More like an issue but I'll just throw it in here:

When configuring the remote folder for auto upload, it's impossible to select a hidden folder with '.' prefix (e.g. "/InstantUpload/.foo-folder"). It's possible to create one, but you can't create an already created (hidden) folder. There's no text input to just type in the path either.

In Web UI you have the option to just search the folder name to make it appear

Apparently the shown folders follows the "Show hidden files" options in the main settings.

[JonasMayerDev]

Solved as this issue is finally solved #12954

[Hannes-Schniz]

Id like to reopen the issue since the two way sync still consumes way too much battery. It drained 60% of my battery in a couple of hours.

[Siggi0904]

It is closed, but what I need is quite urgently a blacklist like in desktop version in the (android) app.
Thanks.