Need new button for complete re-scan-re-upload of folders selected for synchronization

[thsnielsen]

Steps to reproduce

1. While having folders checked for synchronization - unknown problems arise, and for some odd reason synchronization stops (as i have seen a number of times) then normal troubleshooting starts - are there blocked files on the server - delete them on server - these files are not re-uploaded once back to normal and next time the android client syncs the folder from where the picture or file came from.
   You may even chose to delete the account on the device, and recreate it - then no more files older that the date of the creation of the account are uploaded - no matter if the files are on the server or not
2. Normally synchronization picks up again if you manually tags a file for upload to server, but all new files arrived since the stop of sync until manual upload will not never synced.
   3.i am sure there are other scenarios, where files on the device are left un-uploaded

Expected behaviour

At next scan the non-uploaded files should be detected, and uploaded, but for the case where a file was purposely deleted via the web interface on the server, it creates a conflict - should this file be re-uploaded or not - rather not i would say. But sometimes you just wish to make sure, that the complete folder marked for sync on the phone IS on the %é_è-'çàçe(bloody) server.
Therefore the need for a "redo complete deep dive sync of folders marked for sync" "with/without overwrite" of existing files (the overwrite could be interesting in case of corruption on the server (sd card about to hang up, cryptolock etc)

Actual behaviour

not possible to get missing files onto the server once sync has decided to omit them, unless finding out yourself which files are missing, and manually tagging them for upload

Server configuration

Operating system:Raspbian 4.9.24-v7+ #993 SMP Wed Apr 26 18:01:23 BST 2017 armv7l GNU/Linux

Web server:
apt-cache show nginx
Package: nginx
Version: 1.10.3-1
Installed-Size: 91

Database:
Package: mariadb-server-core-10.1
Source: mariadb-10.1
Version: 10.1.22-3
Architecture: armhf

PHP version:
php -v
PHP 7.0.19-1 (cli) (built: May 11 2017 14:04:47) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.0.19-1, Copyright (c) 1999-2017, by Zend Technologies

Server version: (see your admin page)
12.0.0
Here connected remotely i discovered my USB storage is down, so i have to provide the rest of the details later, when i have cooled down my storage (bloody heatwave)- sorry about that

Calendar version: (see the apps page)
Unknown at this time

Updated from an older installed version or fresh install:
Upgraded from 11

Signing status (ownCloud/Nextcloud 9.0 and above):
Unknown at this time

Login as admin user into your cloud and access
http://example.com/index.php/settings/integrity/failed
paste the results here.
Will come to that ...

List of activated apps:

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your instance's installation folder

Nextcloud configuration:

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your instance's installation folder

or

Insert your config.php content here
Make sure to remove all sensitive content such as passwords. (e.g. database password, passwordsalt, secret, smtp password, …)

Are you using external storage, if yes which one: local/smb/sftp/...
External USB disks used but mounted local (and of course now that i write on this issue, the disk cabinet overheated, and the whole nextcloud storage is down :-/

Are you using encryption: yes/no
Yes

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
No

LDAP configuration (delete this part if not used)

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your instance's installation folder


### Client configuration
Android client 1.4.3

### Logs
#### Web server error log
I guess again no error  because it is working as designed

#### Log file (data/nextcloud.log)
Nothing is synced i guess no relevance for server log - otherwise please ask, and i will provide for the desired scenario

#### Browser log
No log from Android client at hand

[tobiasKaminsky]

Best would be to have auto upload working reliable and not trying to circumvent the problem by adding a manual sync button.

@mario?

[thsnielsen]

I Have high hopes for the 1.5 client, but i see it has declined in percentage completion as if more objectives had been added - is there a target date for the new android client - with the usual precautions regarding statements for the future ;-)

[mario]

@thsnielsen august for beta.

[thsnielsen]

great, i will be 1st to sign up :-)

[mario]

There will never be a "manual auto upload button" however :D But I'll have an APK for you in the next few days to test...

[mario]

Dear users,

thank you for your patience. After lots of work, we’re super excited to be able to give you a test version with new auto upload implementation. Keep in mind we will only accept bugs related to Auto Upload in this APK, as other things might be broken.

https://cloud.nextcloud.com/s/ftZraTruRQjTe0R

This is a dev APK so it will not overwrite your existing play store/f-droid APK unless you’re already running dev version. Also, you might need to remove your existing dev version before installing this one.

Explanation of the current functionality:

• added support for custom folders
• separated image and video folders
• checks for new files happens every 15-20 minutes
• automatic retry every 15-20 minutes or when you force close/open the app
• automatic pause once network is changed (untested!)
• significantly less battery usage

All and any testing is more than appreciated, especially with large and ever-going files like Videos. (like, try to record a 30 min video).

Let me know if you’ve got any questions.

Cheers,
Mario

[mario]

I have added force rescan to the uploads view when in expert mode. It's not in APK yet, but it's there :)

It's basically just a one-time speed up of the "find new files to upload" procedure.

[pieter-groeneweg]

I have the 2.0.0RC6 from beta channel.
I enabled expert mode to have "force rescan".
When clicking "force rescan" it does not appear to do anything... Nothing shows in "Uploads" as "new uploads" or "pending" uploads.
You write "find new files to upload"... though it is files that are not new I need to have synced as they where missed previously by the app. (1.4.3)

[mario]

This is not supported yet

…

On Tue, 12 Sep 2017 at 13:03, pieter-groeneweg ***@***.***> wrote: I have the 2.0.0RC6 from beta channel. I enabled expert mode to have "force rescan". When clicking "force rescan" it does not appear to do anything... Nothing shows in "Uploads" as "new uploads" or "pending" uploads. You write "find new files to upload"... though it is files that are not new I need to have synced as they where missed previously by the app. (1.4.3) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1150 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAWssbpboxtGHNt4pcTigW3dFlQfEtrks5shmUGgaJpZM4OH7ie> .

[thsnielsen]

Hello - thanks for v3 Now autoupload is working on my phone :-) (i still have an odd error briefly popping up hwhen i take a picture; but the photos transfer - BUT on my wifes phone it is not autouploading :-( i would like to help troubleshooting with logs etc. Please tell me what /howto log fear not i am used to it.
Here some startup info - my wifes phone is an Huawei p9 (EVA-L09) updated with the latest original android with no network provider overhead (free, orange, sfr, bouygues, non of that). Android 7.0 (Huawei EMUI 5.0.1) Subscription is with Free, that has a 40-50gb data limit pr month, and of course she is on wifi at home.
The server is a raspberry 3 well updated, nextcloud 13 :-) with nginx, which for documents works fine. On my wifes phone I have only one folder (camera) set to autosync in the long list of proposed items to autosync. Newly taken photos DO show up here.
I have changed the remote folder to upload to to /NosPhotos. My phone also uploads to this folder.
Wifi only is unchecked. Only if on charger ( i have french menus, so the workding do not correspond to the english menus) is unchecked, and use subfolders IS checked.
Worth noticing is that there are 2113 photos in the camera folder.
In the "activities menu, i see on my wifes phone, when i take a photo with MY phone that is synced up, and i also see in the activities on my wifes phone when i delete a photo in the /NosPhotos folder.
btw while autoupload has not worked on my phone up to now, a lot of photos are not synced, and i would still like a re-scan button to have all the non-synced photos uploaded. Please do not hesitate me to provide troubleshooting info.
Thanks for the good work so far.

[mario]

Huawei is special. Can you please set it as protected/disable battery optimizations for nextcloud?

[thsnielsen]

it took a few days to get screen time with my wife's phone, but allowing background execution under battery settings made the autoupload of photos work. Thanks a lot!

[tobiasKaminsky]

Do we want to show sime kind of info on these phones @mario?
Similar is done by davdroid...

[thsnielsen]

For your information. Allowing nextcloud to execute background also caught up with previously not sent photos.
Note that i had expert mode on, and under history of uploads had pressed "actualiser" (french version) at least once - not sure if it matters. But all the photos are arrived on the server - thank you :-)
Now onto the wife next quest - have some bot autotag the photos. I will google a bit, and if not finding anything, make a feature request later. But for now, it looks like this one has been resolved.

[thsnielsen]

oups i hit the close and comment - not sure if it correct according to your procedures. Please tell me if i did wrong for future.